Re: Limitations of vim-python interface with respect to character encodings

On Fri, Jul 16, 2010 at 11:45 AM, Ted <cecinemapasderange@gmail.com> wrote:

I've recently discovered that there are some limitations with respect
to using the python interface to bring non-ascii characters back into
vim.

For example, this command works as expected:

   :py vim.command(u'echo "\u0061"')
   a

However this one gives the following error message:

   :py vim.command(u'echo "\u2026"')
   Traceback (most recent call last):
     File "<string>", line 1, in <module>
   UnicodeEncodeError: 'ascii' codec can't encode characters in
position 10-11: ordinal not in range(128)

Other functions seem to be even more restricted:

   :py vim.current.range.append(u'\u0061')
   Traceback (most recent call last):
     File "<string>", line 1, in <module>
   TypeError: bad argument type for built-in operation

Even output from python is restricted:

:py print u'echo "\u0061"'
echo "a"
:py print u'echo "\u2026"'
Traceback (most recent call last):
 File "<string>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2026' in
position 6: ordinal not in range(128)

I've discovered that this last issue can be remedied via

   import sys, codecs
   sys.stdout, _held = (codecs.getwriter('utf-8')(sys.stdout),
sys.stdout)

Setting python's default encoding from within /etc/python2.6/
sitecustomize.py has a similar effect.

These last two workarounds only fix issues with printing; issues with
appending to buffers are not affected by either of these changes.

Is it possible to fix this by reconfiguring something within Python,
or is this a limitation of the vim-python interface?

Actually, for my script, if i want to use the unicode string into vim environment.
# my setting " set encoding=utf8"
I will always use the encode && decode to transfer the string into vim internal encoding, and then pass it to the vim function.
for example:

python << EOS
uniStr = u"\u2026"
str = uniStr.encode( vim.eval("&encoding") )
print str
EOS

OR, as regards to your example, it would be :
:py print u"\u2026".encode( vim.eval("&encoding") )


and for another example of yours:
:py vim.command(u'echo "\u2026"')
you should not use the "u" to modify the string pass to vim, you'd better to use the vim internal encoding string instead of the unicode string directly, it will be OK if you run it like this ( just remove the "u" decorator )
:py vim.command( 'echo "\u2026"')

OR use it like i suggested:
py vim.command(u'echo "\u2026"'.encode( vim.eval("&encoding") ) )
 

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php