Re: [Vim-l] Unicode issue

On 13/12/11 20:32, Benjamin Fritz wrote:
> First, you've sent your question to a list discussing the pages on the
> wiki, not a Vim help list. You should have sent it to
> vim_use@googlegroups.com, as discussed here:
>
> http://vim.wikia.com/wiki/Vim_Tips_Wiki:Community_Portal#Asking_questions
>
> I've included this list on my reply, you should respond there if you
> have any additional problems.
>
> On Tue, Dec 13, 2011 at 10:09 AM, Edward Garbowski<exg@sabinmetal.com> wrote:
>> I am having an issue with opening a text file with no BOM. I have a
>> text file that opens fine in Notepad, but if I open it in VIM I just
>> get garbage.
>>
>> If I look at the hex code, there is no BOM in the beginning. If I put
>> a BOM in, it works.
>>
>> How can I make VIM work with no BOM on reading a file? Or is this a
>> bug of some kind?
>>
>> I am using version 7.3.46
>>
>
> It sounds like your 'fileencodings' option is not set correctly.
> Probably it contains "ucs-bom" but not "utf-8".
>
> This is probably because either you've deliberately edited your
> 'fileencodings' option, or you've never set Vim up to use a multibyte
> 'encoding' option in the first place.
>
> See http://vim.wikia.com/wiki/Working_with_Unicode and the help
> references listed near the bottom of that tip, to get started.
>

Another possibility would be a file in UTF-16le with no BOM. If it
contains Latin text, Vim (with the standard 'fileencodings') would
display it with most characters followed by a blue ^@ (l^@i^@k^@e^@
^@t^@h^@i^@s^@).

To read a UTF-16 file with no BOM (or any file whose charset, known to
you, is not correctly detected by the 'fileencodings' heuristics), I
recommend telling Vim the encoding to use:

:e ++enc=utf-16le foobar.baz

see :help ++opt

The values recognized for the BOM are as follows (in hex):

UTF-8 EF BB BF
UTF-16be FE FF
UTF-16le FF FE
UTF-32be 00 00 FE FF
UTF-32le FF FE 00 00
UTF-32-3412 FE FF 00 00
UTF-32-2143 00 00 FF FE

I'm not sure Vim knows about the latter two (which I've seen listed
online, I don't remember at the moment if it was on the Unicode site,
the ISO site or the W3C site). A consequence of the above is that the
first codepoint in a UTF-16 file (of either endianness) must not be a null.

Best regards,
Tony.
--
Schizophrenia beats being alone.

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php