Re: Opening large files?

Hi Bram,

2014/4/29 Tue 22:04:23 UTC+9 Bram Moolenaar wrote:
> Ken Takata wrote:
>
> > 2014/4/17 Thu 0:58:01 UTC+9 Adrian wrote:
> > > 'm helping another user open a large, 3Gb, file. The standard windows
> > > editors balk, so I recommended VIM. Unfortunately, even vim crashes
> > > after scrolling some amount. For instance, he can't go straight to
> > > the end of file.
> > >
> > > The work station is Windows 7, 64 bit, with 32Gb of RAM. Are there
> > > any settings to modify to make vim more stable with large files, or is
> > > there some Windows performance limitation and just out of luck?
> >
> > There is a related item in the todo.txt:
> >
> > | Win64: Seek error in swap file for a very big file (3 Gbyte). Check storing
> > | pointer in long and seek offset in 64 bit var.
> >
> > I wrote some patches to fix this, but they seem to be still unstable.
> >
> > https://bitbucket.org/k_takata/vim-ktakata-mq/src/192069dac4356c186b89e0451a254599713d2309/support-largefiles-on-windows.patch?at=default
> > https://bitbucket.org/k_takata/vim-ktakata-mq/src/192069dac4356c186b89e0451a254599713d2309/use-stat_T.patch?at=default
>
> Did you make progress on this?

I wrote "still unstable", but it seems that it was my mistake.
Now I think that the patches are OK.

Sometimes Vim (without the patches) freezes when I open a very big file
(about 2 GB) and scroll up and down using scroll bar. After applying the
patches, Vim sometimes takes very long time for scrolling up and down, but
it wasn't a freeze. (I misunderstood that.)

BTW, I found that 32-bit Vim couldn't handle a very big file properly when
":set noswapfile". In my understanding, this is an expected(?) behavior
because Vim tries to load the whole file into the memory when 'swapfile' is
off, and a 32-bit program can't allocate larger than 2-GiB memory.
(Actually, a 32-bit program can get 3-GiB user space if /LARGEADDRESSAWARE
option is specified for 'link'.)


> Can we also add some tests to verify the fix?

I'm thinking what is the best way to test this.
Something like this?

" Make sure that a line break is 1 byte.
:set ff=unix
:set undolevels=-1
" Input 99 'A's. The line becomes 100 bytes including a line break.
99iA<Esc>
yy
" Put 19,999,999 times. The file becomes 2,000,000,000 bytes.
19999999p
" Moving around in the file randomly.
G
10%
90%
50%
gg
...
" Edit some lines.
...
" Extract some lines and write them to test.out.
...

Regards,
Ken Takata

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.