Re: ex large file

Enda wrote:

> 'echo "" > filetowipe' would be a new file, I want to edit the same file.
>
> I don't want to crash my desktop again. I think the fact that ex creates
> .swp and .swo files might contribute to the problem.
>
> - Enda


Somehow %d (same 1,$d) is using a lot of memory
with big files.

This is what I did:

1) Create a 150 Mb file (10 M lines) with:
$ yes 'this is a test' | sed 10000000q > test.txt

2) Start vim with: vim -u NONE test.txt
On my PC, this is using 202 Mb of virtual memory

3) :%d
Virtual memory usage jumps to 729 Mb

My suspicion was that undo needs lots of memory.
Doing the same steps with :set ul=1 | %d
hence disabling undo saves a lot of memory but
memory usage still jumps from 202 Mb to 462Mb
when replace step 3) with:

:set ul=-1 | %d

I wonder what uses so much memory when undo
is disabled.

I also profiled memory usage with...

$ valgrind --tool=massif ./vim -u NONE

... and it gives the following usage at the peak of memory use
(using a 75 Mb test.txt file):

--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)

21 6,382,232,109 237,533,520 192,136,937 45,396,583 0
80.89% (192,136,937B) (heap allocation functions) malloc/new/new[],
--alloc-fns, etc.
->80.79% (191,902,967B) 0x810FE7B: lalloc (misc2.c:929)
| ->72.35% (171,861,290B) 0x810FD98: alloc (misc2.c:828)
| | ->40.45% (96,088,064B) 0x81F6471: mf_alloc_bhdr (memfile.c:951)
| | | ->40.45% (96,088,064B) 0x81F5B1B: mf_new (memfile.c:392)
| | | ->40.29% (95,703,040B) 0x80F1F14: ml_new_data (memline.c:3503)
| | | | ->40.29% (95,698,944B) 0x80F08C6: ml_append_int (memline.c:2751)
| | | | | ->40.29% (95,698,944B) 0x80F03E6: ml_append (memline.c:2525)
| | | | | ->40.29% (95,698,944B) 0x80C4A2D: readfile (fileio.c:2219)
| | | | | ->40.29% (95,698,944B) 0x804E0D8: open_buffer (buffer.c:140)
| | | | | ->40.29% (95,698,944B) 0x81EF6DB: create_windows (main.c:2632)
| | | | | ->40.29% (95,698,944B) 0x81ED22D: main (main.c:833)
| | | | |
| | | | ->00.00% (4,096B) in 1+ places, all below ms_print's
threshold (01.00%)
| | | |
| | | ->00.16% (385,024B) in 1+ places, all below ms_print's
threshold (01.00%)
| | |
| | ->31.58% (75,001,587B) 0x8110278: vim_strsave (misc2.c:1253)
| | | ->31.57% (74,998,440B) 0x8132825: op_yank (ops.c:2985)
| | | | ->31.57% (74,998,440B) 0x8130043: op_delete (ops.c:1728)
| | | | ->31.57% (74,998,440B) 0x80AF4F0: ex_operators (ex_docmd.c:8500)
| | | | ->31.57% (74,998,440B) 0x80A68CC: do_one_cmd (ex_docmd.c:2668)
| | | | ->31.57% (74,998,440B) 0x80A4194: do_cmdline (ex_docmd.c:1122)
| | | | ->31.57% (74,998,440B) 0x812710A: nv_colon (normal.c:5412)
| | | | ->31.57% (74,998,440B) 0x812070B: normal_cmd (normal.c:1193)
| | | | ->31.57% (74,998,440B) 0x81EDBA3: main_loop (main.c:1294)
| | | | ->31.57% (74,998,440B) 0x81ED5C0: main (main.c:998)

I'll try to investigate more later.

-- Dominique

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php