Re: Vim surprisingly slow?

Nikolay Pavlov wrote:

> 2017-11-16 8:43 GMT+03:00 Dominique Pellé <dominique.pelle@gmail.com>:
> > Dominique Pellé <dominique.pelle@gmail.com> wrote:
> >
> >> Christian Brabandt <cblists@256bit.org> wrote:
> >>
> >>> On Mi, 15 Nov 2017, Dominique Pellé wrote:
> >>>
> >>> > As highlighted in red, notice that we only write 179,281 bytes to file
> >>> > "20150425.txt" but we open/write/close/fsync 5004 times.
> >>> > That repeated I/O patterns kills performance, especially because
> >>> > of the frequent fsync().
> >>> >
> >>> > We also read 1,612,897 bytes from data.json, but that should be
> >>> > fast since we do only 27 read and opened that file only twice
> >>> > according to output of io.pl. Vim reads file deta.json by chunks
> >>> > of 64 KB. 1,612,897 bytes is exactly the file size of data.json.
> >>> >
> >>> > Maybe we can change vim to avoid the constant open/write/close/fsync
> >>> > on file "20150425.txt" which should speed it up greatly.
> >>>
> >>> Thanks for this nice analysis. It is probably the fsync call that is
> >>> slowing Vim down. We can avoid the fsync by disabling the 'fsync'
> >>> option. Also we might want to disable the writebackup option just in
> >>> case it matters. And finally, when a file is written in a different
> >>> encoding, Vim will try to check if conversion is safe by first trying to
> >>> convert the buffer contents to the fileencoding before writing the file.
> >>> That might make also a difference when encoding differs from the file
> >>> encoding.
> >>>
> >>> Perhaps it makes sense to skip fsyncing while being busy in a `:g`
> >>> command. Tim, does this patch make a difference?
> >>>
> >>> diff --git a/src/fileio.c b/src/fileio.c
> >>> index d991822c3..eb626a675 100644
> >>> --- a/src/fileio.c
> >>> +++ b/src/fileio.c
> >>> @@ -4731,7 +4731,7 @@ restore_backup:
> >>> * work (could be a pipe).
> >>> * If the 'fsync' option is FALSE, don't fsync(). Useful for laptops.
> >>> */
> >>> - if (p_fs && fsync(fd) != 0 && !device)
> >>> + if (p_fs && !global_busy && fsync(fd) != 0 && !device)
> >>> {
> >>> errmsg = (char_u *)_("E667: Fsync failed");
> >>> end = 0;
> >>
> >>
> >> Hi Christian
> >>
> >> I measured the same command twice before and after your patch:
> >>
> >> Before patch:
> >>
> >> $ time ./vim -u NONE data.json -S new_engine.vim
> >> real 0m15.533s
> >> user 0m1.648s
> >> sys 0m0.824s
> >>
> >> $ time ./vim -u NONE data.json -S new_engine.vim
> >>
> >> real 0m15.444s
> >> user 0m1.636s
> >> sys 0m0.820s
> >>
> >> After patch:
> >>
> >> $ time ./vim -u NONE data.json -S new_engine.vim
> >>
> >> real 0m0.741s
> >> user 0m0.536s
> >> sys 0m0.200s
> >>
> >> $ time ./vim -u NONE data.json -S new_engine.vim
> >>
> >> real 0m0.731s
> >> user 0m0.548s
> >> sys 0m0.184s
> >>
> >> So your patch makes it more than 20 times faster
> >> on my Linux machine. Nice! Hopefully it's safe to
> >> avoid fsync() in this case.
> >
> > I have not had the time to look in details, but the
> > patch is wrong. After patch, running the command with
> > strace, I do not see any fsync() calls anymore. I thought
> > that the idea of the patch was to avoid intermediate fsync()
> > but still fsync() at the end of the command. But since
>
> No, this is hard (impossible if you consider that command may modify
> following text so that it will not match) to determine in the middle
> of the global command whether it is going to run once more, so the
> patch which intends to do something like this will have to do
> something like caching file descriptors and emptying cache at the end,
> additionally calling fsync().

Assuming the script knows what the last append is going to be, it can
set 'fsync' off for all but the last one. Or write an extra (empty)
line at the end with 'fsync' set.

> > there is no fsync() call anymore, it's not safe.

Correct, you may lose your work on a system crash.

--
While it's true that many normal people whould prefer not to _date_ an
engineer, most normal people harbor an intense desire to _mate_ with them,
thus producing engineerlike children who will have high-paying jobs long
before losing their virginity.
(Scott Adams - The Dilbert principle)

/// Bram Moolenaar -- Bram@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.