Re: Vim surprisingly slow?

On Do, 16 Nov 2017, Dominique Pellé wrote:

> Dominique Pellé <dominique.pelle@gmail.com> wrote:
>
> > Christian Brabandt <cblists@256bit.org> wrote:
> >
> >> On Mi, 15 Nov 2017, Dominique Pellé wrote:
> >>
> >> > As highlighted in red, notice that we only write 179,281 bytes to file
> >> > "20150425.txt" but we open/write/close/fsync 5004 times.
> >> > That repeated I/O patterns kills performance, especially because
> >> > of the frequent fsync().
> >> >
> >> > We also read 1,612,897 bytes from data.json, but that should be
> >> > fast since we do only 27 read and opened that file only twice
> >> > according to output of io.pl. Vim reads file deta.json by chunks
> >> > of 64 KB. 1,612,897 bytes is exactly the file size of data.json.
> >> >
> >> > Maybe we can change vim to avoid the constant open/write/close/fsync
> >> > on file "20150425.txt" which should speed it up greatly.
> >>
> >> Thanks for this nice analysis. It is probably the fsync call that is
> >> slowing Vim down. We can avoid the fsync by disabling the 'fsync'
> >> option. Also we might want to disable the writebackup option just in
> >> case it matters. And finally, when a file is written in a different
> >> encoding, Vim will try to check if conversion is safe by first trying to
> >> convert the buffer contents to the fileencoding before writing the file.
> >> That might make also a difference when encoding differs from the file
> >> encoding.
> >>
> >> Perhaps it makes sense to skip fsyncing while being busy in a `:g`
> >> command. Tim, does this patch make a difference?
> >>
> >> diff --git a/src/fileio.c b/src/fileio.c
> >> index d991822c3..eb626a675 100644
> >> --- a/src/fileio.c
> >> +++ b/src/fileio.c
> >> @@ -4731,7 +4731,7 @@ restore_backup:
> >> * work (could be a pipe).
> >> * If the 'fsync' option is FALSE, don't fsync(). Useful for laptops.
> >> */
> >> - if (p_fs && fsync(fd) != 0 && !device)
> >> + if (p_fs && !global_busy && fsync(fd) != 0 && !device)
> >> {
> >> errmsg = (char_u *)_("E667: Fsync failed");
> >> end = 0;
> >
> >
> > Hi Christian
> >
> > I measured the same command twice before and after your patch:
> >
> > Before patch:
> >
> > $ time ./vim -u NONE data.json -S new_engine.vim
> > real 0m15.533s
> > user 0m1.648s
> > sys 0m0.824s
> >
> > $ time ./vim -u NONE data.json -S new_engine.vim
> >
> > real 0m15.444s
> > user 0m1.636s
> > sys 0m0.820s
> >
> > After patch:
> >
> > $ time ./vim -u NONE data.json -S new_engine.vim
> >
> > real 0m0.741s
> > user 0m0.536s
> > sys 0m0.200s
> >
> > $ time ./vim -u NONE data.json -S new_engine.vim
> >
> > real 0m0.731s
> > user 0m0.548s
> > sys 0m0.184s
> >
> > So your patch makes it more than 20 times faster
> > on my Linux machine. Nice! Hopefully it's safe to
> > avoid fsync() in this case.
>
> I have not had the time to look in details, but the
> patch is wrong. After patch, running the command with
> strace, I do not see any fsync() calls anymore. I thought
> that the idea of the patch was to avoid intermediate fsync()
> but still fsync() at the end of the command. But since
> there is no fsync() call anymore, it's not safe.

Yeah, the patch skips fsyncing completly for :g commands. However, I
think this might be acceptable for several reasons:

First, if you disable the 'fsync' option (or HAVE_FSYNC is not defined),
Vim works perfectly fine. Also fsync() is not used for the writfile()
function, so it seems to be acceptable)

Second, when using `:g` it is more likely that only a small part of the
buffer will be written several times in relatively short times. So it
does not make sense to have to wait for the filesystem until the write
was complete and has reached the disk. This only causes many unnecessary
waits.

However I would agree, that a final fsync() at the end might be nice.
However we don't really know, when this is the case and even if we did
which is the last run of the `:g` command, it wouldn't be safe, because
perhaps in the last iteration nothing would be written to disk.

On the other hand, we can not fsync() at the end of the `:g` command,
because at that time, we would not have a filehandle to fsync().

I suppose skipping the fsync() would be okay in this edge case for :g
command and trust, that the kernel will write the changes to disk. The
only problem would be a system crash or power failure but I believe this
is relatively unlikely.

Christian
--
Der Philosoph hat wie der Hausbesitzer immer Reparaturen.
-- Wilhelm Busch

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.