Re: ccomplete#Complete very slow on large tags file

On Wed, Apr 1, 2020 at 8:02 PM Pratyush Yadav <me@yadavpratyush.com> wrote:
>
> Hi Tony,
>
> On 01/04/20 07:26PM, Tony Mechelynck wrote:
> > On Wed, Apr 1, 2020 at 7:15 PM Pratyush Yadav <me@yadavpratyush.com> wrote:
> > >
> > > Hi,
> > >
> > > I am using the built-in ccomplete#Complete omnifunc for tags-based
> > > completion. It works pretty well with a small or medium sized project.
> > > But on larger projects like Linux or U-Boot, the tags file is pretty big
> > > (~650k lines/70 for U-Boot and ~4.8 mil lines/630MB for Linux).
> > >
> > > On these projects, especially Linux, if I use ccomplete, completion
> > > takes a huge amount of time. And since the completion is synchronous,
> > > vim just freezes up for that time and I have to wait for the completion
> > > results to show up. On Linux the wait times can go more than a minute on
> > > my modest quad core 4th gen i7 and a SATA SSD.
> > >
> > > So, I intend to set aside some time and try to speed up the completion
> > > and make it asynchronous. I haven't read or written much vimscript so I
> > > don't expect it to be easy or quick. So before I do that, I'd like to
> > > know if anyone here attempted this before.
> > >
> > > If someone did succeed, could you please point me to the optimized
> > > version? If someone tried but failed, could you please let me know where
> > > you had trouble? Is it that the ctags format just not suitable for large
> > > projects? Is there some limitation with vim that makes it impossible? Is
> > > it something else entirely?
> > >
> > > --
> > > Regards,
> > > Pratyush Yadav
> >
> > The Vim tags file is pretty large, but how is your tags file built? In
> > particular, is it case-fold-sorted? If your Vim is built with
> > +tag_binary (nowadays it always is, if yours isn't you need a newer
> > model), and 'tagbsearch' is on (which is the default), then finding a
> > tag in an unsorted tags file uses linear time, in a case-folded sorted
> > tags file it uses logarithmic time, which is much faster, especially
> > for large tags files.
>
> Yes, the tags file is sorted and vim is built with +tag_binary and
> 'tagbsearch' is on. The tags file wasn't case-folded sorted, but I tried
> with that too and I see similar performance.
>
> Anyway, just to remove any potential confusion, jumping to tags via
> 'CTRL-]' is blazing fast. So is listing all alternatives with 'g]'. Does
> ccomplete use the same mechanism for searching tags file as those two?
> If so, then I wonder why it is so much slower.
>
> Also if I have a struct foo, doing something like 'foo->' takes a huge
> amount of time because I think there is no other way to get a list of
> members of a struct without a linear search, but that is a different
> problem entirely.
>
> --
> Regards,
> Pratyush Yadav

Hm, check the help for 'complete' and for the following completion
options (in the current helpfile, options.txt dated 2020 Mar 02, there
are 127 lines for them all). Maybe something will ring a bell with
you.

Best regards,
Tony.

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vim_use/CAJkCKXuSvSNjo3AtWb6MJT1zexLjSYr1yyGWHxPQThmuGHJ1TA%40mail.gmail.com.