Sunday, May 28, 2017

Re: Bug/non-determinism in output of maparg() and map commands

2017-05-28 23:16 GMT+03:00 Brett Stahlman <brettstahlman@gmail.com>:
> On Sun, May 28, 2017 at 1:10 PM, Nikolay Aleksandrovich Pavlov
> <zyx.vim@gmail.com> wrote:
>
> [...]
>>>>
>>>> You may also ask Brett Stahlman whether my proposal is enough, it does
>>>> not look like he thinks it is not, just that it may be less
>>>> convenient.
>>>
>>> I believe it would be possible to do pretty much anything with the
>>> functions you propose, though for some common use cases, it puts more of
>>> a burden on the user than may be warranted. Yes,
>>> some_more_complex_filter_if_needed could be a function that tests
>>> canonical lhs for conflict/ambiguity, retaining only the ambiguous maps,
>>> and a subsequent call to mappings_load() (passing the filtered results
>>> with 'rhs' key removed) could be used to delete the conflicting
>>> mappings. But this feels a bit roundabout and error-prone compared to...
>>>
>>> let handles = maplist('<F8>', 'nv')
>>> call disable_maps(handles)
>>>
>>> Question: Would the map handles be mode-specific? E.g., for a map
>>> defined with :map, would there be 1 or 3 handles? I'm assuming that your
>>> mappings_dump() would return a single dict with a modes flag of 'nvo' or
>>> something like that. It occurs to me that either approach could allow
>>> finer-grained control than the :*map commands provide, and could obviate
>>> the need for kludges like this (from the help on omap-info):
>>>
>>> {{{
>>> To enter a mapping for Normal and Visual mode, but not Operator-pending mode,
>>> first define it for all three modes, then unmap it for Operator-pending mode:
>>> :map xx something-difficult
>>> :ounmap xx
>>> Likewise for a mapping for Visual and Operator-pending mode or Normal and
>>> Operator-pending mode.
>>> }}}
>>>
>>> Also note that your approach does more than simply *disable*; it
>>> actually *removes* the mappings as far as Vim is concerned. Granted, if
>>> we keep a reference to disabled_mappings, it would be easy enough to
>>> recreate them later, but I would imagine this is significantly less
>>> efficient than simply toggling an internal flag indicating enabled
>>> status. Also, as I mentioned before, I don't think checking for
>>> ambiguity/conflict is something that *should* be done by a plugin
>>> developer. Sure, a plugin developer should be capable of writing an
>>> algorithm to do this (assuming he has access to a canonical lhs), but
>>> forcing each plugin developer to do it just feels wrong to me, even if
>>> it ends up being a smallish predicate function/filter (and I suspect
>>> doing it robustly would end up being more than a one-liner). Thus, at
>>> the very least, I would propose having mappings_dump() accept an input
>>> that requests ambiguous/conflicting maps only, or else a separate
>>> function such as mappings_get_conflicts().
>>
>> This is *not* as simple as "toggling the internal flag". You want to
>> disable a mapping and use your own one instead, this means that there
>> will be other changes to the mapping code. Much likely the result will
>> be *less* efficient for a few first years: your plugin has one set of
>> users, but for features needed for your plugin *all* users will pay.
>> This is much less the case for my variant: additional functions also
>> cost space and CPU ticks for O(log N) function lookup (BTW, Neovim has
>> O(1) built-in function lookup), but this is all non-users will pay.
>
> There's no relationship between the old and new mapping. Disabling the
> old (which may not even have the same lhs as the new) simply allows
> Vim to find the new mapping without ambiguity or conflict. A disabled
> mapping would be shadowed by a <buffer> mapping in much the same way
> as a global mapping is shadowed by a buf-local mapping: i.e., Vim sees
> that the global would work, but prefers the buf-local. Depending on
> how they're stored, Vim may still have to iterate over the disabled
> mappings, but the disabled flag prevents their being considered as
> candidates.

That "still have to iterate" is what I am talking about.

>
>>
>> Though I would not talk about efficiency here: the question is whether
>> your plugin implemented on top of one feature or the other will switch
>> "modes" in ~50 ms I guess. If it can do this with my approach then my
>> approach is good enough. Also I have shared details regarding how my
>> proposed API will look like, but I have not seen detailed proposal
>> which uses handles.
>>
>>>
>>> As I see it, your approach gives the plugin developer access to all map
>>> information, and lets him do whatever he wants with it, provided he's
>>> willing to do the work of slicing and dicing the data using Vim's list
>>> and dict manipulation functions. By providing all relevant data, while
>>> making few, if any, assumptions about the things a plugin developer
>>> might want to do with that data, you make nothing particularly easy, but
>>> everything possible. This is not wrong. It's simply a lower-level
>>> approach than what I was originally envisioning. Your argument is
>>> essentially that it's better to re-use existing data structures, in
>>> conjunction with sequence manipulation primitives like map() and
>>> filter(), than to invent a DSL whose purpose is to reduce tedious
>>> boilerplate as well as the potential for error associated with the most
>>> common map-related use cases. The advantage, I suppose, is that your
>>> approach is probably less work for Bram, fewer bugs for him to work
>>> through, and ultimately pretty flexible for a competent plugin developer
>>> who doesn't mind a litle extra data massaging. Whereas creating a DSL
>>> forces you (hopefully only once) to think through the likely use cases
>>> and make decisions about the sorts of things that *should* be done with
>>> the DSL, your approach leaves such decisions to each and every plugin
>>> developer: very flexible, but at the cost of forcing each plugin
>>> developer to reinvent the wheel for the most common cases. Ideal, in my
>>> opinion, would be a DSL that facilitated the common cases, without
>>> precluding resort to the brute-force, low-level primitives for those use
>>> cases not envisioned by the DSL designers...
>>>
>>> Though I'm not set on the "handle" abstraction, I do think the fact that
>>> Bram and I were independently thinking about the insulation it would
>>> provide from implementation details is probably an indication that the
>>> advantages are at least worth considering. (OTOH, I can also see a
>>> certain attractive simplicity in the use of maparg-style dictionaries as
>>> "handles".) I do think the prolog/epilog callbacks I mentioned in an
>>> earlier post would be a powerful mechanism for plugin developers. (Of
>>> course, it should be possible to implement these with your approach as
>>> well.)
>>
>> They may be emulated using various techniques. Better if there would
>> be standard (distributed with Vim) autoload-based "plugin" for adding
>> callbacks or out of two different plugins using this feature one would
>> not see "original" mapping.
>>
>> I would say though that attaching callbacks to mappings could be done
>> without plugin handles and especially without enabling/disabling
>> functionality. Without them you would need mappings_canonicalize() to
>> have a robust way to tell what you need to attach callbacks to, but in
>> any case having canonical form is a good idea also for obtaining
>> handles. So I would suggest to not take this feature into account when
>> deciding whether handles are needed.
>
> Agreed. Efficiency is not the dominant consideration here...
>
>>
>>>
>>> Idea: It would be even more powerful if there were a way to attach the
>>> callbacks to a *partial* map sequence, or alternatively, a way to
>>> "monitor" an "in-progress" map sequence that hasn't yet triggered: e.g.,
>>> a callback function that fires each time a key is pressed, which
>>> receives not only the last key pressed, but also the entire sequence
>>> entered thus far (and possibly even the time remaining on the map
>>> timeout). The default behavior would be to allow the sequence entry to
>>> continue normally, but the callback function would also have the ability
>>> to change the sequence currently recognized to something completely
>>> different: i.e., the callback would be able to manipulate the typeahead
>>> buffer. I'm thinking this sort of capability could be useful for the
>>> handling of `->' in the transliteration mode you were describing in an
>>> earlier email. In my own case, I'm wanting to implement a temporary
>>> "escape" from the special mode I described earlier: though I'm
>>> overriding builtin maps in special mode, I'd like to allow the user to
>>> execute a builtin (without getting out of special mode) by prefixing it
>>> with some sort of escape (e.g., `\', or `,'). I can achieve something
>>> like what I want by defining maps for things like `\d', and using
>>> feedkeys() or normal! in the map handler to execute builtin d, but this
>>> forces me to define a bunch of maps I don't really need. It would be
>>> nice if I could monitor the progress of a map as it's entered and simply
>>> "lie" to Vim about what's been entered when the callback logic decides
>>> it's appropriate. At any rate, I've digressed from the original topic a
>>> bit...
>>
>> Callbacks on partial mappings is a thing which statusline plugins'
>> authors are going to like, but this also needs neither handles nor
>> enable/disable functionality to work.
>>
>>>
>>>>
>>>> ---
>>>>
>>>> And why do you think that "do not make plugins reinwent the wheel" is
>>>> the same statement as "write needed functionality in C code"? You can
>>>> always add a new file to `autoload`, writing VimL code is easier then
>>>> writing C code.
>>>
>>> You mean implementing a sort of "DSL" layer (using the primitive
>>> functions you've proposed) in Vim script, and including it in the
>>> official distribution as an autoload file? If so, I suppose this could
>>> be done, but as part of the official distribution, it would still need
>>> to be thought through and tested pretty thoroughly, so I don't see much
>>> advantage over implementing it in source, where it would be
>>> significantly more efficient. But perhaps I misunderstood what you were
>>> suggesting...
>>
>> I think you have only written VimL code, but not C one. VimL does not have
>> - Memory leaks. You can provoke one, of course, but 90% of code will
>> not ever cause any memory leaks unless you intentionally wrote a code
>> which causes memory leak (e.g. an always-growing cache of something
>> which is never cleaned up).
>> - Using some resource after freeing it. Especially applies to memory.
>> You can do something with trying to use channels after closing, but at
>> maximum this will show you a nice error which even will not crash Vim.
>> Similar error with using memory after freeing it at best will crash
>> Vim and provide a big message with lots of hex codes which may even
>> frighten some unexperienced user. At worst all you get is crashing Vim
>> a message to stderr (GVim users may never see it depending on how they
>> run GVim) about catching deadly signal SEGV without any details at
>> all. (As you see, Vim will crash in any case.)
>> - Crash due to out-of-bounds array access, with same amount of data as
>> above. VimL will just show a nice error message. Python will show
>> nicer error message, but the point is that it will not crash.
>>
>> And do not forget about one other good thing about VimL: if a user has
>> a distribution with two-years-old Vim to fix problem in C code he will
>> need C compiler, git, some shell experience and at least six commands
>> to update Vim (clone, configure, build, install, update bashrc or
>> whatever; plus one command per shell to load the update or reboot;
>> plus maybe something to make *.desktop files with GVim menu entries
>> point to a newly compiled Vim location). With VimL code user just
>> needs to drop a new version of the file into `~/.vim/autoload`.
>>
>> I was suggesting autoload because it is easier to write and maintain,
>> not because you will need to write less *functionality*. Same DSL
>> implemented purely in VimL on top of my proposal will run slower, but
>> take much less lines of code.
>
> I have programmed a great deal in both C and VimL. While I'm not sure
> I agree that the possibility of memory leaks is a compelling reason
> for implementing the functionality in a script, I don't really have
> strong feelings on this. Would the functionality be documented in the
> official Vim help? I can't think of any user-facing Vim functions
> currently implemented outside Vim. Though it's certainly easier to
> update/override a distributed script than to download or build a new
> executable, as a plugin developer, I would probably not want to rely
> upon tweaked distribution scripts, since most of the plugin users
> would get the updated scripts only when they got the updated
> executable. Sure, the plugin README could instruct users with older
> Vim versions to install an updated autoload file manually outside the
> plugin directory, but users have grown accustomed to one-click plugin
> installation methods, and might consider this an annoyance...

This is not actually different from the current situation: was needing
to check for `exists('*win_id2win')`, will need to check for existence
of autoload functions (though probably better with EAFP since they are
autoload: run, catch the case when function not exists, assume old
version). The only bad thing is that for some rare cases you would not
be able to construct `bug_present()` condition where you could
previously rely on the availability of Vim patch. Not a big deal I
guess: you better not do this thing anymore in presence of Neovim
which may have neither a bug nor a patch which has fixed a bug (not
actually not have the patch, not have the information available).

Also while Vim with Bram for some reason does not rely on runtime,
Neovim has a growing set of runtime files required for normal
operation. And we are not against bundling libraries, though I am not
aware of any except for my autoload/shada.vim and autoload/msgpack.vim
which are so far mainly used for standard plugin operation.

I have programmed for Neovim in VimL, C and lua and can say for sure
that probability of making a hidden bug in C code is far greater then
probability of making a hidden bug in lua or VimL. Especially if you
do refactoring of core functionality (which you will need to in order
to be able to hide mappings) and not just adding methods for accessing
already-existing internal structures (which is all you need for my
proposal; I can bet that diff for necessary functionality may only
contain additions (better actually not though, in order to share some
code with maparg())). Since I do not propose to alter existing
functionality (except for some small refactorings to share code) new
features are most likely to not break anything not using them even if
new bugs are introduced.

>
> Thanks,
> Brett S.
>
>>
>>>
>>> Sincerely,
>>> Brett Stahlman
>>>
>>>>
>>>>>
>>>>> --
>>>>> A village. Sound of chanting of Latin canon, punctuated by short, sharp
>>>>> cracks. It comes nearer. We see it is a line of MONKS ala SEVENTH SEAL
>>>>> flagellation scene, chanting and banging themselves on the foreheads with
>>>>> wooden boards.
>>>>> "Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD
>>>>>
>>>>> /// Bram Moolenaar -- Bram@Moolenaar.net -- http://www.Moolenaar.net \\\
>>>>> /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
>>>>> \\\ an exciting new programming language -- http://www.Zimbu.org ///
>>>>> \\\ help me help AIDS victims -- http://ICCF-Holland.org ///

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

No comments: