Re: Bug/non-determinism in output of maparg() and map commands

2017-05-28 19:05 GMT+03:00 Brett Stahlman <brettstahlman@gmail.com>:
> On Sun, May 28, 2017 at 6:50 AM, Nikolay Aleksandrovich Pavlov
> <zyx.vim@gmail.com> wrote:
>> 2017-05-28 8:57 GMT+03:00 Bram Moolenaar <Bram@moolenaar.net>:
>>>
>>> Nikolay Pavlov wrote:
>>>
>>> [..]
>>>
>>>> >> Enable, disable, query, execute plus two callbacks. *Four* functions
>>>> >> and two callbacks in place of just two simple functions, mostly using
>>>> >> the functionality that is already there. Five if you remember about
>>>> >> :mkvimrc and that somebody may want to replace that on top of new API:
>>>> >> query will need a mirror function for creating a mapping then.
>>>> >
>>>> > You are completely missing the point: those two functions don't provide
>>>> > the functionality we are talking about here.
>>>>
>>>> Why do you think so? They provide everything what is needed to
>>>> implement the functionality we are talking about here in VimL.
>>>
>>> This whole discussion started about the difficutly of using the result
>>> of maparg(). As far as I can see you only provide a function to get more
>>> mappings, it doesn't help with disabling and restoring mappings.
>>> Dealing with that is not easy at all (dealing with modes,
>>> escaping/unescaping, etc). I would not want every plugin writer to
>>> re-invent the wheel.
>>>
>>> The basic discussion first need to be held is whether to refer to a
>>> mapping just by its LHS (plus mode) or a unique handle can be used. The
>>> latter is useful if we disable a mapping but not delete it, it can then
>>> be shadowed by another mapping.
>>>
>>> We probably also need some kind of re-enable or restore function, that
>>> takes the result of maparg() and defines a mapping with it. That avoids
>>> string manipulation and using :exe (this is tricky because of the way
>>> mappings need some characters to be escaped).
>>
>> Three functions:
>>
>> 1. Get mappings: mappings_dump(). Needs to output lhs and rhs in some
>> canonical form. Takes an argument which allows some basic filtering:
>> e.g. select only mappings belonging to buffer with number N or
>> globals, select only mappings with given modes and select only
>> mappings starting with some prefix.
>> 2. Create mappings: mappings_load(). Takes output from
>> mappings_dump(), with the help of mappings_canonicalize() it should be
>> possible to create your own mappings.
>> 2.1. Something to delete mappings: mappings_load() with missing rhs or
>> mappings_clear() which takes output of mappings_dump() and deteletes
>> those mappings.
>> 3. Create string for lhs or rhs in canonical form given something like
>> `"\<F1>"` or `"<F1>"` (not sure what would be better) as the input:
>> mappings_canonicalize().
>>
>> So "disabling" is
>>
>> let disabled_mappings = mappings_dump(some_filter)
>> call filter(disabled_mappings, some_more_complex_filter_if_needed)
>> call mappings_load(map(deepcopy(disabled_mappings),
>> \'[remove(v:val, "rhs"), v:val][-1]'))
>>
>> "Enabling" is
>>
>> call mappings_load(disabled_mappings)
>>
>> One of the key points is that mappings_dump() and mappings_load() need
>> to work with canonical form of lhs and rhs strings and thus
>> dictionaries returned by them are not entirely compatible with
>> maparg() results. Canonical form needs to be independent of
>> &cpoptions, allow distinguishing `<LT>F1>` and `<F1>` and
>> `mappings_canonicalize(mappings_canonicalize(x))` should be identical
>> to `mappings_canonicalize(x)`.
>>
>> ---
>>
>> You may also ask Brett Stahlman whether my proposal is enough, it does
>> not look like he thinks it is not, just that it may be less
>> convenient.
>
> I believe it would be possible to do pretty much anything with the
> functions you propose, though for some common use cases, it puts more of
> a burden on the user than may be warranted. Yes,
> some_more_complex_filter_if_needed could be a function that tests
> canonical lhs for conflict/ambiguity, retaining only the ambiguous maps,
> and a subsequent call to mappings_load() (passing the filtered results
> with 'rhs' key removed) could be used to delete the conflicting
> mappings. But this feels a bit roundabout and error-prone compared to...
>
> let handles = maplist('<F8>', 'nv')
> call disable_maps(handles)
>
> Question: Would the map handles be mode-specific? E.g., for a map
> defined with :map, would there be 1 or 3 handles? I'm assuming that your
> mappings_dump() would return a single dict with a modes flag of 'nvo' or
> something like that. It occurs to me that either approach could allow
> finer-grained control than the :*map commands provide, and could obviate
> the need for kludges like this (from the help on omap-info):
>
> {{{
> To enter a mapping for Normal and Visual mode, but not Operator-pending mode,
> first define it for all three modes, then unmap it for Operator-pending mode:
> :map xx something-difficult
> :ounmap xx
> Likewise for a mapping for Visual and Operator-pending mode or Normal and
> Operator-pending mode.
> }}}
>
> Also note that your approach does more than simply *disable*; it
> actually *removes* the mappings as far as Vim is concerned. Granted, if
> we keep a reference to disabled_mappings, it would be easy enough to
> recreate them later, but I would imagine this is significantly less
> efficient than simply toggling an internal flag indicating enabled
> status. Also, as I mentioned before, I don't think checking for
> ambiguity/conflict is something that *should* be done by a plugin
> developer. Sure, a plugin developer should be capable of writing an
> algorithm to do this (assuming he has access to a canonical lhs), but
> forcing each plugin developer to do it just feels wrong to me, even if
> it ends up being a smallish predicate function/filter (and I suspect
> doing it robustly would end up being more than a one-liner). Thus, at
> the very least, I would propose having mappings_dump() accept an input
> that requests ambiguous/conflicting maps only, or else a separate
> function such as mappings_get_conflicts().

This is *not* as simple as "toggling the internal flag". You want to
disable a mapping and use your own one instead, this means that there
will be other changes to the mapping code. Much likely the result will
be *less* efficient for a few first years: your plugin has one set of
users, but for features needed for your plugin *all* users will pay.
This is much less the case for my variant: additional functions also
cost space and CPU ticks for O(log N) function lookup (BTW, Neovim has
O(1) built-in function lookup), but this is all non-users will pay.

Though I would not talk about efficiency here: the question is whether
your plugin implemented on top of one feature or the other will switch
"modes" in ~50 ms I guess. If it can do this with my approach then my
approach is good enough. Also I have shared details regarding how my
proposed API will look like, but I have not seen detailed proposal
which uses handles.

>
> As I see it, your approach gives the plugin developer access to all map
> information, and lets him do whatever he wants with it, provided he's
> willing to do the work of slicing and dicing the data using Vim's list
> and dict manipulation functions. By providing all relevant data, while
> making few, if any, assumptions about the things a plugin developer
> might want to do with that data, you make nothing particularly easy, but
> everything possible. This is not wrong. It's simply a lower-level
> approach than what I was originally envisioning. Your argument is
> essentially that it's better to re-use existing data structures, in
> conjunction with sequence manipulation primitives like map() and
> filter(), than to invent a DSL whose purpose is to reduce tedious
> boilerplate as well as the potential for error associated with the most
> common map-related use cases. The advantage, I suppose, is that your
> approach is probably less work for Bram, fewer bugs for him to work
> through, and ultimately pretty flexible for a competent plugin developer
> who doesn't mind a litle extra data massaging. Whereas creating a DSL
> forces you (hopefully only once) to think through the likely use cases
> and make decisions about the sorts of things that *should* be done with
> the DSL, your approach leaves such decisions to each and every plugin
> developer: very flexible, but at the cost of forcing each plugin
> developer to reinvent the wheel for the most common cases. Ideal, in my
> opinion, would be a DSL that facilitated the common cases, without
> precluding resort to the brute-force, low-level primitives for those use
> cases not envisioned by the DSL designers...
>
> Though I'm not set on the "handle" abstraction, I do think the fact that
> Bram and I were independently thinking about the insulation it would
> provide from implementation details is probably an indication that the
> advantages are at least worth considering. (OTOH, I can also see a
> certain attractive simplicity in the use of maparg-style dictionaries as
> "handles".) I do think the prolog/epilog callbacks I mentioned in an
> earlier post would be a powerful mechanism for plugin developers. (Of
> course, it should be possible to implement these with your approach as
> well.)

They may be emulated using various techniques. Better if there would
be standard (distributed with Vim) autoload-based "plugin" for adding
callbacks or out of two different plugins using this feature one would
not see "original" mapping.

I would say though that attaching callbacks to mappings could be done
without plugin handles and especially without enabling/disabling
functionality. Without them you would need mappings_canonicalize() to
have a robust way to tell what you need to attach callbacks to, but in
any case having canonical form is a good idea also for obtaining
handles. So I would suggest to not take this feature into account when
deciding whether handles are needed.

>
> Idea: It would be even more powerful if there were a way to attach the
> callbacks to a *partial* map sequence, or alternatively, a way to
> "monitor" an "in-progress" map sequence that hasn't yet triggered: e.g.,
> a callback function that fires each time a key is pressed, which
> receives not only the last key pressed, but also the entire sequence
> entered thus far (and possibly even the time remaining on the map
> timeout). The default behavior would be to allow the sequence entry to
> continue normally, but the callback function would also have the ability
> to change the sequence currently recognized to something completely
> different: i.e., the callback would be able to manipulate the typeahead
> buffer. I'm thinking this sort of capability could be useful for the
> handling of `->' in the transliteration mode you were describing in an
> earlier email. In my own case, I'm wanting to implement a temporary
> "escape" from the special mode I described earlier: though I'm
> overriding builtin maps in special mode, I'd like to allow the user to
> execute a builtin (without getting out of special mode) by prefixing it
> with some sort of escape (e.g., `\', or `,'). I can achieve something
> like what I want by defining maps for things like `\d', and using
> feedkeys() or normal! in the map handler to execute builtin d, but this
> forces me to define a bunch of maps I don't really need. It would be
> nice if I could monitor the progress of a map as it's entered and simply
> "lie" to Vim about what's been entered when the callback logic decides
> it's appropriate. At any rate, I've digressed from the original topic a
> bit...

Callbacks on partial mappings is a thing which statusline plugins'
authors are going to like, but this also needs neither handles nor
enable/disable functionality to work.

>
>>
>> ---
>>
>> And why do you think that "do not make plugins reinwent the wheel" is
>> the same statement as "write needed functionality in C code"? You can
>> always add a new file to `autoload`, writing VimL code is easier then
>> writing C code.
>
> You mean implementing a sort of "DSL" layer (using the primitive
> functions you've proposed) in Vim script, and including it in the
> official distribution as an autoload file? If so, I suppose this could
> be done, but as part of the official distribution, it would still need
> to be thought through and tested pretty thoroughly, so I don't see much
> advantage over implementing it in source, where it would be
> significantly more efficient. But perhaps I misunderstood what you were
> suggesting...

I think you have only written VimL code, but not C one. VimL does not have
- Memory leaks. You can provoke one, of course, but 90% of code will
not ever cause any memory leaks unless you intentionally wrote a code
which causes memory leak (e.g. an always-growing cache of something
which is never cleaned up).
- Using some resource after freeing it. Especially applies to memory.
You can do something with trying to use channels after closing, but at
maximum this will show you a nice error which even will not crash Vim.
Similar error with using memory after freeing it at best will crash
Vim and provide a big message with lots of hex codes which may even
frighten some unexperienced user. At worst all you get is crashing Vim
a message to stderr (GVim users may never see it depending on how they
run GVim) about catching deadly signal SEGV without any details at
all. (As you see, Vim will crash in any case.)
- Crash due to out-of-bounds array access, with same amount of data as
above. VimL will just show a nice error message. Python will show
nicer error message, but the point is that it will not crash.

And do not forget about one other good thing about VimL: if a user has
a distribution with two-years-old Vim to fix problem in C code he will
need C compiler, git, some shell experience and at least six commands
to update Vim (clone, configure, build, install, update bashrc or
whatever; plus one command per shell to load the update or reboot;
plus maybe something to make *.desktop files with GVim menu entries
point to a newly compiled Vim location). With VimL code user just
needs to drop a new version of the file into `~/.vim/autoload`.

I was suggesting autoload because it is easier to write and maintain,
not because you will need to write less *functionality*. Same DSL
implemented purely in VimL on top of my proposal will run slower, but
take much less lines of code.

>
> Sincerely,
> Brett Stahlman
>
>>
>>>
>>> --
>>> A village. Sound of chanting of Latin canon, punctuated by short, sharp
>>> cracks. It comes nearer. We see it is a line of MONKS ala SEVENTH SEAL
>>> flagellation scene, chanting and banging themselves on the foreheads with
>>> wooden boards.
>>> "Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD
>>>
>>> /// Bram Moolenaar -- Bram@Moolenaar.net -- http://www.Moolenaar.net \\\
>>> /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
>>> \\\ an exciting new programming language -- http://www.Zimbu.org ///
>>> \\\ help me help AIDS victims -- http://ICCF-Holland.org ///

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.