Re: substitue( optimize

2018-02-14 11:31 GMT+03:00 Ni Va <nivaemail@gmail.com>:
> Entirely agree with your VimL principle : less command is fast/better, inner
> system apis... Principle that works for any Language.

Not actually. What I mean here is that you should use VimL expressions
where normally you would use commands: `let [a, b] = [1, 2]` should be
a bit faster then `let a = 1|let b = 2`, though both are semantically
equivalent and first even allocates a list to be disposed of when
command finishes. In many cases this reasoning applies to `call
map(copy(list), …)` vs `for val in list`, sometimes even if `for`
variant contains `:break` command while `map()` obviously can't.

The principle may be reworded as "parsing VimL expressions is far
faster then parsing VimL commands". Reasoning is simple: in modern
scripting languages code follows the path

source code ---(parser)--> AST ---(translator)--> bytecode[
---(compiler)--> machine code]

Only after undergoing all these translations code is actually
executed. In VimL there is no parser, no translator and no compiler.
Vim just stores source code strings (possibly with some minor
conversions like reencoding (:h :scriptencoding) or joining lines (:h
line-continuation)) and executes them as they are parsed, never
storing parsing results in any format for future executions. Thus
while in other programming languages equivalent commands and
expressions will produce nearly the same or even identical
byte/machine code, no matter how parsing commands is more/less
complicated then parsing expressions (if there is separate between
commands and expressions at all), in VimL you pay a price for parsing
command every time you execute it and commands parser is just slower
(`map()` vs `:for`), in addition to the fact that "one big expression
with side-effects" approach in many cases means that you need to call
expressions parser single time vs multiple times for multiple commands
with the same effect.

>
> Out the fact that it is not here a xml treenode parser but inlines binary
> file modifications, do you think modifying large xml file (1Million line
> 40Mo) can be as fast as libxml (ox ruby, python) or xpath parser ?

For the above reasons (no parser and no translator) VimL can never
hope to achieve performance comparable to a C library. This may change
in Neovim as I am working on adding parser and translator after some
time, but work is going rather slowly; currently there only is parser
for expressions and it is only used for command-line highlighting (I
mean, highlighting expressions in `<C-r>=` and the like prompts): no
translator is available and neither there is code that executes AST
without translating.

>
> 2018-02-14 9:20 GMT+01:00 Nikolay Aleksandrovich Pavlov <zyx.vim@gmail.com>:
>>
>> 2018-02-14 10:41 GMT+03:00 Ni Va <nivaemail@gmail.com>:
>> > Le mardi 13 février 2018 14:30:38 UTC+1, Luc Hermitte a écrit :
>> >> Hello Nicholas,
>> >>
>> >> > After reading a file and getting 1 million lines' List, I need to
>> >> > make similar substitutions over matched lines ~500 times with this
>> >> > func:
>> >> >
>> >> >
>> >> > fun! foo#set_new_val(ressource) abort "{{{
>> >> >
>> >> > " position
>> >> > let pos = foo#getpos_ress(a:ressource)
>> >> > let pres = foo#geth_presence(a:ressource)
>> >> > let s:xml[pos] = substitute(s:xml[pos],'false\|true',pres,"") <<<
>> >> > HERE
>> >> >
>> >> > " Slave/Voie01 to Voie08
>> >> > for idx in [1,2,3,4,5,6,7,8] <<< HERE
>> >> > let pos = foo#getpos_ress(a:ressource.'/bar0'.string(idx))
>> >> > let s:xml[pos] = substitute(s:xml[pos],'false\|true',pres,"")
>> >> > <<<
>> >> > HERE
>> >> > endfor
>> >> >
>> >> > endfu"}}}
>> >> >
>> >> >
>> >> > 1/ Can I optimize this function ?
>> >>
>> >> So far my experiments on the subject taught me that we are best
>> >> avoiding :for and :while constructs when performances matter.
>> >> If you can use map() instead, do!
>> >>
>> >> HTH,
>> >>
>> >> --
>> >> Luc Hermitte
>> >
>> > Yess Luc !
>> >
>> > I finally reached to the same conclusion.
>> >
>> > 1/ But mapping Dict or List, inside the substitute func is needed no ?
>> > 2/ Other point, passing by register with emmbedded pattern is it faster
>> > ?
>> >
>> > let @a = '\w\+\(<\/PRES\)'
>> > let @b = pres.'\1'
>> >
>> > call map(voies_no, 'substitute(s:xml[v:val],"\=@a","\=@b","")')
>>
>> Using registers here makes no sense, you can just use variables. `\=`
>> is not going to work like you typed here. For loop with preceding
>> substitute from original function look like they can be rewritten into
>>
>> call map(
>> \[a:ressource] + map(range(1, 8), 'a:ressource."/bar0".v:val'),
>> \'extend(
>> \ s:xml, {
>> \ extend(l:, {"pos": foo#getpos_ress(v:val)}).pos
>> \ : substitute(s:xml[pos],''\c\vfalse|true'',pres,"")})')
>>
>> Note that my main VimL optimization principle is "less commands is
>> better". "Avoid loops" is one of the special cases only: by "less
>> commands" I mean not "less commands in code", but "less commands to
>> execute". I used to write a `:while` loop which contained nothing, but
>> conditional expression and all the work was done there (`extend(l:,
>> {})`, list literals as a replacement for comma operator and no
>> optimizer messing with sequence points rock (until you need to debug
>> this mess)). Note that for this reason lambdas (with the current
>> implementation) are a no-go: they create functions with `return`
>> command. If you need optimization and you have a choice between lambda
>> and some kind of `eval()` (e.g. map string literal) then use `eval()`.
>>
>> >
>> >
>> > Thanks you
>> >
>> > --
>> > --
>> > You received this message from the "vim_use" maillist.
>> > Do not top-post! Type your reply below the text you are replying to.
>> > For more information, visit http://www.vim.org/maillist.php
>> >
>> > ---
>> > You received this message because you are subscribed to the Google
>> > Groups "vim_use" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> > an email to vim_use+unsubscribe@googlegroups.com.
>> > For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> --
>> You received this message from the "vim_use" maillist.
>> Do not top-post! Type your reply below the text you are replying to.
>> For more information, visit http://www.vim.org/maillist.php
>>
>> ---
>> You received this message because you are subscribed to a topic in the
>> Google Groups "vim_use" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/vim_use/dY1x4ok__7w/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> vim_use+unsubscribe@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> --
> You received this message from the "vim_use" maillist.
> Do not top-post! Type your reply below the text you are replying to.
> For more information, visit http://www.vim.org/maillist.php
>
> ---
> You received this message because you are subscribed to the Google Groups
> "vim_use" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to vim_use+unsubscribe@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.