Re: syntax files: how to prevent embedded syntax from leaking out of a region?

On Wednesday, October 4, 2023 at 8:02:42 PM UTC-5 M wrote:

> a consequence of having many poorly written syntax files. Main distribution included

Your idea of a different override for syntax-include is the practical solution. As you pointed out, syntax-include is dependent on the authors of all the syntax files in the distribution to beware of what happens when their syntax is syntax-included. It's the moral equivalent of cooperative multitasking or running a processor in real mode -- you have to trust every other syntax not to step on yours. A region needs some way to absolutely stop an included syntax from highlighting past the region.

My project is a free syntax file for Noweb source files.* A single Noweb source file can contain all the objects in multiple languages that make up a package. Those objects can be represented in the Noweb source file as fragments, to be pieced together by Noweb. For example, an inner syntax region can easily contain the open-brace for a block but not the close-brace.

An included syntax file will seize on that open-brace and highlight forward until it finds a matching close-brace. But it actually needs to stop at the end of the region that contains the code fragment. Because the close-brace it finds is likely to be in a different inner syntax region, maybe even one written in a different language!

As a plan "B", I hate to have to reinvent stock syntaxes, and I would probably do a poor job. This is my first major foray into Vim syntax highlighting. I might eventually take a stab at it, for languages I use where I have run into the problem. Plan "C" is to just document the problem, and give advice if I can for writing code fragments that sidestep the problem. Or plan "D", I could disable recognition of certain languages, which is what I do now, but I had been thinking of that as a stopgap measure during development, not a solution.

* https://metaed.com/papers/vim-noweb/

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vim_use/9f9dd00c-784b-4518-9560-152f0aff5c3cn%40googlegroups.com.

Re: syntax files: how to prevent embedded syntax from leaking out of a region?

"extend" clearly has the highest priority, so it cannot be overridden. But that's not the problem by itself. The real trouble is a misuse of it. It looks like every person out there starts playing with syntaxes by adding "keepend" and "extend" almost everywhere. While "extend" should only be used for those very special elements that essentially "violate normal rules", like escape sequences or such.

In other words, every inner syntax region is already "extend" by default, so an outer region _may_ put "keepend" to it if needed. But the inner region _under exceptional circumstances_ still may overpower it all with "extend". Maybe, we could add yet another override specially for syntax-include but, imo, that could lead to even more confusion. Instead, we should  better clean up existing syntax files from that garbage.

So, in my opinion, the whole issue is a consequence of having many poorly written syntax files. Main distribution included.

Kind regards,

Matvey

чт, 5 окт. 2023 г., 00:09 Edward McGuire <metaed@gmail.com>:

On Wednesday, October 4, 2023 at 1:55:55 PM UTC-5 Christian Brabandt wrote:
> Either nobody knows the answer or nobody understood the problem.

Thank you Christian, I'm glad to know it got out there anyway.

So maybe it will help for me to restate the problem, maybe not.

This has to do with syntax highlighting when a file written in one language has embedded in it a code block written in a different language.

The Vim distribution includes two great practical examples of how to do this.

Both examples highlight the code inside the block using different syntax rules than the code outside the block.

One is in help document "syntax.txt" under the tag "sh-embed".

This shows how to extend the sh syntax so that embedded awk code is highlighted using awk syntax.

The other is syntax file "ant.vim", which has a powerful technique to support multiple languages embedded in Ant syntax and highlight each language using the correct syntax file.

The problem I'm having is that sometimes code past the end of the block gets highlighted using the syntax rules for the code inside the block.

Here is an excerpt from the syntax I developed.

The excerpt highlights most of the buffer using the preinstalled "tex.vim".

Then, within any embedded code block introduced by a <<blockname>>= code block introducer, and terminated by an @ or another <<blockname>>= code block introducer, the embedded code block is highlighted by the preinstalled "make.vim".

syntax include @SyntaxTeX syntax/tex.vim

unlet b:current_syntax

syntax region Normal start=/\v%^/ end=/\v%$/ contains=@SyntaxTeX

syntax match nowebCodeChunkDeclarationMake "\v^[<][<][mM]akefile(|\s.*)[>][>][=]\s*$"

\ skipnl

\ containedin=@SyntaxTeX

\ nextgroup=nowebCodeChunkBodyMake

highlight link nowebCodeChunkDeclarationMake PreProc

syntax include @SyntaxMake syntax/make.vim

unlet b:current_syntax

syntax region nowebCodeChunkBodyMake

\ keepend

\ start="\v.*"

\ end="\v^([@]($| ))|([<][<].*[>][>][=][ \t]*$)"me=s-1

\ contained

\ contains=@SyntaxMake

The key bit is the "keepend" keyword on the embedded code block.

It is there to force the block terminator to also terminate any contained item.

It should stop the embedded language from reading past the block terminator.

But the embedded language syntax can override this using "extend".

That can cause the embedded language syntax highlighting to continue past the end of the code block delimiter.

And the problem seems to be that some of the preinstalled syntaxes do use "extend".

The question is how to prevent the preinstalled syntaxes from ever highlighting past the end of the delimiter.

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vim_use/7864bd47-6f44-42c5-8449-53e7740eae17n%40googlegroups.com.

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vim_use/CAFsTB%2BKM0Sg8njdvXVuy2tdbu7vgZjmk1SQRn8EcoNmn6mLGqA%40mail.gmail.com.

Re: syntax files: how to prevent embedded syntax from leaking out of a region?

On Wednesday, October 4, 2023 at 1:55:55 PM UTC-5 Christian Brabandt wrote:
> Either nobody knows the answer or nobody understood the problem.

Thank you Christian, I'm glad to know it got out there anyway.

So maybe it will help for me to restate the problem, maybe not.

This has to do with syntax highlighting when a file written in one language has embedded in it a code block written in a different language.

The Vim distribution includes two great practical examples of how to do this.

Both examples highlight the code inside the block using different syntax rules than the code outside the block.

One is in help document "syntax.txt" under the tag "sh-embed".

This shows how to extend the sh syntax so that embedded awk code is highlighted using awk syntax.

The other is syntax file "ant.vim", which has a powerful technique to support multiple languages embedded in Ant syntax and highlight each language using the correct syntax file.

The problem I'm having is that sometimes code past the end of the block gets highlighted using the syntax rules for the code inside the block.

Here is an excerpt from the syntax I developed.

The excerpt highlights most of the buffer using the preinstalled "tex.vim".

Then, within any embedded code block introduced by a <<blockname>>= code block introducer, and terminated by an @ or another <<blockname>>= code block introducer, the embedded code block is highlighted by the preinstalled "make.vim".

syntax include @SyntaxTeX syntax/tex.vim

unlet b:current_syntax

syntax region Normal start=/\v%^/ end=/\v%$/ contains=@SyntaxTeX

syntax match nowebCodeChunkDeclarationMake "\v^[<][<][mM]akefile(|\s.*)[>][>][=]\s*$"

\ skipnl

\ containedin=@SyntaxTeX

\ nextgroup=nowebCodeChunkBodyMake

highlight link nowebCodeChunkDeclarationMake PreProc

syntax include @SyntaxMake syntax/make.vim

unlet b:current_syntax

syntax region nowebCodeChunkBodyMake

\ keepend

\ start="\v.*"

\ end="\v^([@]($| ))|([<][<].*[>][>][=][ \t]*$)"me=s-1

\ contained

\ contains=@SyntaxMake

The key bit is the "keepend" keyword on the embedded code block.

It is there to force the block terminator to also terminate any contained item.

It should stop the embedded language from reading past the block terminator.

But the embedded language syntax can override this using "extend".

That can cause the embedded language syntax highlighting to continue past the end of the code block delimiter.

And the problem seems to be that some of the preinstalled syntaxes do use "extend".

The question is how to prevent the preinstalled syntaxes from ever highlighting past the end of the delimiter.

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vim_use/7864bd47-6f44-42c5-8449-53e7740eae17n%40googlegroups.com.

Re: syntax files: how to prevent embedded syntax from leaking out of a region?

On Mi, 04 Okt 2023, Edward McGuire wrote:

> This question got no replies. Did it fail to be circulated? Does nobody know the answer? Is it a stupid question? :)

Either nobody knows the answer or nobody understood the problem.

Thanks,
Christian
--
There is never time to do it right, but always time to do it over.

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vim_use/ZR21MlwibqavLbR/%40256bit.org.

Re: syntax files: how to prevent embedded syntax from leaking out of a region?

This question got no replies. Did it fail to be circulated? Does nobody know the answer? Is it a stupid question? :)

On Thursday, September 14, 2023 at 1:22:07 PM UTC-5 Edward McGuire wrote:

On Friday, July 21, 2023 at 3:18:39 PM UTC-5 Edward McGuire wrote:
> how can I reliably ALWAYS limit embedded language highlighting to the region that contains it, without the chance that the embedded language can break out?

Still looking for a solution to this -- how to prevent pre-installed language syntaxes from breaking out of syntax regions marked "keepend".

Cheers!

Edward

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vim_use/73c0b602-cb84-4597-b4f2-1256a69e1a1en%40googlegroups.com.

language specific glyphs

Modern fonts provide for `language specific typography' by mapping,
depending on the language or other preference, distinct sets of glyphs
to the same Unicode code points. Such is the case e.g. with Cyrillic,
where one may expect certain letters to look differently in, say,
Bulgarian, Russian, and Serbian. (And an example of a font that
provides this capability is Iosevka, which is my current favourite of
a monospaced font.)

In CSS, for example, the use of language specific glyphs is enabled
by setting `font-language-override' accordingly, while in a word
processor a generic setting, such as simply `language' is typically
used to the same end.

So, my question is: is it possible to choose language specific glyphs
in Vim? I was unable to find such an option, but I may be missing
something.

Thanks,
Boyko

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vim_use/CALdOZqmroSWX4KcYHWBjjgzQsVdix5%3Dgs9YPhbA%2BC7S-snF2BQ%40mail.gmail.com.

Re: Vim explained in 6 Kbyte - Translation

I just saw it, thank you.

I will try to help with the translation of the other content too in order to help with the spread of Vim.

Thank you once again. I am truly grateful for your work.

Harvey

On Monday, October 2, 2023 at 10:18:43 PM UTC+1 Christian Brabandt wrote:

On Mo, 02 Okt 2023, Harvey R wrote:

> That's wonderful. Thank you ever so much.
>
> And my apologies for posting twice. At first, I used groups.google.com to post, but then nothing appeared (so, I thought that emailing was a must)
>
> But anyway, it's sorted now. Thank you once again for your hard work.

That is no problem. It is here now: https://www.vim.org/6kbyte.php

Thanks,
Christian
--
Outside of a dog, a book is man's best friend. Inside of a dog, it is too
dark to read.

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vim_use/e38262ae-fccd-4253-9324-8fe5fc3ad136n%40googlegroups.com.