Latex files: ex option "hll" doesn't highlight current line

[Vouivre]

Hi Kyryl,

I had trouble to understand what was the ex option hll. Now I understand better. The ex option hll works perfectly with a c file but not with a tex file.

In a tex it can happens that a line begins with a \. For example:

\usepackage[T1]{fontenc}
\usepackage{geometry}

In that case, it doesn't work. In vi.c the match is done by

 syn_addhl("^.+$", 2, 1); 

The . matches any character but there is a problem with \. Moreover, a line can have some spaces/tabs and then a backslash. The following can be found in tex files:

\usepackage{xparse}
\begin{displaymath}
\end{displaymath}
           \begin{displaymath}
           \end{displaymath}

Where the spaces at the beginning of the lines can be spaces or tabs. Hhhhmmm, the following can also happen

This is some text with \textbf{boldface font} inside the text.

So I had changed

 syn_addhl("^.+$", 2, 1); 

to

syn_addhl("^([ |        ]+)?(\\\\)?.+$", 2, 1);

In the class, there is a space and a tab. But it wouldn't work for the last example.

Do you have any idea how to match \ with a regex ?

[kyx0r]

Hi Cedric,
It's hard for me to understand what you want to achieve exactly, but try messing around with hll
definition in conf.c

change the continuation bit from
{"tex", NULL, {14 | SYN_BD}, {1}, 0, 2},
to
{"tex", NULL, {14 | SYN_BD}, {0}, 0, 2},

is that what you want to achieve?
As far as I can tell everything in nextvi works correctly regarding syntax hl. If you want some very
complex special behavior for hll, you can create a special ft type for it, swap ft at runtime do what you want and swap back to original default ft. Nextvi has the means to do all that.

[Vouivre]

I'll try to explain it better: I want to use the the syntax highlighting, in particular the ex features hlp, hll and hlw for tex files. Those features works perfectly fine for c files, for example if I use them on vi.c but not for tex files. Usual syntax highlighting works for tex files, but not hll, hlp or hlw. With your change, I was able to get hll working as expected for a tex file.

Could you briefly summarize what is this continuation bit ?

[kyx0r]

Continuation byte defines where syntax highlight continues in a case of a match for a corresponding group.
In BRE, a group is any regular expression that gets captured inside ()
Possible values for continuation are 0, 1, -1 as described in vi.h L316 read it first as I am going to
explain what happens with hll below.

The pattern for hll is "^.+$" (actually the $ isn't needed anymore I should remove it later)
This pattern does not have any capturing groups except the implicit capturing group granted to
all entries in conf.c by rset_make(). The implicit group creates the separation needed between the
regular expressions, that's how it knows what matched what. Therefore end[0] byte is the only
corresponding byte that has a meaning full effect on entry {"tex", NULL, {14 | SYN_BD}, {0}, 0, 2},

When "^.+" matches the whole line, the code in syn_hilight() will check the continuation options
for this set. ( {"tex", NULL, {14 | SYN_BD}, {0}, 0, 2} is considered a set. so for instance the "tex" ft currently
has 5 sets like this in conf.c) When the set and some corresponding group in the set matches it's corresponding
continuation option is read. When it reads 1, like it currently is set by default, instead of terminating and calling it
"i've matched the whole line, no chars left, done." it would try to dig deeper and find more possible matches, the ones
other than whole line. It can do that because of the ^ anchor, which is why change 62. is so important. The ^ anchor ensures
that the regex won't simply say "hey just match the whole line again" every time string gets advanced by 1 when we
try to dig in deeper into the string.

The brother anchor $ can also have this property, though there isn't really any tangible use case for it in nextvi other than
to try and continue past the '\n' which can happen when using keybind R for example. But this is rather rare use, I did not code
nextvi to take advantage of $.

So what you did there when changing 1 to 0, is basically telling nextvi to be lazy and just go with the first thing that matches.
Which is totally fine if that's the result you wanted.

Generally these are the things to consider when determining how things get highlighted:

1. continuation options
2. order of the set, the topmost take priority

There are a few implications, when you remove the continuation byte from hll that means nothing else will be able to highlight
the line, due to posix submatch rule longest match first.
So it is possible to come up with a combination of syntax rules where a very special order is required and it can't be achieved using the default options. In that case you have the option to write C code that will dynamically construct appropriate regex and adjust pattern when needed based on some conditions you define in code, usually this operation is cheap enough to be viable. If I optimize re_comp and factor out the recursion it would be even more viable than ever.

Dammit, that was way longer than briefly, i'd call it breezily instead, ha.

[kyx0r]

Btw, Cedric, I don't use latex so I am not very familiar with whatever syntax it has. If you want to add something to it, like a more sensible default thing feel free to do so. I mean anything if you wanna add new programming language or whatevs I don't really mind you doing so. Adding stuff to conf does not really incur any meaningful performance loss, or some bad memory usage (though it does create a few dead static bytes) it's fine.

If you think having 0 there instead of 1 is better default for tex I guess you know better than me.

If you want hlp working for tex, gotta copy that set defined in c set with ->func set to 3

[Vouivre]

Thank you very much for this loooonnnnnggg and detailed explanation! I appreciate it a lot and ...... I feel embarassed that you took so much time!

Thank you!