Conceal support for \mathbf and \mathrm

[lervag]

This was raised by @Konfekt in #1883 (comment) and also discussed several years ago in #557. Since VimTeX v2.0, this should be relatively straightforward to implement.

Before I implement anything, I wanted to be sure I "understand" the feature. We want:

• \mathbf{text} to be displayed as simply text but bolded.
• \mathrm{text} to be displayed as text (not bold).

Is there anything more?

[lervag]

For reference, here is how this is implemented in tex-conceal.vim.

[whisperity]

In addition, there is \text which is frequently used in math mode to fall back to text mode, which might have to be handled the same way \mathrm is.

[Konfekt]

These should be all text style modification commands:

\text
\textnormal
\textrm
\textbf
\textit
\textup
\textsl
\textsc
\textsf
\texttt
\mathbb
\mathbf
\mathscr
\mathcal
\mathrm
\mathfrak

For example \mathbb and \mathcal are already implemented in some cases.

[patnr]

Not sure if this is a good idea. But here goes.

It'd be great if it were possible to conceal custom macros/commands. Use case: I have a "master" macro that I use to define my vector and matrix text-styles. Typically this re-directs to mathbf, but depending on journal, it could re-direct to something else. It'd be great if I could use conceal on these with the bold style for example. If not implemented, perhaps it could be documented how to do it?

[Konfekt]

@patricknraanes Are you asking for something like adding a line

syn match texMathSymbol '\\C' contained conceal cchar=ℂ

to ~/.vim/after/syntax/tex/conceal.vim to conceal your custom command \C to ℂ?

[patnr]

Thanks, but I was thinking more something like the following example:

\newcommand{\mat}[1]{{\mathbf{{#1}}}}
\newcommand{\vect}[1]{{\bm{#1}}}

And then in the text I might write $ \mat{A} \vect{x} = \vect{b}$ which would conceal as A x = b somehow (how?)

The advantage of this system is that I can easily change the text styles for vectors and matrices. And I'd be cool if I could get conceal working for it.

[lervag]

These should be all text style modification commands:

\text
\textnormal
...
\mathrm
\mathfrak

For example \mathbb and \mathcal are already implemented in some cases.

Ok, so, this is developing into several related features in one issue. I understand that it might seem "trivial" to know which of the various commands should come under which category, but I don't know this. So, please, can someone please categorize according to my following list and also preferably show explicit examples (especially for the last group). This woul be a very big help for me to properly implement this.

1. Commands that should be fully concealed (e.g. \text).

2. Commands that should be concealed, but whose argument should be simply modified with bold or italics (\textit, \mathbf, etc).

3. Commands that should be concealed, but whose argument should be replaced by different characters (e.g. \mathcal{C} to ℂ). For this category, I would much appreciate a very explicit list of desired replacements.

@patrickraanes It should be simple to allow custom commands for the two categories above. I could see something like the following as an idea:

let g:vimtex_syntax_custom_cmds = {
      \ 'bold': ['mat', 'vect'],
      \ 'italic': ['todo'],
      \ 'hide': ['footnote'],
      \}

This would not differentiate math mode, but since we are concealing, it shouldn't be a problem.

If you want a more dynamic version, then I think I'll pass: parsing the \newcommands will be very challenging and I don't quite see a robust way to make this "just work" (unless I had a lot of spare time, which I don't).

To keep things separated, could you open a new issue for this with a reference here?

[Konfekt]

Commands that should be fully concealed (e.g. \text).

\text
\textnormal
\textrm
\textup
\texttt
\textsf
\textsc

\mathrm

Commands that should be concealed, but whose argument should be simply modified with bold or italics (\textit, \mathbf, etc).

\textbf (bold)
\textit (italic)
\textsl (italic)

\mathbf (bold)

Commands that should be concealed, but whose argument should be replaced by different characters (e.g. \mathcal{C} to ℂ). For this category, I would much appreciate a very explicit list of desired replacements.

\mathfrak

See the \mathfrak list of tex-conceal.vim

\mathcal
\mathscr

See the \mathcal list of tex-conceal.vim

\mathbb

See the \mathbb list of tex-conceal.vim

[Konfekt]

As a sidenote, not all fonts have all the characters used for concealing; for example, Dejavu Sans Mono only the more common blackboard math characters (though, for example, Droid Sans Mono is quite complete and has more of them).

[lervag]

@Konfekt Thanks for clearing things up!

As a sidenote, not all fonts have all the characters used for concealing; for example, Dejavu Sans Mono only the more common blackboard math characters (though, for example, Droid Sans Mono is quite complete and has more of them).

Yes, this is an important note. I guess I should give a comment about this in the docs.

[lervag]

@Konfekt

As a sidenote, not all fonts have all the characters used for concealing; for example, Dejavu Sans Mono only the more common blackboard math characters (though, for example, Droid Sans Mono is quite complete and has more of them).

Do you have an idea of a reference that I could use to guide people on how to install/use fonts with good unicode support?

[lervag]

I think we are getting close now. Do you miss anything here now?

[Konfekt]

Thank you, great work! I will give it a whirl.

Here's the competitor's list of Unicode fonts, and here some general installation instructions for good looking fonts on Linux.

[lervag]

Thanks for the links! I'll write up something and/or add links to the docs to help users interested in the subject.

[Konfekt]

I am not sure that the fonts in the list are available monospaced. Another list. There's also this free alternative to DejaVu. And Droid is very readable and complete; my gvimrc loads it.

[lervag]

I personally use Juliamono, which I find to be both quite pretty and very complete wrt. symbols.

[lervag]

Ok, I've added some text on the subject (see :help vimtex-syntax-conceal).

[mawkler]

Is it possible to set the highlight of the text inside these commands? I would like for the text inside \texttt{} to have a different highlight compared to regular text, so that it still stands out color wise while its surrounding command is concealed.

[lervag]

No, this requires more work, and we are getting closer to implementing a full syntax language on top of the builtin. That is, this type of customization would mean that you should specify the highlighting group yourself, in which case you must specify both styling and colors.

One simple possibility is to add a new option name that would allow you to specify the name, which would allow you to control the highlighting. E.g. something like this:

let g:vimtex_syntax_custom_cmds = [{'name': 'test', 'conceal': v:true}]
highlight texTestArg gui=bold guifg=red

Note, if we go in this direction, you should note that custom highlights like this should not go in your vimrc directly; see this gist.

What do you think? This would only involve a minimal change to vimtex, but would allow to specify the custom highlight in a quite detailed manner.

[mawkler]

One simple possibility is to add a new option name

I'm not quite following. When checking :h g:vimtex_syntax_custom_cmds it looks like there already is a name option.

I don't really know what the best solution is, but if the simple fix that you mention works I'm happy with that :)

[lervag]

One simple possibility is to add a new option name

I'm not quite following. When checking :h g:vimtex_syntax_custom_cmds it looks like there already is a name option.

Sorry, you're right - I forgot that. This actually means my suggestion is already possible to use.

I don't really know what the best solution is, but if the simple fix that you mention works I'm happy with that :)

You can try it, e.g. something like this (see the previously linked gist for explanation of the custom highlight mechanisms):

let g:vimtex_syntax_custom_cmds = [
      \ {'name': 'test', 'conceal': v:true},
      \]

function! MyCustomHighlights() abort
  highlight texCTestArg gui=underline guifg=blue
endfunction

augroup MyColors
  autocmd!
  autocmd ColorScheme * call MyCustomHighlights()
augroup END

[patnr]

I may have misunderstood something, but I believe the following feature is still missing (unintentionally): that the \text, \textrm and \textnormal conceal automatically (they are of course already eminently concealable using g:vimtex_syntax_custom_cmds)

[mawkler]

@lervag I tried your example code but I don't see any difference in highlighting. I'm guessing that it should highlight the text in the argument to \test{}?

:hi texCTestArg does output texCTestArg xxx gui=underline guifg=blue as expected though.

[lervag]

@lervag I tried your example code but I don't see any difference in highlighting. I'm guessing that it should highlight the text in the argument to \test{}?

:hi texCTestArg does output texCTestArg xxx gui=underline guifg=blue as expected though.

Sorry, my mistake: If you use Vim or neovim in a terminal you need set termguicolors for the gui= and guifg= to be relevant. Alternatively, you can write something like this:

highlight texCTestArg cterm=underline ctermfg=blue

[lervag]

I may have misunderstood something, but I believe the following feature is still missing (unintentionally): that the \text, \textrm and \textnormal conceal automatically (they are of course already eminently concealable using g:vimtex_syntax_custom_cmds)

I think this should work as expected now.

[patnr]

Like a charm! And thanks for #2040 too!

[lervag]

Happy to hear it, and you're welcome :)

[mawkler]

Sorry, my mistake: If you use Vim or neovim in a terminal you need set termguicolors for the gui= and guifg= to be relevant. Alternatively, you can write something like this:

highlight texCTestArg cterm=underline ctermfg=blue

@lervag I did try that as well but I still didn't see any change.

:hi texCTestArg now outputs texCTestArg xxx cterm=underline ctermfg=12

[lervag]

@Melkster With the following vimrc and test file everything seems to work well for me. Did you remember to update?

% test.tex
\documentclass{minimal}
\begin{document}

Hello World! \test{testing}

\end{document}

" minimal vimrc
set nocompatible
let &rtp = '~/.vim/bundle/vimtex,' . &rtp
let &rtp .= ',~/.vim/bundle/vimtex/after'
filetype plugin indent on
syntax enable

nnoremap q :qall!<cr>

set termguicolors

let g:vimtex_syntax_custom_cmds = [
      \ {'name': 'test', 'conceal': v:true},
      \]

function! MyCustomHighlights() abort
  highlight texCTestArg gui=underline guifg=blue
endfunction

augroup MyColors
  autocmd!
  autocmd ColorScheme * call MyCustomHighlights()
augroup END

call MyCustomHighlights()

silent edit test.tex

If I now do nvim -u test.vim, it looks like this:

[mawkler]

@lervag Thank you, your test.vim works for me!

I seem to discovered two problems that I was having. The first one is related to nvim-treesitter because if I add it to your test.vim the highlighting no longer works. I'll try to look more into why that is.

The second problem was that I tried to put the config in ~/.config/nvim/after/syntax/tex.vim. Shouldn't that work? The file does run because :hi texCTestArg outputs the correct result, but the text inside \test{} isn't highlighted.

[clason]

nvim-treesitter now includes a latex (and bibtex) parser, so you probably have tree-sitter-based highlighting enabled -- which disables vanilla vim syntax highlighting (and hence conceal, which relies on it). (You do have highlighting in this case, but with different highlight groups that you probably don't have set up.)

What does :TSInstallInfo say?

[lervag]

@lervag Thank you, your test.vim works for me!

Glad to hear it.

I seem to discovered two problems that I was having. The first one is related to nvim-treesitter because if I add it to your test.vim the highlighting no longer works. I'll try to look more into why that is.

Ah; I've yet to delve into the treesitter stuff. I'll probably stay on the fence for some time, but I do see how this can give useful improvements to neovim.

The second problem was that I tried to put the config in ~/.config/nvim/after/syntax/tex.vim. Shouldn't that work? The file does run because :hi texCTestArg outputs the correct result, but the text inside \test{} isn't highlighted.

Ah, no, that won't work. That is, the VimTeX options must be defined before VimTeX loads, and the after/... directory is specifically intended to load things after.

[clason]

Ah; I've yet to delve into the treesitter stuff. I'll probably stay on the fence for some time, but I do see how this can give useful improvements to neovim.

It's surprisingly good for being that new; of course, it doesn't come close to the support for specific packages vimtex has.

One advantage is that you can easily inject Lua syntax highlighting for luatex's \directlua{...} (which I have not got around implementing...)

You can also pretty easily (and efficiently) do something like in_math_zone() snippet, see nvim-treesitter/nvim-treesitter#1184 (comment)

I've been trying to dampen enthusiasm for getting tree-sitter support into vimtex as you probably don't want to add to your maintenance burden, but people would be excited ;)

[lervag]

It's surprisingly good for being that new; of course, it doesn't come close to the support for specific packages vimtex has.

That sounds promising :)

You can also pretty easily (and efficiently) do something like in_math_zone() snippet, see nvim-treesitter/nvim-treesitter#1184 (comment)

Yes, I can imagine that things like this is much easier if you have a parsed syntax tree to work with!

Although, even with treesitter, there has to be a lot of manual code and bookkeeping of the various LaTeX commands and packages. I imagine it may look better, be easier to read, and be more compact, than the current VimTeX implementation based on the Vim syntax scripts.

I've been trying to dampen enthusiasm for getting tree-sitter support into vimtex as you probably don't want to add to your maintenance burden, but people would be excited ;)

Hah, yes. Perhaps for VimTeX 3.0? Perhaps I'll leave the Vim crowd behind and go pure neovim as well? (I'm kidding; for now.)

But seriously: I think it is important to follow the development of the tools, and it seems clear that treesitter is a huge upgrade to the syntax parsing and highlighting infrastructure. I don't expect to make any huge leaps in VimTeX in near future, but as I expect to still be writing LaTeX documents 10 years from now, I also imagine there will come a time when I'll seriously consider to drop the Vim syntax scripts in favor of treesitter.

[clason]

Although, even with treesitter, there has to be a lot of manual code and bookkeeping of the various LaTeX commands and packages. I imagine it may look better, be easier to read, and be more compact, than the current VimTeX implementation based on the Vim syntax scripts.

That depends on how much you like Lisp ;)

(You are right; the fact that there are so many packages defining their own commands and environments is a major hassle, and tree-sitter can't help there much, if at all.)

Hah, yes. Perhaps for VimTeX 3.0? Perhaps I'll leave the Vim crowd behind and go pure neovim as well?

Come to the dark side... :)

(I'm kidding; for now.)

But seriously: I think it is important to follow the development of the tools, and it seems clear that treesitter is a huge upgrade to the syntax parsing and highlighting infrastructure. I don't expect to make any huge leaps in VimTeX in near future, but as I expect to still be writing LaTeX documents 10 years from now, I also imagine there will come a time when I'll seriously consider to drop the Vim syntax scripts in favor of treesitter.

I think thinking of tree-sitter as just "faster syntax" or "prettier highlighting" is seriously selling it short. At the core, it gives you powerful tools of working with an (incrementally, error-resilient) parsed syntax tree -- and there are a lot of things you can do with that. Two early things are https://github.com/nvim-treesitter/nvim-treesitter-refactor and https://github.com/nvim-treesitter/nvim-treesitter-textobjects. Another thing I'm quite excited for is https://github.com/vigoux/architext.nvim (think in_math_zone on steroids). You could also do breadcrumbs or context-dependent comment strings or ...

(Currently, enabling tree-sitter for a language disables the built-in syntax files, which turns off many of vimtex's features; that is something that should be handled more granularly. You can tell nvim-treesitter not to do that with the additional_vim_regex_highlighting = true setup option, but this may lead to unexpected results (as this will apply both highlights, with the later one "winning").)

But it's early days; official support has been postponed to neovim 0.6 (which hopefully won't take as long as 0.5...), and the current version should be considered "early access".

[lervag]

Although, even with treesitter, there has to be a lot of manual code and bookkeeping of the various LaTeX commands and packages. I imagine it may look better, be easier to read, and be more compact, than the current VimTeX implementation based on the Vim syntax scripts.

That depends on how much you like Lisp ;)

I really like Lisp, or at least the ideas. I've just never had a really good opportunity of using it for anything.

I think thinking of tree-sitter as just "faster syntax" or "prettier highlighting" is seriously selling it short.

For the record: I never meant to imply that. It's exactly what I meant with syntax parsing, and I'm intrigued by the opportunities this concept provides. In any case, thanks for the interesting references and info!

But it's early days; official support has been postponed to neovim 0.6 (which hopefully won't take as long as 0.5...), and the current version should be considered "early access".

I consider myself somewhat experimental, but there's a time cost to most things, and I am unfortunately quite time bound. So I'll probably start to consider treesitter more seriously when I get a sense that is is more mature. I'm looking forward to it! :)

[mawkler]

What does :TSInstallInfo say?

@clason This is the output from :TSInstallInfo

bash            [✓] installed
beancount       [✓] installed
bibtex          [✓] installed
c               [✓] installed
c_sharp         [✓] installed
clojure         [✓] installed
comment         [✓] installed
cpp             [✓] installed
css             [✓] installed
dart            [✓] installed
elm             [✗] not installed
fennel          [✓] installed
glimmer         [✓] installed
go              [✓] installed
gomod           [✓] installed
graphql         [✓] installed
haskell         [✗] not installed
html            [✓] installed
java            [✓] installed
javascript      [✓] installed
jsdoc           [✓] installed
json            [✓] installed
jsonc           [✓] installed
julia           [✓] installed
kotlin          [✓] installed
latex           [✓] installed
ledger          [✓] installed
lua             [✓] installed
nix             [✓] installed
ocaml           [✓] installed
ocaml_interface [✓] installed
php             [✓] installed
python          [✓] installed
ql              [✓] installed
query           [✓] installed
r               [✓] installed
regex           [✓] installed
rst             [✓] installed
ruby            [✓] installed
rust            [✓] installed
scala           [✗] not installed
scss            [✗] not installed
sparql          [✓] installed
svelte          [✓] installed
teal            [✓] installed
toml            [✓] installed
tsx             [✓] installed
turtle          [✓] installed
typescript      [✓] installed
verilog         [✓] installed
vue             [✓] installed
yaml            [✓] installed
zig             [✓] installed

[mawkler]

the VimTeX options must be defined before VimTeX loads

@lervag Of course! That makes sense. Keeping

let g:vimtex_syntax_custom_cmds = [
      \ {'name': 'test', 'conceal': v:true},
      \]

in my .vimrc (init.vim) and moving

function! MyCustomHighlights() abort
  highlight texCTestArg gui=underline guifg=blue
endfunction

augroup MyColors
  autocmd!
  autocmd ColorScheme * call MyCustomHighlights()
augroup END

call MyCustomHighlights()

to ~/.config/nvim/after/syntax/tex.vim works (with nvim-treesitter uninstalled).

[clason]

What does :TSInstallInfo say?

@clason This is the output from :TSInstallInfo

This confirms my suspicion. I suggest disabling the latex parser/highlight for now when using VimTeX.

[mawkler]

@clason This might be a stupid question, but how do I uninstall the latex parser for nvim-treesitter? I tried :TSUninstall latex but it still says installed in :TSInstallInfo...

This is my config, I even tried removing ensure_installed = "maintained" but I can't seem to uninstall anything even if I run :TSUninstall all:

require('nvim-treesitter.configs').setup {
  ensure_installed = "maintained",
  highlight = {
    enable = true,
  },
}

[clason]

Don't use ensure_installed = "maintained" :) Did you restart neovim after uninstalling?

You can add disable = "latex" to each module, as in

highlight = {
    enable = true,
   disable = "latex",
  },

[mawkler]

@clason I tried a bunch of stuff, including removing ensure_installed and restarting with no success. Luckily your advice

You can add disable = "latex" to each module

worked for me. Thanks! :)