syntax colouring for robust-externalize

[dflvunoooooo]

Pre-checks*

Please change the following [ ] to [x] for confirmation.

• The feature request has not been suggested in this repository.

The Missed*

Is your feature request related to a problem? Please provide a clear and concise description of what the problem is.

• vscode dosn't recognize gnuplot or tikz syntax inside the provided environment of the package "robust-extrenalize" github link. The most commen ones are called "CachMe" and "CacheMeCode". As such, vscode does not colorize the code correctly and chektex complains about an unclosed math environment, when $ is used inside a gnuplot code.

\documentclass{scrartcl}

\usepackage{robust-externalize}
\usepackage{gnuplot-lua-tikz}
\usepackage{filecontents}

\begin{filecontents}{test.dat}   
    0.0000 0.0
    1.0000 1.0
    2.0000 2.0
    3.0000 3.0
\end{filecontents}

\begin{document}
    \begin{CacheMeCode}{gnuplot, tikz terminal}
        plot 'test.dat' using 1:($2-1)
    \end{CacheMeCode}
\end{document}

The Solution*

Please provide a solution you would like to have.
It would be great, if vscode would recognize those environments and colourize the code correctly and chektex wouldn't complain.

Anything Else?

Add any other context about the feature request below.
Might be related to #1963.

Edit: corrected typo.

[James-Yu]

ChkTeX issues should not be reported here. As to the highlighting, it seems that the package essentially makes any languages embeddable in latex, which may not be properly handled by the syntax highlighting engine vscode is using. @jlelong do you think we may support this feature?

[James-Yu]

Double checked the package and noted that the environment names are indeed provided by cacheEnvironment. Such customization cannot be supported due to limit of highlighting engine, unfortunately.

[dflvunoooooo]

I understand. But can the highlighting not at least be disabled for the CacheMe and CacheMeCode environments? The environments always start with one of those two.

[James-Yu]

May you kindly give insights on this request? @jlelong

[jlelong]

Sorry for missing the issue in the first place. The documentation of robust-externalize https://ctan.tetaneutral.net/macros/latex/contrib/robust-externalize/robust-externalize.pdf is not so easy to thumb to find the typical syntax that needs support. Could you give more details on this.

From what I understand, there are two different levels of support:

1. Declaring the content of CacheMe and CacheMeCode environments as verbatim. This would solve the $ issue but would not yield the proper syntax highlighting for the embedded language
2. For a restricted selection of languages, use the embedded language syntax inside CacheMe and CacheMeCode environments.

1 is easy. I need to think of 2 more in depth. More precisely, it really depends on the syntax used by CacheMe and CacheMeCode environments to specify the embedded language.

Btw, the latex syntax is developed at https://github.com/jlelong/vscode-latex-basics

[dflvunoooooo]

No problem! Thank you for coming back. Yeah, it is a flexible package. There are a lot of possibilities to set things up.

But, I don't know if this problem with the started math does happen with other languages or may happen with tikz. If not and only gnuplot is the problem, then the CacheMeCode environment, for the code itself, and PlaceholderFromCode environment, for any outsourced code, would be enough. If not, there is at least also CacheMe, tikzpictureC, \cacheEnvironmen and probably a lot more. The author or the contributors should know more.

To get the syntax right for the language inside those environments could be near impossible. There are ready made presets for some languages, python and tikz for example, but one can create ones own preset and give it any name. If the name does not contain the language, the latex configuration would have to be processed.

This is as far as I understand it. I am still new to robust-externalize.

[tobiasBora]

Thanks for your interest in my package. So usually, indeed, only environments CacheMeCode, PlaceholderPathFromCode, PlaceholderFromCode and SetPlaceholderCode (alias of PlaceholderFromCode) can contain non-LaTeX code. These 4 environments accept one mandatory argument, and may accept an optional argument before the mandatory argument. The mandatory arguments contains name of a placeholder like __FOO__ for the last three (no information is provided about the language as the placeholder is basically just a variable containing some code), while CacheMeCode accepts a list of pgf styles (same syntax as tikzset), like python, set placeholder eval={__FOO__}{\foo}. These styles might contain the name of the language, but it is not a guarantee as users might define their own presets like my matplotlib or even a compilation command like set compilation command={python "__ROBEXT_SOURCE_FILE__"}.

So not sure how to best handle that case if you want to support syntax highlight inside this environment. I would say that the best option would be:

• check if the line before contains a comment like % lang: python.
• If not, try to see if the name of the style is the name of a language.
• If not, try to search in the style if it contains a well known language name as a substring.
• If not print verbatim

[jlelong]

Thanks for these explanations. Could you provide for a syntax usage description for every environment to support.

VS Code syntax engine does allow to easily check the content of the line above. So looking for % lang: python would be too complicated. The grammar file is already several thousand lines long!

If I understand well, we can expect the standard syntax to be

\begin{CacheMe}[language,......]
  non latex code
\end{CacheMe}

I should be able to incorporate this syntax (with the embedded language properly highlighted) in the current latex grammar without adding too many lines. If the language name is not the first argument of the environment, then I suggest to consider its content as verbatim.

[tobiasBora]

I see. You are quite correct except that you actually want:

\begin{CacheMeCode}{language,......}
  non latex code
\end{CacheMeCode}

Though, you might want to allow also (note the comma turned into a space)

\begin{CacheMeCode}{language ......}
  non latex code
\end{CacheMeCode}

to allow users to create presets like python matplotlib as users rarely use the default preset.

[tobiasBora]

For the three others environments accepting no style, it might also be possible to check if the placeholder name starts with __LANGUAGE like in:

\begin{PlaceholderFromCode}{__PYTHON_TMP__}
def foo():
    return 42
\end{PlaceholderFromCode}

[jlelong]

Even though the latex syntax is developed at https://github.com/jlelong/vscode-latex-basics, I am reopening this issue to keep track of the progress.

[jlelong]

The support of robust-externalize, as mentioned in this issue, has been added by jlelong/vscode-latex-basics#77

[dflvunoooooo]

Thank you very much for your great work!