Broken equations

[meator]

Describe the bug
I am trying to render simple equations via doxygen's builtin LaTeX capabilities. But they seem to be (slightly) broken.

Here is my MRE:

/**
\mainpage
\f{align*}{
  \mathit{VAL} &= [7, 6, 3, 4, 3, 8, 1, 7, 9, 5, 4, 9] \\
  \mathit{IR} &= [1, 1, 2, 2, 2, 3, 3, 3, 5, 5, 6, 6] \\
  \mathit{IC} &= [1, 3, 2, 4, 5, 3, 5, 6, 1, 5, 2, 6] \\
\f}
*/

I have named this file test.dox to make doxygen find it. It also requires EXTRA_PACKAGES = {amsmath} in the
Doxyfile for the align* environment.

The rendered equation has a weird gray background (it should be black) and the last line
(beginning with IC) doesn't have square brackets around 1, 3, 2, 4, 5, 3, 5, 6, 1, 5, 2, 6.

This is what doxygen produces for the dark version of the equation: _formulas_dark.txt (This file is actually .tex but GitHub doesn't allow me to upload files with that extension).

When I run pdflatex on this, I get the expected output: _formulas_dark.pdf. But doxygen's output is broken.

Screenshots
Here is the (broken) equation (generated by Dockerfile below):

This is the same example but with FORMULA_FONTSIZE = 20 to make it more visible:

It looks like there are some traces of the missing square brackets in the screenshots if you look carefully.

To Reproduce
Here is the entire directory: latex.tar.gz.

I have created a Dockerfile that can reproduce the problem:

FROM ghcr.io/void-linux/void-linux:latest-full-x86_64

# These commands can take a while to complete.
RUN <<EOF
# Update
echo noextract=/etc/hosts > /etc/xbps.d/hosts.conf
xbps-install -ySu xbps
xbps-install -ySu

# Install essential packages.
xbps-install -y gcc grml-zsh-config zsh ncurses-term texlive texlive-latexextra texlive-dvi nano
# Clear unnecessary cache.
rm /var/cache/xbps/*
EOF

# Use zsh (Void uses dash which is very limited).
RUN chsh -s /usr/bin/zsh
CMD ["/usr/bin/zsh"]

# XBPS doesn't package the latest version of doxygen yet, so this installs it.
ADD https://github.com/doxygen/doxygen/releases/download/Release_1_9_7/doxygen-1.9.7.linux.bin.tar.gz /
RUN tar --strip-components=1 -C /usr -xf doxygen-1.9.7.linux.bin.tar.gz doxygen-1.9.7/bin

# Set up the project.
RUN mkdir project
WORKDIR project
RUN cat <<EOF > test.dox
/**
\mainpage
\f{align*}{
  \mathit{VAL} &= [7, 6, 3, 4, 3, 8, 1, 7, 9, 5, 4, 9] \\\\
  \mathit{IR} &= [1, 1, 2, 2, 2, 3, 3, 3, 5, 5, 6, 6] \\\\
  \mathit{IC} &= [1, 3, 2, 4, 5, 3, 5, 6, 1, 5, 2, 6] \\\\
\f}
*/
EOF
RUN doxygen -g && echo 'EXTRA_PACKAGES = {amsmath}' >> Doxyfile

RUN echo "echo Hi. Meator here. Use \\\'doxygen\\\' to reproduce." >> ~/.zshrc

Version
The Dockerfile uses version 1.9.7 (a2f34e1ed90f83713dc347b5134920f32c455d46), but I have tested this on 1.9.7
and 1.9.5 (both compiled through my package manager). I am running Void Linux.

Additional context
It was working flawlessly for me before but suddenly it broke. I have created the Dockerfile to try to verify that I didn't
break anything on my system but the clean Dockerfile is also broken. Could this be caused by a missing dependency?
I get no warnings nor errors when running doxygen.

Void is rolling release, so an update could have broken it. I know for sure that this wasn't caused by an update of doxygen
itself, it could have been one of the tools that are used by doxygen to convert LaTeX into image. If I knew exactly how
doxygen converts LaTeX into image, I could maybe debug this further. How does it do it?

[albert-github]

As you already wrote

I know for sure that this wasn't caused by an update of doxygen itself, it could have been one of the tools that are used by doxygen to convert LaTeX into image.

I think this might be correct as when I run I do see the things correct and also the _formulas_dark.tex that you showed looks OK.

Doxygen just creates a tex file and runs some external programs on, you can see these commands by using the debug option -d extcmd with doxygen (so doxygen -d extcmd Doxyfile).

I think both the brackets and background color problem are caused by the same problem.

[meator]

Is there a way to stop doxygen deleting latex files after completion?

[albert-github]

At the moment this is only possible when you edit the formula.cpp file and set there the RM_TMP_FILES to false (around line 42).

[meator]

Doxygen just creates a tex file and runs some external programs on, you can see these commands by using the debug option -d extcmd with doxygen (so doxygen -d extcmd Doxyfile).

At the moment this is only possible when you edit the formula.cpp file and set there the RM_TMP_FILES to false (around line 42).

Thanks @albert-github !

These are the relevant commands:

Executing external command `latex -interaction=batchmode _formulas_dark >/dev/null`
Executing external command `latex -interaction=batchmode _formulas_dark >/dev/null`
Generating image form_0_dark.png for formula
Executing external command `dvips -q -D 600 -n 1 -p 1 -o _form0_dark_tmp.ps _formulas_dark.dvi`
Executing external command `gs -q -dBATCH -dNOPAUSE -P- -dNOSAFER -sDEVICE=eps2write -o _form0_dark_tmp.eps -f _form0_dark_tmp.ps`
Executing external command `gs -q -dNOSAFER -dBATCH -dNOPAUSE -dEPSCrop -sDEVICE=pnggray -dGraphicsAlphaBits=4 -dTextAlphaBits=4 -r180 -sOutputFile=form_0_dark.png _form0 _dark_tmp_corr.eps`

So it's _formulas_dark -> _formulas_dark.dvi -> _form0_dark_tmp.ps -> _form0_dark_tmp.eps ?? _dark_tmp_corr.eps (or _form0_dark_tmp_corr.eps ?) -> form_0_dark.png.

There's also _form0 whose origin is unknown to me. Is it a file? I will omit this file from the following examples because doxygen doesn't output it even with RM_TMP_FILES (false) and because gs returns Last OS error: No such file or directory.

As you can see, there's ??. This conversion isn't in the logs. Doxygen has done something to _form0_dark_tmp.eps that wasn't mentioned by -d extcmd. And I can conform that this ?? is exactly where the issue occurs.

Here is gs -q -dNOSAFER -dBATCH -dNOPAUSE -dEPSCrop -sDEVICE=pnggray -dGraphicsAlphaBits=4 -dTextAlphaBits=4 -r180 -sOutputFile=form_0_dark.png _form0_dark_tmp.eps:

Here is gs -q -dNOSAFER -dBATCH -dNOPAUSE -dEPSCrop -sDEVICE=pnggray -dGraphicsAlphaBits=4 -dTextAlphaBits=4 -r180 -sOutputFile=form_0_dark.png _form0_dark_tmp_corr.eps:

The files are here: files.tar.gz

How is _corr file created?

[meator]

https://github.com/doxygen/doxygen/blob/master/src/formula.cpp#L429-L465 It is modified here. I will try to debug further.

[doxygen]

@meator The problem with the images with a dark background was that the bounding box was including the background (so basically the whole page was seen as content, not just the formula). This is why we use the bounding box of the formula with the white background (light mode) and apply it to the EPS file with the dark background as well (for dark mode). Doesn't explain the gray background, of course.

[albert-github]

@doxygen when I run the example from @meator with HTML_COLORSTYLE=TOGGLE the formula nicely fits in with the background. I thin that there is something else going wrong, hopefully @meator can find out on his system what is wrong.

[doxygen]

Might also be related to the ghostscript version. I saw a similar issue here: ImageMagick/ImageMagick#4797

[meator]

HTML_COLORSTYLE=TOGGLE doesn't help.

The difference between _form0_dark_tmp.eps and _form0_dark_tmp_corr.eps is really just the bounding box, but it breaks it.

--- _form0_dark_tmp.eps
+++ _form0_dark_tmp_corr.eps
@@ -1,7 +1,7 @@
 %!PS-Adobe-3.0 EPSF-3.0
 %%Invocation: gs -q -dBATCH -dNOPAUSE -P- -dNOSAFER -sDEVICE=eps2write ? ? -f ?
-%%BoundingBox: 0 0 612 792
-%%HiResBoundingBox: 0.00 0.00 612.00 792.00
+%%BoundingBox: 234 573 379 643
+%%HiResBoundingBox: 234.036 573.03 378.864 642.744
 %%Creator: GPL Ghostscript 10011 (eps2write)
 %%LanguageLevel: 2
 %%CreationDate: D:20230528140215+02'00'

What ghostscript version are you running?

[doxygen]

@meator Checked on my Linux machine with gs 9.50: output is ok. Then tried on my Mac with ghostscript 10.01.1: I see the same issue as you reported ☹️

[meator]

Yay! It's reproducible! Now is it a doxygen issue or a ghostscript issue?

By the way here's mine:

GPL Ghostscript 10.01.1 (2023-03-27)
Copyright (C) 2023 Artifex Software, Inc.  All rights reserved.

[doxygen]

Looks like I can get it to work correctly by changing sDEVICE=pnggray to -sDEVICE=pngalpha:

diff --git a/src/formula.cpp b/src/formula.cpp
index 65597ec384..0c73baa96a 100644
--- a/src/formula.cpp
+++ b/src/formula.cpp
@@ -468,7 +468,7 @@ static bool createPNG(const QCString &formBase,const QCString &outFile,double sc
 {
   const size_t argsLen = 4096;
   char args[argsLen];
-  qsnprintf(args,argsLen,"-q -dNOSAFER -dBATCH -dNOPAUSE -dEPSCrop -sDEVICE=pnggray -dGraphicsAlphaBits=4 -dTextAlphaBits=4 "
+  qsnprintf(args,argsLen,"-q -dNOSAFER -dBATCH -dNOPAUSE -dEPSCrop -sDEVICE=pngalpha -dGraphicsAlphaBits=4 -dTextAlphaBits=4 "
                "-r%d -sOutputFile=%s %s_tmp_corr.eps",static_cast<int>(scaleFactor*72),qPrint(outFile),qPrint(formBase));
   if (Portable::system(Portable::ghostScriptCommand(),args)!=0)
   {

can you test if this also fixed the problem for you?

[albert-github]

A bit after the party now, but for completeness,.

• I run gs 9.50 on windows
• I ran it now on Cygwin, gs version gives: 10.00.0 and this looks OK

  GPL Ghostscript 10.00.0 (2022-09-21)
  Copyright (C) 2022 Artifex Software, Inc.  All rights reserved.
  

  so might be a regression ...
• TOGGLE logy didn't help but as it was the only way for me to switch between light / dark I mentioned it here.

[meator]

Looks like I can get it to work correctly by changing sDEVICE=pnggray to -sDEVICE=pngalpha:

diff --git a/src/formula.cpp b/src/formula.cpp
index 65597ec384..0c73baa96a 100644
--- a/src/formula.cpp
+++ b/src/formula.cpp
@@ -468,7 +468,7 @@ static bool createPNG(const QCString &formBase,const QCString &outFile,double sc
 {
   const size_t argsLen = 4096;
   char args[argsLen];
-  qsnprintf(args,argsLen,"-q -dNOSAFER -dBATCH -dNOPAUSE -dEPSCrop -sDEVICE=pnggray -dGraphicsAlphaBits=4 -dTextAlphaBits=4 "
+  qsnprintf(args,argsLen,"-q -dNOSAFER -dBATCH -dNOPAUSE -dEPSCrop -sDEVICE=pngalpha -dGraphicsAlphaBits=4 -dTextAlphaBits=4 "
                "-r%d -sOutputFile=%s %s_tmp_corr.eps",static_cast<int>(scaleFactor*72),qPrint(outFile),qPrint(formBase));
   if (Portable::system(Portable::ghostScriptCommand(),args)!=0)
   {

can you test if this also fixed the problem for you?

The background is black, but few things are still missing:

[doxygen]

A bit of a hack, but things seem to start looking better if we slightly increase the bounding box:

diff --git a/src/formula.cpp b/src/formula.cpp
index 0c73baa96a..f4f438d058 100644
--- a/src/formula.cpp
+++ b/src/formula.cpp
@@ -441,11 +441,11 @@ static bool updateEPSBoundingBox(const QCString &formBase,
     {
       if (line.rfind("%%BoundingBox",0)==0)
       {
-        epsOut << "%%BoundingBox: " << x1 << " " << y1 << " " << x2 << " " << y2 << "\n";
+        epsOut << "%%BoundingBox: " << std::max(0,x1-1) << " " << std::max(0,y1-1) << " " << (x2+1) << " " << (y2+1) << "\n";
       }
       else if (line.rfind("%%HiResBoundingBox",0)==0)
       {
-        epsOut << "%%HiResBoundingBox: " << x1hi << " " << y1hi << " " << x2hi << " " << y2hi << "\n";
+        epsOut << "%%HiResBoundingBox: " << std::max(0.0,x1hi-1.0) << " " << std::max(0.0,y1hi-1.0) << " " << (x2hi+1.0) << " " << (y2hi+1.0) << "\n";
       }
       else
       {

can you try this?

[meator]

It looks like it's working now.

Thanks @doxygen !

[albert-github]

I just tested the both changes together for the original example:

• Windows with 9.50 looks OK
• Cygwin with 10.00.0 looks OK

@meator can you also publish the examples the examples with the matrices, so I can test them as well?

[meator]

/**
 * @f[
 * A=
 * \begin{bmatrix}
 *   7 & 0 & 6 & 0 & 0 & 0 \\
 *   0 & 3 & 0 & 4 & 3 & 0 \\
 *   0 & 0 & 8 & 0 & 1 & 7 \\
 *   0 & 0 & 0 & 0 & 0 & 0 \\
 *   9 & 0 & 0 & 0 & 5 & 0 \\
 *   0 & 4 & 0 & 0 & 0 & 9 \\
 * \end{bmatrix}
 * @f]
 *
 * @f{align*}{
 *   \mathit{VAL} &= [7, 6, 3, 4, 3, 8, 1, 7, 9, 5, 4, 9] \\
 *   \mathit{IR} &= [1, 1, 2, 2, 2, 3, 3, 3, 5, 5, 6, 6] \\
 *   \mathit{IC} &= [1, 3, 2, 4, 5, 3, 5, 6, 1, 5, 2, 6] \\
 * @f}
 * @f[
 * A=
 * \begin{bmatrix}
 *   7 & 0 & 6 & 0 & 0 & 0 \\
 *   0 & 3 & 0 & 4 & 3 & 0 \\
 *   0 & 0 & 8 & 0 & 1 & 7 \\
 *   0 & 0 & 0 & 0 & 0 & 0 \\
 *   9 & 0 & 0 & 0 & 5 & 0 \\
 *   0 & 4 & 0 & 0 & 0 & 9 \\
 * \end{bmatrix}
 * \Rightarrow
 * \begin{bmatrix}
 *   7 & 0 & 6 & 0 & 0 & 0 \\
 *   0 & 3 & 0 & 4 & 3 & 0 \\
 *   0 & 0 & 8 & 0 & 1 & 7 \\
 *   - & - & - & - & - & - \\
 *   0 & 0 & 0 & 0 & 0 & 0 \\
 *   9 & 0 & 0 & 0 & 5 & 0 \\
 *   0 & 4 & 0 & 0 & 0 & 9 \\
 * \end{bmatrix}
 * @f]
 *
 * @f{align*}{
 *   \mathit{VAL} &= [7, 3, 6, 8, 4, 3, 1, 7 | 9, 4, 5, 9] \\
 *   \mathit{IR} &= [1, 2, 1, 3, 2, 2, 3, 3 | 5, 6, 5, 6] \\
 *   \mathit{IC} &= [1, 2, 3, 3, 4, 5, 5, 6 | 1, 2, 5, 6] \\
 *   \mathit{index} &= [1, 9] \\
 * @f}
 */

This requires EXTRA_PACKAGES = {amsmath}.

[albert-github]

@meator Thanks for the example.

Just tested it and looks all OK.

[meator]

@albert-github Thanks for making temporary files keepable at runtime!

Yay! It's reproducible! Now is it a doxygen issue or a ghostscript issue?

By the way is this a bug in the new ghostscript or did doxygen do something wrong?

[doxygen]

@meator I'd say it is a bug in ghostscript that I now worked around.

[doxygen]

This issue was previously marked 'fixed but not released',
which means it should be fixed in doxygen version 1.9.8.
Please verify if this is indeed the case. Reopen the
issue if you think it is not fixed and please include any additional information
that you think can be relevant (preferably in the form of a self-contained example).