[1.8.16 regression] HTML output varies from ASLR

[bmwiedemann]

While working on reproducible builds for openSUSE, I found that with our update to doxygen-1.8.16

package builds of at least alsa and fltk packages became unreproducible.

It becomes reproducible again, if we disable ASLR

https://rb.zq1.de/compare.factory-20191223/alsa-compare.out

includes this diff

 <!-- start footer part -->
 <hr class="footer"/><address class="footer"><small>
-Generated by &#160;<a href="http://www.doxygen.org/index.html">
+Generated by &#href_anchor"http://www.doxygen.org/index.html">
 <img class="footer" src="doxygen.png" alt="doxygen"/>
 </a> 1.8.16

Isnt that even invalid HTML?

At least for alsa, this variation has low entropy, so one might randomly get identical results in 2 builds.

[albert-github]

This indeed looks like invalid HTML code.
A bit strange that you don't get a difference for the line:

 </a> 1.8.16

When looking at the footers as produced by doxygen -w html ... with 1.8.16 and 1.8.17 I don't see a difference, also a small test case didn't give a problem (with a default doxygen settings file).

• Can you please attach a, small, self contained example (source+configuration file in a tar or zip) that allows us to reproduce the problem? Please don't add external links as they might not be persistent.

[bmwiedemann]

fltk-mini.tar.gz

tar xzf fltk-mini.tar.gz
cd fltk-mini
./testdoxygen.sh

This one (sometimes) produced variations like

--- html/Nodes.xml      2020-01-03 11:22:38.408000000 +0000
+++ html2/Nodes.xml     2020-01-03 11:26:47.592000000 +0000
@@ -19450,14 +19450,14 @@
               <Anchor>a20dc86e7758a88b4d1f06323e2de9896</Anchor>
              </Node>
              <Node>
-              <Name>FL_BUTTON</Name>
+              <Name>FL_Button</Name>
               <Path>Enumerations_8H.html</Path>
-              <Anchor>a9c1d8f71cc0a6c159eac4f250ead7687</Anchor>
+              <Anchor>a1df1e46a5113628156ed7b7f995217d0</Anchor>
              </Node>
              <Node>
-              <Name>FL_Button</Name>
+              <Name>FL_BUTTON</Name>
               <Path>Enumerations_8H.html</Path>
-              <Anchor>a1df1e46a5113628156ed7b7f995217d0</Anchor>
+              <Anchor>a9c1d8f71cc0a6c159eac4f250ead7687</Anchor>
              </Node>
              <Node>
               <Name>FL_BUTTON1</Name>

[albert-github]

What is the relation of this DOCSET Nodes.xml problem in relation to original HTML problem?

I don't think the DOCSET problem is a real problem, it is just that 2 items are interchanged (in Enumerations.H there is a define for FL_BUTTON(n) as well as for FL_Button). I would not have expected that these are interchanged but would always be in the same order.

[bmwiedemann]

Both only appeared after we upgraded from 1.8.15 to 1.8.16 and both become reproducible when disabling ASLR, so they might have a common root cause. Can also be two distinct reasons, though.

[albert-github]

I rechecked as I tested 1.8.16 versus 1.8.17 (1.8.17 is the newest release), now I also tested 1.8.15 but my conclusions remain the same.
I'm still missing the example for the original problem.

[bmwiedemann]

Also worth mentioning that we add 6 patches on top of 1.8.16
https://github.com/bmwiedemann/openSUSE/tree/master/packages/d/doxygen

Can you reproduce the fltk problem with a clean 1.8.16 build?

[albert-github]

When you mean the second mentioned problem, yes I do see a difference between 1.8.15 and 1.8.16 around the line 19453 (and even more differences). Between 1.8.16 and 1.8.17 I do see also a lot of differences.

[bmwiedemann]

This bug is not about differences between versions, but about differences between multiple builds with the same (1.8.16) version.

[bmwiedemann]

fltk-mini.tar.gz

I made a minimal reproducer of the non-determinism with just 6 lines of .h and 3 lines of Doxyfile.

test with

# cat testdoxygen.sh
#!/bin/sh
cd documentation
for i in $(seq 1 10) ; do
  rm -rf html
  doxygen >/dev/null 2>&1
  md5sum html/* 2>/dev/null |md5sum
done

[bmwiedemann]

alsa-mini.tar.gz

condensed another minimal reproducer of non-determinism from the alsa doc portion.

[albert-github]

With the windows version 1.8.16 and 1.8.17 I see 2 different outputs, the difference is:

80,81c80,81
< <a id="a5c67d1ba0a241dd8eb002297cb6791ca"></a>
< <h2 class="memtitle"><span class="permalink"><a href="#a5c67d1ba0a241dd8eb002297cb6791ca">&#9670;&nbsp;</a></span>ctl <span class="overload">[1/2]</span></h2>
---
> <a id="ab1b63d6a4c461339b1c534e228c24289"></a>
> <h2 class="memtitle"><span class="permalink"><a href="#ab1b63d6a4c461339b1c534e228c24289">&#9670;&nbsp;</a></span>ctl <span class="overload">[1/2]</span></h2>
87c87
<           <td class="memname">struct snd_tplg_ctl_template* snd_tplg_tlv_dbscale_template::ctl[0]</td>
---
>           <td class="memname">const char* snd_tplg_tlv_dbscale_template::ctl</td>
91c91
< <p>array of widget controls </p>
---
> <p>control name or NULL if no control </p>
95,96c95,96
< <a id="ab1b63d6a4c461339b1c534e228c24289"></a>
< <h2 class="memtitle"><span class="permalink"><a href="#ab1b63d6a4c461339b1c534e228c24289">&#9670;&nbsp;</a></span>ctl <span class="overload">[2/2]</span></h2>
---
> <a id="a5c67d1ba0a241dd8eb002297cb6791ca"></a>
> <h2 class="memtitle"><span class="permalink"><a href="#a5c67d1ba0a241dd8eb002297cb6791ca">&#9670;&nbsp;</a></span>ctl <span class="overload">[2/2]</span></h2>
102c102
<           <td class="memname">const char* snd_tplg_tlv_dbscale_template::ctl</td>
---
>           <td class="memname">struct snd_tplg_ctl_template* snd_tplg_tlv_dbscale_template::ctl[0]</td>
106c106
< <p>control name or NULL if no control </p>
---
> <p>array of widget controls </p>

So there are 2 items interchanged in the output, not a real difference.

These 2 builds are the official builds and probably made with CMAKE_BUILD_TYPE:STRING=Release. When trying the current master (should have no influence on the sorting yet) I don't see the differences, but this build is made with CMAKE_BUILD_TYPE:STRING=Debug`.
(One can check this in e.g. CmakeCache.txt).

I guess that the optimizer does some none-reproducible things (This is not unknown for e.g. numerical calculations either).

Maybe you can try the problem with CMAKE_BUILD_TYPE:STRING=Debug i.e. specify with cmake -DCMAKE_BUILD_TYPE=Debug.

[bmwiedemann]

gcc's optimizer is fully deterministic. Also, the differences happen with a single doxygen binary without recompilation - that is what this bug is about.

[doxygen]

@bmwiedemann I agree that doxygen should produce the same output every time given a constant input, so I'll treat this as a bug.

[doxygen]

Looks like this commit is the cause. If one of the strings is a null pointer the result is not predictable.

[bmwiedemann]

I tested the patch on top of our doxygen-1.8.16 and it makes all package doc builds reproducible again. Yay!

[doxygen]

This issue was previously marked 'fixed but not released',
which means it should be fixed in doxygen version 1.8.18.
Please verify if this is indeed the case. Reopen the
issue if you think it is not fixed and please include any additional information
that you think can be relevant (preferably in the form of a self-contained example).