navtree js output varies from ASLR

[bmwiedemann]

Describe the bug
doxygen produces nondeterministic navtreedata/index.js output from certain input (e.g. the eigen3 3.3.8 docs package in openSUSE), unless ASLR is disabled.

Expected behavior
output shoud be deterministic

To Reproduce
I made a minimal reproducer with just a few lines:
eigen3-min.tar.gz

tar xf eigen3-min.tar.gz
cd doxygennondeterminism
for i in $(seq 1 10) ; do ./build.sh ; done|sort|uniq -c
6 95846f882fbebd706b1cf5ee43e44cdf -
4 ded422b2f1ba4a1b1cbbba0a929c2554 -

Version
doxygen-1.8.20-3.1 of openSUSE-Tumbleweed-20201029

Additional context
Similar in nature to #7474

This PR was done while working on reproducible builds for openSUSE.

[albert-github]

I don't have an openSUSE system so I had to try it on windows.

I had to use a little bit an adjusted script (as I also have Cygwin I can use a lot of *nix commands as well):

rm -rf html
doxygen
md5sum html/navtree*.js|md5sum
md5sum html/navtree*.js

When running doxygen version 1.8.20 I get 2 different outputs

904d8112325aef5e415d70b8a5a49330 *-
2368aabf7d98fcf12ea9c2c346e96a24 *html/navtree.js
79ff46ed63b7500b6d56fd1f7e1df540 *html/navtreedata.js
e0180e4b4e0cf752a2f7647cc305454a *html/navtreeindex0.js

and

0469831f90d6ad2b114cda9628381d1c *-
2368aabf7d98fcf12ea9c2c346e96a24 *html/navtree.js
716c3828f321638e00d763ddcde804f1 *html/navtreedata.js
a2076aa20c74ac749015f5cc5301ee6f *html/navtreeindex0.js

Fortunately I saved the html directories (output_1_8_20.tar.gz) and could run a diff on them:

diff -r html_1/navtreedata.js html_2/navtreedata.js
40c40
< ""
---
> "annotated.html"
diff -r html_1/navtreeindex0.js html_2/navtreeindex0.js
3d2
< "":[0,0,0],
10c9,10
< "test1_8h_source.html":[1,0,0]
---
> "test1_8h_source.html":[1,0,0],
> "":[0,0,0]

I also ran with the current master version of doxygen (1.9.0 (67132b8)) and here I got every time (output_master.tar.gz):

3be8d8d938c7a6a18904b98973577c48 *-
2368aabf7d98fcf12ea9c2c346e96a24 *html/navtree.js
716c3828f321638e00d763ddcde804f1 *html/navtreedata.js
8ca4477087e438008150cd79eee546e2 *html/navtreeindex0.js

(I also got this result on Cygwin with the current master).

So my first conclusion is that there is a problem regarding the reproducibility in the 1.8.20 version and that this has been "fixed but not released" (in the current master).
Please check your problem with the current master.

[bmwiedemann]

I tried to build the master version, but it failed to compile with gcc10:

/home/abuild/rpmbuild/BUILD/doxygen-1.8.20/vhdlparser/VhdlParser.cc: In member function 'QCString vhdl::parser::VhdlParser::interface_subprogram_declaration()':
/home/abuild/rpmbuild/BUILD/doxygen-1.8.20/vhdlparser/VhdlParser.cc:11202:1: error: control reaches end of non-void function [-Werror=return-type]

[albert-github]

The code generated for VhdlParser.cc in the mentioned function looks like:

QCString VhdlParser::interface_subprogram_declaration() {QCString s;
    switch ((jj_ntk==-1)?jj_ntk_f():jj_ntk) {
    case PROCEDURE_T:{
      if (!hasError) {
      s = iproc();
      }
      if (!hasError) {
return s;
      }
      break;
      }
    case FUNCTION_T:
    case IMPURE_T:
    case PURE_T:{
      if (!hasError) {
      s = ifunc();
      }
      if (!hasError) {
return s;
      }
      break;
      }
    default:
      jj_la1[295] = jj_gen;
      jj_consume_token(-1);
      errorHandler->handleParseError(token, getToken(1), __FUNCTION__, this), hasError = true;
    }
assert(false);
}

this is generated code by javaCC from the file: vhdlparser.jj. At the moment we can not do a lot here.

I see that you use the compile option -Werror=return-type maybe you can disable this option or just make a warning out of it.

I now also understand why openSUSE has a patch with the 1.8.20 version, regarding vhdlParser.cc: https://build.opensuse.org/package/view_file/openSUSE:Factory/doxygen/vhdlparser-no-return.patch?expand=1
you could apply this patch on the master file.

[bmwiedemann]

I got doxygen master built now with SUSE patches added and confirmed that the whole eigen3 docs become reproducible then.

[albert-github]

Good to hear that you got the doxygen master version compiled and that the problem does surface anymore.

In the mean time we had a look at the problem of the warnings / errors regarding the return type and made a workaround, see proposed pull request #8164. This will most likely result in the fact that openSUSE won't need their patch for this issue anymore.

I think the basic problem as addressed in this issue is fixed but not released.

[doxygen]

This issue was previously marked 'fixed but not released',
which means it should be fixed in doxygen version 1.9.0.
Please verify if this is indeed the case. Reopen the
issue if you think it is not fixed and please include any additional information
that you think can be relevant (preferably in the form of a self-contained example).

[bmwiedemann]

I'm still seeing a similar issue with doxygen-1.9.1 and our libzypp package docs.

[albert-github]

The original problem was for the eigen3 package, the new comment (#8158 (comment)) is regarding the libzypp package.

Can you provide the following information:

• does openSuse still use patches, and yes which / where to find them
• where can we find the source of the libzypp package / better a small reproducible example of the problem.
• build instructions for the documentation of the libzypp package

Probably best to open a new issue for the problem with the libzypp package (as eigen3 was working).

[bmwiedemann]

Somehow I missed your comment earlier.

• we still have 4 patches in https://code.opensuse.org/package/doxygen/tree/master
• I will try to make a smaller reproducer for libzypp and file a new issue.