Newer version using a lot of RAM compared to older version

[macawkitty]

Describe the bug
Doxygen 1.9.7 is using upwards of 20GB RAM compared to 1.9.1 which uses ~11GB with the same config and on the same machine.

Expected behavior
Newer version needs RAM at most as much as the older version needed.

[albert-github]

Without knowing which OS , where you got the exectables (and type of executables: release, debug, ...), and example it is hard to tell what the problem is.

Newer version needs RAM at most as much as the older version needed.

cannot be guaranteed by any program due to e.g. new features, bug fixes etc.

[doxygen]

To make it easier to quantify the issue, I ran doxygen on its own code with version 1.9.1 and then again on master using valgrind's massif tool.

Version 1.9.1 shows a relatively small memory usage peak after parsing (148 MB):

The latest version has a significantly higher peak (430 MB)

After that the memory usage is still higher, but not extremely so (162MB → 208MB, or a 28% increase).

Still need to investigate why the difference is so big and if it can be reduced.
Note that version 1.9.1 was still using qtools, whereas later versions switched to STL classes.
Note also that I set NUM_PROC_THREADS to 1 and HAVE_DOT to NO to prevent threading or external tool impact.

[albert-github]

To make it easier to quantify the issue, I ran doxygen on its own code with version 1.9.1 and then again on master using valgrind's massif tool.

It is not 100% clear to me what you computed, did you compute:

1. either
   • doxygen 1.9.1 on the doxygen 1.9.1 code
   • doxygen master on the doxygen master code
2. or
   • doxygen 1.9.1 on the doxygen 1.9.1 code
   • doxygen master on the doxygen 1.9.1code
3. or
   • doxygen 1.9.1 on the doxygen master code
   • doxygen master on the doxygen master code

(as version 1. would not make any sense, in my opinion, for comparing the results).

[doxygen]

@albert-github I used option 3) both executables run on the master code. I also did a dry run for each before doing the actual valgrind run to avoid any caching effects.

[doxygen]

With the referenced commit, the memory profile looks a lot better, and more similar to version 1.9.1:

@macawkitty Can you check if this also makes a significant difference in your project?

[doxygen]

This issue was previously marked 'fixed but not released',
which means it should be fixed in doxygen version 1.9.8.
Please verify if this is indeed the case. Reopen the
issue if you think it is not fixed and please include any additional information
that you think can be relevant (preferably in the form of a self-contained example).