doxygen 1.9.1 segmentation fault when generating libstdc++ documentation

[saifikhan]

Hi:

Working on generating GCC libstdc++ documentation using doxygen 1.9.1

Some indicative settings that are being used in user.cfg.in

LOOKUP_CACHE_SIZE = 9
EXTRACT_ALL = YES
EXTRACT_PRIVATE = YES
EXTRACT_PRIV_VIRTUAL = YES
EXTRACT_PACKAGE = YES
EXTRACT_ANON_NSPACES = YES
INTERNAL_DOCS = YES
DOT_GRAPH_MAX_NODES = 512

output snipped
...
...
Generating caller graph for function std::signbit
Generating call graph for function std::sin
Generating caller graph for function std::sin
Generating call graph for function std::sinh
Generating caller graph for function std::sinh
Generating caller graph for function std::size
Generating call graph for function std::skipws
Generating caller graph for function std::skipws
Generating caller graph for functio../../../../src/gcc/libstdc++-v3/scripts/run_doxygen: line 183: 276389 Segmentation fault (core dumped) doxygen ${outdir}/${mode}.cfg
make[1]: [Makefile:933: stamp-html-doxygen] Error 139 (ignored)

The extracted core dump is as follows:

       PID: 276389 (doxygen)
       UID: 1000 (strikr)
       GID: 1000 (strikr)
    Signal: 11 (SEGV)
 Timestamp: Tue 2021-03-30 12:25:05 BST (3h 7min ago)

Command Line: doxygen /strikr/infra/build/gcc/doc/doxygen/html.cfg
Executable: /usr/bin/doxygen
Control Group: /user.slice/user-1000.slice/session-8.scope
Unit: session-8.scope
Slice: user-1000.slice
Session: 8
Owner UID: 1000 (strikr)
Boot ID: 95325163742f48709d8181f8e17944c7
Machine ID: 384944d6d31246b7af2d037610dbb286
Hostname: freedom
Storage: /var/lib/systemd/coredump/core.doxygen.1000.95325163742f48709d8181f8e17944c7.276389.1617103505000000.zst
Message: Process 276389 (doxygen) of user 1000 dumped core.

            Stack trace of thread 276389:
            #0  0x0000562fd0975fc8 n/a (doxygen + 0x334fc8)
            #1  0x0000562fd097b1d6 n/a (doxygen + 0x33a1d6)
            #2  0x0000562fd0982142 n/a (doxygen + 0x341142)
            #3  0x0000562fd07700a5 n/a (doxygen + 0x12f0a5)
            #4  0x0000562fd07af3ae n/a (doxygen + 0x16e3ae)
            #5  0x0000562fd07507b5 n/a (doxygen + 0x10f7b5)
            #6  0x00007fa5acc17b25 __libc_start_main (libc.so.6 + 0x27b25)
            #7  0x0000562fd076247e _start (doxygen + 0x12147e)
            
            Stack trace of thread 277697:
            #0  0x00007fa5ad10e9ba __futex_abstimed_wait_common64 (libpthread.so.0 + 0x159ba)
            #1  0x00007fa5ad108260 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf260)
            #2  0x00007fa5acfe5bb1 __gthread_cond_wait (libstdc++.so.6 + 0xc9bb1)
            #3  0x0000562fd0c5626c n/a (doxygen + 0x61526c)
            #4  0x0000562fd0c58f8e n/a (doxygen + 0x617f8e)
            #5  0x00007fa5acfebbc4 execute_native_thread_routine (libstdc++.so.6 + 0xcfbc4)
            #6  0x00007fa5ad102299 start_thread (libpthread.so.0 + 0x9299)
            #7  0x00007fa5accef053 __clone (libc.so.6 + 0xff053)
            
            Stack trace of thread 277695:
            #0  0x00007fa5ad10e9ba __futex_abstimed_wait_common64 (libpthread.so.0 + 0x159ba)
            #1  0x00007fa5ad108260 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf260)
            #2  0x00007fa5acfe5bb1 __gthread_cond_wait (libstdc++.so.6 + 0xc9bb1)
            #3  0x0000562fd0c5626c n/a (doxygen + 0x61526c)
            #4  0x0000562fd0c58f8e n/a (doxygen + 0x617f8e)
            #5  0x00007fa5acfebbc4 execute_native_thread_routine (libstdc++.so.6 + 0xcfbc4)
            #6  0x00007fa5ad102299 start_thread (libpthread.so.0 + 0x9299)
            #7  0x00007fa5accef053 __clone (libc.so.6 + 0xff053)
            
            Stack trace of thread 277698:
            #0  0x00007fa5ad10e9ba __futex_abstimed_wait_common64 (libpthread.so.0 + 0x159ba)
            #1  0x00007fa5ad108260 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf260)
            #2  0x00007fa5acfe5bb1 __gthread_cond_wait (libstdc++.so.6 + 0xc9bb1)
            #3  0x0000562fd0c5626c n/a (doxygen + 0x61526c)
            #4  0x0000562fd0c58f8e n/a (doxygen + 0x617f8e)
            #5  0x00007fa5acfebbc4 execute_native_thread_routine (libstdc++.so.6 + 0xcfbc4)
            #6  0x00007fa5ad102299 start_thread (libpthread.so.0 + 0x9299)
            #7  0x00007fa5accef053 __clone (libc.so.6 + 0xff053)
            
            Stack trace of thread 277696:
            #0  0x00007fa5ad10e9ba __futex_abstimed_wait_common64 (libpthread.so.0 + 0x159ba)
            #1  0x00007fa5ad108260 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf260)
            #2  0x00007fa5acfe5bb1 __gthread_cond_wait (libstdc++.so.6 + 0xc9bb1)
            #3  0x0000562fd0c5626c n/a (doxygen + 0x61526c)
            #4  0x0000562fd0c58f8e n/a (doxygen + 0x617f8e)
            #5  0x00007fa5acfebbc4 execute_native_thread_routine (libstdc++.so.6 + 0xcfbc4)
            #6  0x00007fa5ad102299 start_thread (libpthread.so.0 + 0x9299)
            #7  0x00007fa5accef053 __clone (libc.so.6 + 0xff053)
            
            Stack trace of thread 277699:
            #0  0x00007fa5ad10e9ba __futex_abstimed_wait_common64 (libpthread.so.0 + 0x159ba)
            #1  0x00007fa5ad108260 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf260)
            #2  0x00007fa5acfe5bb1 __gthread_cond_wait (libstdc++.so.6 + 0xc9bb1)
            #3  0x0000562fd0c5626c n/a (doxygen + 0x61526c)
            #4  0x0000562fd0c58f8e n/a (doxygen + 0x617f8e)
            #5  0x00007fa5acfebbc4 execute_native_thread_routine (libstdc++.so.6 + 0xcfbc4)
            #6  0x00007fa5ad102299 start_thread (libpthread.so.0 + 0x9299)
            #7  0x00007fa5accef053 __clone (libc.so.6 + 0xff053)

Any pointers or work arounds for now ?

[albert-github]

Maybe some ideas:

• Did you start with NUM_PROC_THREADS set to another value than 1? If so star with NUM_PROC_THREADS=1
• Try the most recent resent development version of doxygen
• From where did you get your doxygen executable? The information like 0x0000562fd0c5626c n/a (doxygen + 0x61526c) is not very informative, would be good to create / build he doxygen executable in the CMake phase with -DCMAKE_BUILD_TYPE=RelWithDebInf or even better -DCMAKE_BUILD_TYPE=Debug.

[saifikhan]

Thanks for your response.

Yes I'm using NUM_PROC_THREADS=1

cloned and built doxygen debug version from the git repo

doxygen -v
1.9.2 (9be70b2)

crash site

Generating caller graph for function __pstl::__utils::__set_symmetric_difference_construct
Generating call graph for function __pstl::__utils::__set_union_construct
Generating caller graph for function __pstl::__utils::__set_union_construct
Generating call g../../../../src/gcc/libstdc++-v3/scripts/run_doxygen: line 183: 304625 Segmentation fault      (core dumped) doxygen ${outdir}/${mode}.cfg
make[1]: [Makefile:933: stamp-html-doxygen] Error 139 (ignored)
echo timestamp > stamp-html-doxygen
make[1]: Leaving directory '/strikr/infra/build/gcc/doc'

here is the stack trace

Stack trace of thread 304625:
#0  0x000055e8d710b2c8 _ZNK10MemberList22writeDocumentationPageER10OutputListPKcPK17DefinitionMutable (doxygen + 0x3b82c8)
#1  0x000055e8d7116be6 _ZN16NamespaceDefImpl16writeMemberPagesER10OutputList (doxygen + 0x3c3be6)
#2  0x000055e8d7116940 _ZN16NamespaceDefImpl18writeDocumentationER10OutputList (doxygen + 0x3c3940)
#3  0x000055e8d6e79b45 generateNamespaceDocs (doxygen + 0x126b45)
#4  0x000055e8d6e879ac _Z14generateOutputv (doxygen + 0x1349ac)
#5  0x000055e8d6e41352 main (doxygen + 0xee352)
#6  0x00007f7dd322eb25 __libc_start_main (libc.so.6 + 0x27b25)
#7  0x000055e8d6e4124e _start (doxygen + 0xee24e)

environment

• ArchLinux
• gcc 10.2.0
• linux 5.11.10
• glibc 2.33-4

Please let me know.

[albert-github]

• I assume you used: -DCMAKE_BUILD_TYPE=RelWithDebInf
• maybe you can run with gdb and at the moment of the crash see whether you can get some information about the variables used by the subroutines
• How long does it take, maybe better to run with -DCMAKE_BUILD_TYPE=Debug (might be a bit slower ...) and also direct run with gdb so you can get some information about the variables at the moment of the crash
• Where can we find the source code used?
• Please attach the used Doxyfile to this issue.

[saifikhan]

yes i configured the build with -DCMAKE_BUILD_TYPE=Debug in the first instance itself.

let me try running gdb along and update you.

[saifikhan]

• maybe you can run with gdb and at the moment of the crash see whether you can get some information about the variables used by the subroutines

...
...
Generating docs for compound __pstl::__utils::__serial_move_merge...
Generating docs for namespace __pstl::execution

Program received signal SIGSEGV, Segmentation fault.

0x000055555590c2c8 in MemberList::writeDocumentationPage (this=0x5555737c4f80, ol=..., scopeName=0x555559d29c40 "__pstl::execution", 
    container=0x55557113abd0) at /strikr/src/doxygen/doxygen/src/memberlist.cpp:777
777         if (md->isDetailedSectionLinkable())

(gdb) bt

#0  0x000055555590c2c8 in MemberList::writeDocumentationPage (this=0x5555737c4f80, ol=..., scopeName=0x555559d29c40 "__pstl::execution", 
    container=0x55557113abd0) at /strikr/src/doxygen/doxygen/src/memberlist.cpp:777
#1  0x0000555555917be6 in NamespaceDefImpl::writeMemberPages (this=0x55557113abd0, ol=...) at /strikr/src/doxygen/doxygen/src/namespacedef.cpp:1042
#2  0x0000555555917940 in NamespaceDefImpl::writeDocumentation (this=0x55557113abd0, ol=...) at /strikr/src/doxygen/doxygen/src/namespacedef.cpp:1029
#3  0x000055555567ab45 in generateNamespaceDocs () at /strikr/src/doxygen/doxygen/src/doxygen.cpp:8939
#4  0x00005555556889ac in generateOutput () at /strikr/src/doxygen/doxygen/src/doxygen.cpp:11702
#5  0x0000555555642352 in main (argc=2, argv=0x7fffffffe898) at /strikr/src/doxygen/doxygen/src/main.cpp:38

(gdb)

does this help ?

[saifikhan]

source code used is available at https://github.com/gcc-mirror/gcc

and please see the attached Doxyfile.txt

[doxygen]

@saifikhan yes it helps, toMemberDefMutable() can return a null pointer when using inline namespaces, and I see this part of the code was not properly guarded for that. Please check if my commit prevents the crash.

[doxygen]

You use SEPARATE_MEMBER_PAGES = YES which is a not often used option; which is probably why this hasn't been found before.

[saifikhan]

@doxygen

• pulled in the commit
• build a new doxygen binary 1.9.2 (b2ead3a228cc44b8d3b67a4e71723f30031fa273)
• started building the documentation

here is the observation

• with /path/to/bin/doxygen the doc generation went much further and then encountered segmentation fault when it was about to exit the libstdc++ directory
• with gdb /path/to/bin/doxygen the doc generation completed successfully.

this was quite a intensive doc generation build with 15,000+ forks and about 2 GB of HTML docs were generated. thanks.

[saifikhan]

@doxygen have run doxygen on the full source gcc tree. there is no crash. it works. thanks for the bug fix. much appreciated !

[albert-github]

Still strange that the normal (i.e. without gdb ) run finished with a segmentation fault but that the same run with gdb ran.

• I looked at your project and Doxyfile (you specified in doxygen 1.9.1 segmentation fault when generating libstdc++ documentation #8471 (comment)) but the numbers specified by you are a bit large to me as it is more or less just the include directory plus some files.
  • 15000+ forks in your case do the, approximately, correspond with the number of dot files converted? (I have only 252 dot files converted)
  • 2GB of HTML documentation is a bit a large seen my observations about the number of files / directories beforehand (I get around 32MB)
• with "when it was about to exit the libstdc++ directory" you mean after "finished..." message or is there another last message at your console?

[saifikhan]

• 15000+ forks in your case do the, approximately, correspond with the number of dot files converted? (I have only 252 dot files converted)

gdb inferior processes is what i meant.

[saifikhan]

• 2GB of HTML documentation is a bit a large seen my observations about the number of files / directories beforehand (I get around 32MB)

please see https://dl.strikr.io/tmp/doxygen.log

[albert-github]

At the end of my gdb process I get:

[Inferior 1 (process 1888) exited normally]

so this looks to me a normal message, just indicating that gdb has no process attached anymore from which information can be extracted (my interpretation).

In the given log file I see that formulas are generated and also that directories like /strikr/src/gcc/libstdc++-v3/include/experimental are being processed. The late is not present in the Doxyfile I got. Looks like the presented Doxyfile is different from the one used (probably the file is recreated at during the CMake process).

[saifikhan]

• 2GB of HTML documentation is a bit a large seen my observations about the number of files / directories beforehand (I get around 32MB)

the 'xz' compressed gcc docs and toolchain may be downloaded from https://sourceforge.net/projects/strikr/files/gcc/

[saifikhan]

@albert-github @doxygen finally here is the GNU C++ library documentation using doxygen 1.9.2.

Please see https://ref.strikr.io/libstdcxx/api/modules.html

Here is a reasonably involved diagram https://ref.strikr.io/libstdcxx/api/a01426.png

thanks once again !