gives up after seeing egregiously complex template code

[ec1oud]

Describe the bug
Generating docs for compound QtPrivate::IsContainerCompatibleWithModelRoleDataSpan< T, std::enable_if_t< std::conjunction_v< std::is_convertible< decltype(std::data(std::declval< T & >())), QModelRoleData * >, std::is_convertible< decltype(std::size(std::declval< T & >())), qsizetype >, std::is_convertible< typename std::iterator_traits< decltype(std::begin(std::declval< T & >()))>::value_type, QModelRoleData >, std::is_convertible< decltype(std::begin(std::declval< T & >()) !=std::end(std::declval< T & >())), bool >, std::negation< std::is_same< std::decay_t< T >, QModelRoleDataSpan > > > > >...
Exiting...

To Reproduce
https://codereview.qt-project.org/c/qt/qt5/+/354231 again, but this time modified to reduce the scope:

-INPUT                  = .
+INPUT                  = qtbase/src qtdeclarative/src qtwebengine/src/pdf qtwebengine/src/pdfquick qtwebengine/src/pdfwidgets

Version
1.8.20 (downgraded because 1.9.3 is crashing much earlier than that)

[albert-github]

It is not good that doxygen crashes, but some crashes were fixed in the current master (also some significant speedup improvements) so best to have a go with the current master and report back.

You wrote "https://codereview.qt-project.org/c/qt/qt5/+/354231 again, but this time modified to reduce the scope:" what again (I know that you probably meant the procedure described in #9295 (as the 2 issues were submitted nearly at the same time),but it should be specified here and not just vaguely mentioned.

• it would be very useful to have a stack trace (of the master version)
• Can you please attach a, small, self contained example (source+configuration file in a tar or zip) that allows us to reproduce the problem? Please don't add external links as they might not be persistent.

[albert-github]

I tried to run with the current master and in the doxygen.log file I saw that the directory ../doc/dev/qt didn't exists and could not be created, so I created it. Now it is reading the files....

[albert-github]

Some further remarks.

I got the message:

warning: tag INPUT: input source 'qtwebengine/src/pdf' does not exist
warning: tag INPUT: input source 'qtwebengine/src/pdfquick' does not exist
warning: tag INPUT: input source 'qtwebengine/src/pdfwidgets' does not exist

I don't think it of influence here.

The steps I executed:

git clone https://code.qt.io/qt/qt5.git
cd qt5/
git checkout 6.0
perl init-repository

place the qt.doxy in the current directory and change the INPUT tag as specified

doxygen qt.doxy

• What is the problem with the qtwebengine/src/... and do I need them in this stage (how can I get them when necessary)? (doesn't look like to be necessary)

In my run it stops at:

Generating docs for compound QtPrivate::IsContainerCompatibleWithModelRoleDataSpan< T, std::enable_if_t< std::conjunction_v< std::is_convertible< decltype(std::data(std::declval< T & >())), QModelRoleData * >, std::is_convertible< decltype(std::size(std::declval< T & >())), qsizetype >, std::is_convertible< typename std::iterator_traits< decltype(std::begin(std::declval< T & >()))>::value_type, QModelRoleData >, std::is_convertible< decltype(std::begin(std::declval< T & >()) !=std::end(std::declval< T & >())), bool >, std::negation< std::is_same< std::decay_t< T >, QModelRoleDataSpan > > > > >...

and when I give (after a while) a ^Z to stop the process and request a stack trace I get:

Thread 1 "doxygen" received signal SIGTSTP, Stopped (user).
0x00007ffad828d8c4 in ntdll!ZwWaitForMultipleObjects () from /cygdrive/c/WINDOWS/SYSTEM32/ntdll.dll
(gdb) where
#0  0x00007ffad828d8c4 in ntdll!ZwWaitForMultipleObjects () from /cygdrive/c/WINDOWS/SYSTEM32/ntdll.dll
#1  0x00007ffad5f1cb10 in WaitForMultipleObjectsEx () from /cygdrive/c/WINDOWS/System32/KERNELBASE.dll
#2  0x00007ffad5f1ca0e in WaitForMultipleObjects () from /cygdrive/c/WINDOWS/System32/KERNELBASE.dll
#3  0x000000018004794a in cygwait(void*, _LARGE_INTEGER*, unsigned int) () from /usr/bin/cygwin1.dll
#4  0x000000018015680d in pthread::join(pthread**, void**, _LARGE_INTEGER*) () from /usr/bin/cygwin1.dll
#5  0x0000000180156b16 in pthread_join () from /usr/bin/cygwin1.dll
#6  0x000000018018fbdb in _sigfe () from /usr/bin/cygwin1.dll
#7  0x000000037d763047 in cygstdc++-6!_ZNSt6thread4joinEv () from /usr/bin/cygstdc++-6.dll
#8  0x00000001007cdbf0 in DotWorkerThread::wait (this=0x838707c60) at /cygdrive/d/Programs/Doxygen/Doxygen-.git/doxygen/src/dotrunner.h:94
#9  DotWorkerThread::~DotWorkerThread (this=this@entry=0x838707c60, __in_chrg=<optimized out>) at /cygdrive/d/Programs/Doxygen/Doxygen-.git/doxygen/src/dotrunner.cpp:295
#10 0x00000001007a6e23 in std::default_delete<DotWorkerThread>::operator() (__ptr=0x838707c60, this=<optimized out>) at /usr/lib/gcc/x86_64-pc-cygwin/11/include/c++/bits/unique_ptr.h:85
#11 std::unique_ptr<DotWorkerThread, std::default_delete<DotWorkerThread> >::~unique_ptr (this=0x81594f8e0, __in_chrg=<optimized out>) at /usr/lib/gcc/x86_64-pc-cygwin/11/include/c++/bits/unique_ptr.h:361
#12 std::_Destroy<std::unique_ptr<DotWorkerThread, std::default_delete<DotWorkerThread> > > (__pointer=0x81594f8e0) at /usr/lib/gcc/x86_64-pc-cygwin/11/include/c++/bits/stl_construct.h:140
#13 std::_Destroy_aux<false>::__destroy<std::unique_ptr<DotWorkerThread, std::default_delete<DotWorkerThread> >*> (__last=<optimized out>, __first=0x81594f8e0)
    at /usr/lib/gcc/x86_64-pc-cygwin/11/include/c++/bits/stl_construct.h:152
#14 std::_Destroy<std::unique_ptr<DotWorkerThread, std::default_delete<DotWorkerThread> >*> (__last=<optimized out>, __first=<optimized out>) at /usr/lib/gcc/x86_64-pc-cygwin/11/include/c++/bits/stl_construct.h:185
#15 std::_Destroy<std::unique_ptr<DotWorkerThread, std::default_delete<DotWorkerThread> >*, std::unique_ptr<DotWorkerThread, std::default_delete<DotWorkerThread> > > (__last=0x81594f928, __first=<optimized out>)
    at /usr/lib/gcc/x86_64-pc-cygwin/11/include/c++/bits/alloc_traits.h:746
#16 std::vector<std::unique_ptr<DotWorkerThread, std::default_delete<DotWorkerThread> >, std::allocator<std::unique_ptr<DotWorkerThread, std::default_delete<DotWorkerThread> > > >::~vector (
    this=0x1013dd350 <DotManager::instance()::theInstance+112>, __in_chrg=<optimized out>) at /usr/lib/gcc/x86_64-pc-cygwin/11/include/c++/bits/stl_vector.h:680
#17 DotManager::~DotManager (this=0x1013dd2e0 <DotManager::instance()::theInstance>, __in_chrg=<optimized out>) at /cygdrive/d/Programs/Doxygen/Doxygen-.git/doxygen/src/dot.cpp:110
#18 0x00000001801f8996 in __call_exitprocs () from /usr/bin/cygwin1.dll
#19 0x00000001801ade60 in exit () from /usr/bin/cygwin1.dll
#20 0x0000000180048403 in cygwin_exit () from /usr/bin/cygwin1.dll
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

@doxygen do you have an idea?
(executable used: 1.9.4 (a7daf77)).

@ec1oud An idea might also be to look into the setting of NUM_PROC_THREADS (for the master version).

[albert-github]

@doxygen I just did a quick rerun by setting:

INPUT                  = qtbase/src/corelib/itemmodels

and it looks like doxygen hangs at the same place (but gets to that place very fast).

FYI the used qt.doxy: qt.dox.tar.gz

[doxygen]

@albert-github Looks like doxygen tries to terminate because an error writing a man page file (maybe due to long name):

    frame #21: 0x00000001006fefad doxygen`term(fmt="Could not open file %s for writing\n") at message.cpp:260:3 [opt]
    frame #22: 0x0000000100582d03 doxygen`OutputGenerator::startPlainFile(this=0x0000000102b08bb0, name=<unavailable>) at outputgen.cpp:67:5 [opt]
    frame #23: 0x0000000100530458 doxygen`ManGenerator::startFile(this=0x0000000102b08bb0, (null)=<unavailable>, manName=<unavailable>, (null)=<unavailable>, (null)=<unavailable>) at mangen.cpp:168:3 [opt]

and while terminating it get stuck waiting for a dot runner to finish:

  * frame #0: 0x00007fff6df5f55e libsystem_kernel.dylib`__ulock_wait + 10
    frame #1: 0x00007fff6e0225c2 libsystem_pthread.dylib`_pthread_join + 347
    frame #2: 0x00007fff6b1245e6 libc++.1.dylib`std::__1::thread::join() + 24
    frame #3: 0x00000001002c23d0 doxygen`DotWorkerThread::~DotWorkerThread() [inlined] DotWorkerThread::wait(this=<unavailable>) at dotrunner.h:94:29 [opt]
    frame #4: 0x00000001002c23c8 doxygen`DotWorkerThread::~DotWorkerThread() [inlined] DotWorkerThread::~DotWorkerThread(this=0x0000000101e3b050) at dotrunner.cpp:295 [opt]
    frame #5: 0x00000001002c23ba doxygen`DotWorkerThread::~DotWorkerThread(this=0x0000000101e3b050) at dotrunner.cpp:292 [opt]
    frame #6: 0x000000010029b522 doxygen`DotManager::~DotManager() [inlined] std::__1::default_delete<DotWorkerThread>::operator(this=<unavailable>, __ptr=0x0000000101e3b050)(DotWorkerThread*) const at memory:2368:5 [opt]
    frame #7: 0x000000010029b51a doxygen`DotManager::~DotManager() [inlined] std::__1::unique_ptr<DotWorkerThread, std::__1::default_delete<DotWorkerThread> >::reset(this=<unavailable>, __p=0x0000000000000000) at memory:2623 [opt]
    frame #8: 0x000000010029b515 doxygen`DotManager::~DotManager() [inlined] std::__1::unique_ptr<DotWorkerThread, std::__1::default_delete<DotWorkerThread> >::~unique_ptr(this=<unavailable>) at memory:2577 [opt]
    frame #9: 0x000000010029b515 doxygen`DotManager::~DotManager() [inlined] std::__1::unique_ptr<DotWorkerThread, std::__1::default_delete<DotWorkerThread> >::~unique_ptr(this=<unavailable>) at memory:2577 [opt]
    frame #10: 0x000000010029b515 doxygen`DotManager::~DotManager() [inlined] std::__1::allocator<std::__1::unique_ptr<DotWorkerThread, std::__1::default_delete<DotWorkerThread> > >::destroy(this=<unavailable>, __p=<unavailable>) at memory:1936 [opt]
    frame #11: 0x000000010029b515 doxygen`DotManager::~DotManager() [inlined] void std::__1::allocator_traits<std::__1::allocator<std::__1::unique_ptr<DotWorkerThread, std::__1::default_delete<DotWorkerThread> > > >::__destroy<std::__1::unique_ptr<DotWorkerThread, std::__1::default_delete<DotWorkerThread> > >(__a=<unavailable>, __p=<unavailable>) at memory:1798 [opt]
    frame #12: 0x000000010029b515 doxygen`DotManager::~DotManager() [inlined] void std::__1::allocator_traits<std::__1::allocator<std::__1::unique_ptr<DotWorkerThread, std::__1::default_delete<DotWorkerThread> > > >::destroy<std::__1::unique_ptr<DotWorkerThread, std::__1::default_delete<DotWorkerThread> > >(__a=<unavailable>, __p=<unavailable>) at memory:1635 [opt]
    frame #13: 0x000000010029b515 doxygen`DotManager::~DotManager() [inlined] std::__1::__vector_base<std::__1::unique_ptr<DotWorkerThread, std::__1::default_delete<DotWorkerThread> >, std::__1::allocator<std::__1::unique_ptr<DotWorkerThread, std::__1::default_delete<DotWorkerThread> > > >::__destruct_at_end(this=0x0000000100fe3aa8, __new_last=0x0000000101e563c0) at vector:426 [opt]
    frame #14: 0x000000010029b4f8 doxygen`DotManager::~DotManager() [inlined] std::__1::__vector_base<std::__1::unique_ptr<DotWorkerThread, std::__1::default_delete<DotWorkerThread> >, std::__1::allocator<std::__1::unique_ptr<DotWorkerThread, std::__1::default_delete<DotWorkerThread> > > >::clear(this=0x0000000100fe3aa8) at vector:369 [opt]
    frame #15: 0x000000010029b4f8 doxygen`DotManager::~DotManager() [inlined] std::__1::__vector_base<std::__1::unique_ptr<DotWorkerThread, std::__1::default_delete<DotWorkerThread> >, std::__1::allocator<std::__1::unique_ptr<DotWorkerThread, std::__1::default_delete<DotWorkerThread> > > >::~__vector_base(this=0x0000000100fe3aa8) at vector:463 [opt]
    frame #16: 0x000000010029b4e1 doxygen`DotManager::~DotManager() [inlined] std::__1::vector<std::__1::unique_ptr<DotWorkerThread, std::__1::default_delete<DotWorkerThread> >, std::__1::allocator<std::__1::unique_ptr<DotWorkerThread, std::__1::default_delete<DotWorkerThread> > > >::~vector(this=0x0000000100fe3aa8 size=17) at vector:555 [opt]
    frame #17: 0x000000010029b4e1 doxygen`DotManager::~DotManager() [inlined] std::__1::vector<std::__1::unique_ptr<DotWorkerThread, std::__1::default_delete<DotWorkerThread> >, std::__1::allocator<std::__1::unique_ptr<DotWorkerThread, std::__1::default_delete<DotWorkerThread> > > >::~vector(this=0x0000000100fe3aa8 size=17) at vector:550 [opt]
    frame #18: 0x000000010029b4e1 doxygen`DotManager::~DotManager(this=0x0000000100fe3a68) at dot.cpp:110 [opt]
    frame #19: 0x00007fff6dec613c libsystem_c.dylib`__cxa_finalize_ranges + 319
    frame #20: 0x00007fff6dec6412 libsystem_c.dylib`exit + 55
    frame #21: 0x00000001006fefad doxygen`term(fmt="Could not open file %s for writing\n") at message.cpp:260:3 [opt]

Setting GENERATE_MAN to NO works around the problem.

[albert-github]

@doxygen That would indeed be the explanation as the string is very long and, by head, the man pages take the name shown as the name for the file.

• Is there a possibility to shorten the name for the man pages or have something like an "alias"?
• Would it be an idea to replace the term here with an err?
• is there a way to signal to the dotrunner / dotManager that it should terminate as well?

[doxygen]

@albert-github The referenced commit addresses the issue where doxygen would hang after termination. It does not address the cause of termination (i.e. using a too long file name).

The name of the file is used to refer to the man page, i.e. man ClassX will look for e.g. the file man/man3/ClassX.3, so the name should match the class. With template classes this is anyway a bit of a problem, so a better solution for man pages is needed. Not sure how/if this is done when writing the man pages by hand. I see this repo for STL which could serve as inspiration.

[ec1oud]

Aha. Writing man pages isn't so important to me anyway; that part was just a "what if" experiment that I forgot about. Yes I can imagine it's hard to name them when templates are involved, but if we wanted a set of man pages for Qt I guess we'd limit it to public API anyway, and exclude anything else that's problematic.

BTW git checkout 6.0 should be git checkout dev. 6.0 is already obsolete, and we were not shipping qtwebengine at that time (porting took longer).

Anyway if you can make it work with INPUT = . that's even better; I just reduced the file set to work around #9295.

[albert-github]

• good that doxygen doesn't hang anymore on the dot thread.
  can this happen in other parts as well ? I'm thinking about plantuml (and other parts where Managers are used).
• I think that the original man pages were designed there was probably no notion of anything like a template (maybe not even about a class), maybe only in some theoretical papers. So it might be very well that there are some problems in that area. I think it would be good to study the results of the mentioned link, although it just parses some html pages to man pages, and probably just those specific ones, but we might get some inspiration from it.
• @ec1oud good that you don't really need the man pages and when needed you probably only need the public parts that probably won't incorporate this problem. Still it is something that from the doxygen perspective has to be looked into.