Can't be built with LLVM 19 due to removal of generic char_traits implementation

[d125q]

Doxygen can't be built with LLVM 19 because there's no char_traits<unsigned char>. (See D157058.) Using C++20 with char8_t here and here makes it build, but there are some warnings from the logger about using char8_t * with an %s format specifier.

[albert-github]

• Which version of doxygen did you try to build?
• probably relevant reference: https://en.cppreference.com/w/cpp/string/char_traits
• "logger"
  • with the "logger" I assume you mean the deps/spdlog
  • what are the warnings, are they about Char (i.e. with capital C)?
    if so I think you should file a report at https://github.com/gabime/spdlog/issues

[d125q]

Sorry for not providing a whole lot of information in the first place.

I am building from HEAD; commit 3ce1c91 at the time of writing. The error is:

$LLVM_PREFIX/include/c++/v1/string:780:43: error: implicit instantiation of undefined template 'std::char_traits<unsigned char>'

Applying the following patch

diff --git a/CMakeLists.txt b/CMakeLists.txt
index f1103d6..af394ec 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -81,8 +81,8 @@ if (build_wizard)
     endif()
 endif()
 
-# use C++17 standard for compiling
-set(CMAKE_CXX_STANDARD 17)
+# use C++20 standard for compiling
+set(CMAKE_CXX_STANDARD 20)
 set(CMAKE_CXX_STANDARD_REQUIRED ON)
 set(CMAKE_CXX_EXTENSIONS ON)
 
@@ -165,7 +165,7 @@ if (CMAKE_SYSTEM_NAME MATCHES "Windows")
 endif()
 
 # needed for JavaCC
-set(JAVA_CC_EXTRA_FLAGS "-DJAVACC_CHAR_TYPE=\"unsigned char\"")
+set(JAVA_CC_EXTRA_FLAGS "-DJAVACC_CHAR_TYPE=\"char8_t\"")
 set(CMAKE_CXX_FLAGS       "${CMAKE_CXX_FLAGS}       ${JAVA_CC_EXTRA_FLAGS}")
 set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} ${JAVA_CC_EXTRA_FLAGS}")

diff --git a/src/qcstring.h b/src/qcstring.h
index 5e049c2..b02bce0 100644
--- a/src/qcstring.h
+++ b/src/qcstring.h
@@ -92,7 +92,7 @@ inline int qstricmp_sort( const char *str1, const char *str2 )
 
 int qstrnicmp( const char *str1, const char *str2, size_t len );
 
-using JavaCCString = std::basic_string<unsigned char>;
+using JavaCCString = std::basic_string<char8_t>;
 
 /** This is an alternative implementation of QCString. It provides basically
  *  the same functions but uses std::string as the underlying string type

makes things work again; however, there are a couple of warnings since formatting char8_t * with %s is undefined behavior:

[257/268] Building CXX object src/CMakeFiles/doxymain.dir/vhdljjparser.cpp.o
In file included from src/vhdljjparser.cpp:33:
vhdlparser/VhdlParserErrorHandler.hpp:23:61: warning: format specifies type 'char *' but the argument has type 'value_type *' (aka 'char8_t *') [-Wformat]
   23 |       warn(m_fileName,actual->beginLine,"syntax error '%s'",actual->image.data());
      |                                                        ~~   ^~~~~~~~~~~~~~~~~~~~
      |                                                        %s
src/message.h:61:48: note: expanded from macro 'warn'
   61 |    warn_(removeLongPathMarker(file),line,fmt,##__VA_ARGS__); \
      |                                          ~~~   ^~~~~~~~~~~
In file included from src/vhdljjparser.cpp:33:
vhdlparser/VhdlParserErrorHandler.hpp:30:65: warning: format specifies type 'char *' but the argument has type 'value_type *' (aka 'char8_t *') [-Wformat]
   30 |       warn(m_fileName,last->beginLine,"unexpected token: '%s'", unexpected->image.data());
      |                                                           ~~    ^~~~~~~~~~~~~~~~~~~~~~~~
      |                                                           %s
src/message.h:61:48: note: expanded from macro 'warn'
   61 |    warn_(removeLongPathMarker(file),line,fmt,##__VA_ARGS__); \
      |                                          ~~~   ^~~~~~~~~~~
2 warnings generated.

See for example another project which was affected by the removal of the generic char_traits implementation: jtv/libpqxx#726.

[albert-github]

Thanks for the clarification

• your mentioning of the "logger" is in fact a problem with the doxygen warn / warn_ functions and has nothing to do with the spdlog
• You set set(CMAKE_CXX_STANDARD 20), doxygen attempts to support a bit older compilers as well. What is the reason for setting the standard to 20 and what happens when you leave it on 17?

[d125q]

My impression was that char8_t only exists in C++20, that's why there's set(CMAKE_CXX_STANDARD 20).

[albert-github]

Indeed the char8_t is first mentioned in the foreword of the cpp-20 standard as "addition of char8_t type".
I think it would be better to replace the unsigned char with char, might also solve the "warn" problem, maybe you can test this (I don't have a LLVM-19 compiler available).

[doxygen]

Note that the use of unsigned char instead of just char was introduced as a part of this fix #10104. Switching to signed char will probably reintroduce this issue.

[albert-github]

Indeed I also think that switching back to char is not a good idea. The char8_t is OK as it represents a "unsigned char"
In the C++-20 standard see e.g. "6.8.1 Fundamental types" or the stack overflow answer https://stackoverflow.com/a/57402487/1657886

I did a small test setting both the unsigned char to char8_t and the C++ version to 20.
With the MSCV compiler:

Microsoft (R) C/C++ Optimizing Compiler Version 19.28.29914 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

I a problem:

deps\TinyDeflate\gunzip.hh(438): error C2039: 'result_of': is not a member of 'std'
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\include\iterator(21): note: see declaration of 'std'
...\deps\TinyDeflate\gunzip.hh(438): note: see reference to class template instantiation 'gunzip_ns::result_of<F(Args...)>' being compiled
...\deps\TinyDeflate\gunzip.hh(438): error C2955: 'gunzip_ns::result_of': use of class template requires template argument list
...\deps\TinyDeflate\gunzip.hh(429): note: see declaration of 'gunzip_ns::result_of'
...\deps\TinyDeflate\gunzip.hh(438): error C2143: syntax error: missing ',' before '<'
...\deps\TinyDeflate\gunzip.hh(1421): error C2338: Deflate: Mismatched parameters. Expected something for an input.

(might be fixed with newer compilers though)

With the gcc compiler on Cygwin (gcc version 11.4.0), I got besides the mentioned problems with "%s" etc.

In file included from /usr/lib/gcc/x86_64-pc-cygwin/11/include/c++/x86_64-pc-cygwin/bits/c++allocator.h:33,
                 from /usr/lib/gcc/x86_64-pc-cygwin/11/include/c++/bits/allocator.h:46,
                 from /usr/lib/gcc/x86_64-pc-cygwin/11/include/c++/string:41,
                 from /usr/lib/gcc/x86_64-pc-cygwin/11/include/c++/bits/locale_classes.h:40,
                 from /usr/lib/gcc/x86_64-pc-cygwin/11/include/c++/bits/ios_base.h:41,
                 from /usr/lib/gcc/x86_64-pc-cygwin/11/include/c++/ios:42,
                 from /usr/lib/gcc/x86_64-pc-cygwin/11/include/c++/istream:38,
                 from /usr/lib/gcc/x86_64-pc-cygwin/11/include/c++/sstream:38,
                 from .../src/dotlegendgraph.cpp:16:
In member function ‘void __gnu_cxx::new_allocator<_Tp>::deallocate(_Tp*, __gnu_cxx::new_allocator<_Tp>::size_type) [with _Tp = char]’,
    inlined from ‘constexpr void std::allocator< <template-parameter-1-1> >::deallocate(_Tp*, std::size_t) [with _Tp = char]’ at /usr/lib/gcc/x86_64-pc-cygwin/11/include/c++/bits/allocator.h:199:35,
    inlined from ‘void std::basic_string<_CharT, _Traits, _Alloc>::_Rep::_M_destroy(const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]’ at /usr/lib/gcc/x86_64-pc-cygwin/11/include/c++/bits/basic_string.tcc:880:39,
    inlined from ‘void std::basic_string<_CharT, _Traits, _Alloc>::_Rep::_M_dispose(const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]’ at /usr/lib/gcc/x86_64-pc-cygwin/11/include/c++/bits/basic_string.h:3352:15,
    inlined from ‘void std::basic_string<_CharT, _Traits, _Alloc>::_Rep::_M_dispose(const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]’ at /usr/lib/gcc/x86_64-pc-cygwin/11/include/c++/bits/basic_string.h:3332:2,
    inlined from ‘std::basic_string<_CharT, _Traits, _Alloc>::~basic_string() [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]’ at /usr/lib/gcc/x86_64-pc-cygwin/11/include/c++/bits/basic_string.h:3768:29,
    inlined from ‘QCString::~QCString()’ at .../src/qcstring.h:100:7,
    inlined from ‘void DotLegendGraph::writeGraph(const QCString&)’ at .../src/dotlegendgraph.cpp:32:27:
/usr/lib/gcc/x86_64-pc-cygwin/11/include/c++/ext/new_allocator.h:145:26: warning: ‘void operator delete(void*, std::size_t)’ called on a pointer to an unallocated object ‘18446744073709551592’ [-Wfree-nonheap-object]
  145 |         ::operator delete(__p
      |         ~~~~~~~~~~~~~~~~~^~~~
  146 | #if __cpp_sized_deallocation
      | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  147 |                           , __t * sizeof(_Tp)
      |                           ~~~~~~~~~~~~~~~~~~~
  148 | #endif
      | ~~~~~~
  149 |                          );

With the Cygwin version I could run and the results (for issue #10104) were OK.

So in fact we have multiple problems regarding the compilers:

• LLVM 19 does not support char_traits(unsigned char) in neither C++17 nor C++20 mode (@d125q correct?)
• char8_t is supported from C++-20
  GitHub actions uses for windows:

   -- The C compiler identification is MSVC 19.39.33523.0
   -- The CXX compiler identification is MSVC 19.39.33523.0
  

  for Ubuntu (GCC):

  -- The C compiler identification is GNU 9.4.0
  -- The CXX compiler identification is GNU 9.4.0
  

  for Ubuntu (Clang):

  -- The C compiler identification is Clang 10.0.0
  -- The CXX compiler identification is Clang 10.0.0
  

  for MacOS:

  -- The C compiler identification is AppleClang 15.0.0.15000040
  -- The CXX compiler identification is AppleClang 15.0.0.15000040
  

  This means that, as in most of the cases we are dependent on what is available, we have to be careful.
• on Cygwin GCC 11.4.0 gives some warnings

One independent improvement I can think of is that the double definition (CMakeLists.txt and qcstring.h should be aligned.

[albert-github]

Regarding the small improvement mentioned at the end of #10928 (comment) I've just pushed a proposed patch, pull request #10929

[albert-github]

Regarding the small improvement mentioned at the end of #10928 (comment) the code has been integrated in master on GitHub.

[doxygen]

I guess now we can change the CMakeLists.txt to detect LLVM 19+ (like this or is there a better way?) and then switch the language to C++20 and set the set(JAVA_CC_EXTRA_FLAGS "-DJAVACC_CHAR_TYPE=\"char8_t\"") flag.

Enabling C++20 unconditionally will probably break older platforms/compilers.