Using Sphinx for documentation

[nilsdeppe]

Feature request:

Using Sphinx to generate the final documentation would be really nice. This is what LLVM, and HPX do nowadays. I've followed https://devblogs.microsoft.com/cppblog/clear-functional-c-documentation-with-sphinx-breathe-doxygen-cmake/ and am able to build and import Doxygen XML output, and render the result in Sphinx. However, there are some issues I've found with the way Sphinx handles template parameters. I've made minimal examples and filed bug reports upstream, sphinx-doc/sphinx#7367 and sphinx-doc/sphinx#7368. Once those are resolved we should be able to use Sphinx to wrap Doxygen documentation. The long-term goal would be to support both Doxygen HTML and Sphinx HTML, but I'm not yet sure that'll be possible. There are a few things I've noticed:

• Using breathe to import Doxygen XML works quite well. However, it uses a ridiculously slow XML parser. There is a discussion of this Memory Consumption of doxyindex & doxygenclass breathe-doc/breathe#315 and slowness breathe-doc/breathe#439 with possible solutions. By slow I mean it takes ~2 min to generate the documentation (my dream would be 2 seconds, and given some numbers from those issues that might be possible).
• There are warnings about Duplicate target detected:, e.g.

  ./docs/index.rst:59: WARNING: Duplicate declaration, template<bool... Bs>tmpl2::flat_any = std::integral_constant< bool, not std::is_same< value_list<bool, Bs...>, value_list<bool, (static_cast<void>(Bs), false)...> >::value>
  ./docs/index.rst:2: WARNING: Duplicate ID: "group__UtilitiesGroup_1ga6e25dbfa539ff4513329969dbcd81c40".
  ./docs/index.rst:2: WARNING: Duplicate explicit target name: "group__UtilitiesGroup_1ga6e25dbfa539ff4513329969dbcd81c40".
  ./docs/index.rst:59: WARNING: Duplicate declaration, template<bool... Bs>constexpr bool tmpl2::flat_all_v = flat_all<Bs...>::value
  ./docs/index.rst:2: WARNING: Duplicate ID: "group__UtilitiesGroup_1gab78389fadf763e3693ce893fad5d7fba".
  

  I have not looked into this. I'm guessing some of the are related to overloads and SFINAE-type switching.
• Using Sphinx would resolve spectre-code.org website broken on mobile #1887
• I get: /docs/index.rst: WARNING: Could not lex literal_block as "cpp". Highlighting skipped. but don't know what this is about.
• I'm not sure if we can share Markdown tutorials between Doxygen and Sphinx. It would be nice if we can since then we can support full documentation in both Sphinx and Doxygen.
• Using exhale did not work. Everything blew up horribly and it's fairly easy to replicate the part of the functionality that we would need, e.g. having a script that parses GroupDefs and generates an rst file that lists all the groups in alphabetical order.
• We will want to document the python bindings. I'm not sure if that can be done through Doxygen, but that would probably be the "best" way to handle it if we want to support both full Doxygen and Sphinx documentation.
• We will want to auto-generate all the Group documentation and
  namespace documentation. For Doxygen groups this should be easy since
  we can just simply parse GroupDefs.hpp and generate an RST file from
  that. The namespaces will be more difficult, but one option would be
  to read and parse Doxygen's XML using Beautifulsoup4 or lxml. There is
  a package, Exhale, that should generate both of these for us but I got
  errors about "multiple references for..." that as far as I can tell are
  related to overloads (and maybe template specializations). Regardless,
  at least the groups will be easy to parse, and if parsing namespaces
  from XML isn't too bad we can also parse the groups that way. We could
  even write a Sphinx extension for ourselves. My only worry with the
  Sphinx extension is that I don't know how much maintenance that'll
  require to keep working as Sphinx changes, while a simple python script
  that generates RST files would fairly be easy to maintain independent
  of what Sphinx does.

Component:

• Code
• Documentation
• Build system
• Continuous integration