Fix HTML docs generation

[michalpasztamobica]

• The ast.n file was moved to a different location causing cmake to fail to generate most of the existing documentation.
• Additionally, the unused rst files are now added to the list, so they are now also visible in the generated HTML.
• Sphinx build depends on generating all necessary rst files.
• Directories must be created before putting files into them.
• gram.lis file dependency added (thanks @bryanpkc!).
• renamed Cmake target gen_sphinx_docs to flang2_gen_sphinx_docs (to match existing flang1_gen_sphinx_docs)

@RichBarton-Arm , @kiranchandramohan , please review.

[kiranchandramohan]

@gklimowicz, @bryanpkc, @shivaramaarao : Please help with review/suggestions.

Note: In general we are interested in improving the documentation of Classic Flang upstream. We will try to share what we have.

[bryanpkc]

I needed to apply this patch before I could configure flang with the additional -DLLVM_INCLUDE_DOCS=on -DLLVM_BUILD_DOCS=on -DLLVM_ENABLE_DOXYGEN=on options:

diff --git a/docs/CMakeLists.txt b/docs/CMakeLists.txt
index ec2bfd84..e1a4ef3e 100644
--- a/docs/CMakeLists.txt
+++ b/docs/CMakeLists.txt
@@ -9,6 +9,8 @@ find_package(Sphinx)

 if (DOXYGEN_FOUND)
 if (LLVM_ENABLE_DOXYGEN)
+  add_custom_target(doxygen ALL)
+
   set(abs_srcdir ${CMAKE_CURRENT_SOURCE_DIR})
   set(abs_builddir ${CMAKE_CURRENT_BINARY_DIR})

Update: I found out that -DLLVM_BUILD_DOCS=on and -DLLVM_ENABLE_DOXYGEN=on were probably intended for an in-tree build. They are not needed for an out-of-tree build of Flang, and I got both Doxygen and Sphinx documentation to build properly with only -DLLVM_INCLUDE_DOCS=on -DLLVM_ENABLE_DOXYGEN=on, followed by make install doxygen-flang. Please ignore the above patch.

[bryanpkc]

LGTM now. Thanks @michalpasztamobica!

[bryanpkc]

@michalpasztamobica I noticed that xflag.n and xref.n are not in the documentation index, and that there are two versions of dinit.n. Would you consider merging the following patch into the PR?

Huawei-CPLLab@9c42e05

[michalpasztamobica]

Nice catch @bryanpkc , I cherry-picked and pushed your patch to this PR.

[gklimowicz]

Works fine on OpenPower LLVM 9.0 and 10.0.

[kiranchandramohan]

Thanks @michalpasztamobica for this patch. Do you know why

1. there are some stray lines at the top of the ILM Definitions HTML file. e.g. given below.

.nr IN 0 1
.de IL
.nr IN +1
.sp
.in 0
.ne 4
.nf
.ta 0.5i 4.5i
\n(IN. \$1 \$3 \$4 \$5 \$6 \$7 \$8 \$9t Type: \$2
.in 0.5i
.ad ..

2. some entries in ILM/ILI Definitions do not have entry number and are not formatted properly. For e.g. consider the following, the entry for RISNAN.

1. BOS stc1 stc2 stc3 Type: misc

Always (and only) appears as the first ILM of an ILM block. (roughly corresponds to the beginning of a source statement.)

stc1 - source statement line number (negated for insert files?).
If more than one ILM block is written for a given
statement, all but the first have 0 specified for the
line number.
For typical statements, this is actually the line
number of the terminating ‘;’.
stc2 - current source file index. Default value is 1.
stc3 - number of words for this ILM block (including the BOS).
.AT spec trm
.IL RISNAN arth lnk
.AT spec trm
...

[michalpasztamobica]

Hi @kiranchandramohan

Ad 1)
According to the nroff quick reference I found these are "number registers":

autoincrement and -decrement
.nr name value auto-increment
The auto-increment is added (or subtracted) to the register on every access where a + (or -) follows the n, as in
\n+Y or \n-(XX.

Then .de defines a macro.
Then inside the macro there's more nroff programming.
Anyway these are not "stray lines", they bring some value to the file, although I admit I don't fully comprehend what the value exactly is.

Ad 2)
I assume you are asking about the source nroff .n files? The intermediate .rst files are generated based on nroff using the n2rst tool and the end product .html files get this formatting from .rst.
By looking at the file as a whole I infer that the reason is that the file was written with an 80-character limitation rule in mind. This at least applies to all paragraphs. There are longer lines when enumerations or lists come into play, but whenever I see a paragraph the lines are always below 80 characters.

This limitation might not make much sense and it does make the generated html look a bit ugly. I think this could be taken care of by some clever sed script. However - I would strongly suggest that we make a separate PR for this, as all other changes will be lost in an ocean of line breaks removals.

[kiranchandramohan]

@shivaramaarao is the patch OK with you?

[shivaramaarao]

looks good.

[kiranchandramohan]

@michalpasztamobica please squash.

[michalpasztamobica]

@kiranchandramohan , I squashed, just leaving @bryanpkc 's commit separate to keep the authorship in order. I hope this is OK?