Using README.md as directory description.

[albert-github]

Discussed in #11176

^{Originally posted by equetzal October 9, 2024}
So, I've been working on setting doxygen on a large project, and I have encountered an interesting situation where I'm not sure how to solve. This project has lot of directories, where each directory contains projects, libraries, components, etc. Each directory does have a README.md file that briefly describes the content of the directory. This means that my project has more than 50 readme files.

Usually, doxygen process all the README.md files as a "page", so it creates an entry in the related pages using the title of the H1 header. So, for Doxygen to process this markdown file as a directory description I've been adding \dir to the markdown file; something like this:

\dir
# Some Title
Some description of this directory.

This properly archives the result of making the content of the readme file to be included in the directory description. However, it keeps the 'page' in the related pages with the title README and its content being blank. This result in over 50 blank pages called "README", which I cannot \subpage because there's no anchor/reference to it.

Other solution that I tried was to use the USE_MDFILE_AS_MAINPAGE set to README.md, this kind of 'merge' all the contents of the readme files into a single page, and because of the \dir directive, the result is not having the "README" pages in the related pages, while keeping the directory description. But the problem with this is that the actual mainpage gets discarded as multiple \mainpage directives are found, and my index.html is basically blank, which is also not ideal for the use that we have for the landing page.

Is there a way to achieve the result of processing all the README files as directory-level descriptions, while keeping a specific markdown file for the index?

[albert-github]

Comment from #11176 (comment)

So I've attached the examples to you. I set (readme) in the titles the README.md files to differentiate them.
doxygen_example.zip

doxygen -v 
1.9.7

So if I run doxygen as usual, without USE_MDFILE_AS_MAINPAGE = README.md and no README file having the \dir
command this is the output I get:

• Directory without description
• A 'page' with the content of the README file
  
  

If I add \dir at the beginning of the README files.

• The directory view uses the README content as the "Detailed Description"
• The 'pages' that used to have the title of the README are now titled "README" but are now blank
• The index is still intact, using index.md as is the one with the \mainpage
  
  

If I add USE_MDFILE_AS_MAINPAGE = README.md on top of the previous step

• The "README" pages disappear, while the directory view still uses the content as the "Detailed description"
• Now the index is gone, with warnings like (example), for each README file.

Search for main page...
D:/doxygen_example/example_readme_as_mainpage/doxygen_input/awesome_ui_component/README.md:1: warning: found more than one \mainpage comment block! (first occurrence: D:/doxygen_example/example_readme_as_mainpage/doxygen_input/astonishing_feature_component/README.md, line 1), Skipping current block!

Ideally..

The README 'pages' wouldn't appear in the related pages (Either because they are a README file or because their post-process content is empty), the content of the README with the\dir would go in each directory detailed description, and the single file with the \mainpage tag, should still be the index file.

[albert-github]

I think the point under ideally summarizes quite well what should be the case, i.e. when no documentation on a page remains it should not be shown at all (maybe with the exception when an explicit page name is given with the \page command)

(@equetzal unfortunately the conversion from discussion to issue lost the original submitter)

[doxygen]

I agree that a hierarchy of README.md files are not currently processed in a meaningful way. I'll see how I can improve this. Ideally, even the \dir should not be needed.

[albert-github]

I think that the \dir should be present (probably best in the form of <!--! \dir -->) as otherwise there might be problems with existing projects where there is a README.md present, in a subdirectory, with just a general description of the project.

[doxygen]

@albert-github Please check if the referenced commit correctly implements the proposal. I've added a new option IMPLICIT_DIR_DOCS to switch between needing an explicit \dir command or not.

[albert-github]

I didn't do actual checking but I see:

    <option type='bool' id='IMPLICIT_DIR_DOCS' defval='1'>

form a compatibility point of view I think it is better to have defval='0' especially to prevent problems with project that already use a README.txt as also indicated in #11179 (comment)

[doxygen]

@albert-github I did make the default deliberately incompatible, because I think the new behavior is better and more natural for all cases I could think of. If it is not enabled by default, people would first have to discover this option. But if you have an obvious use-case or know a project that depends on the option being disabled, then let me know and I'll reconsider.

[albert-github]

The new usage is probably a bit more naturally, but compatibility is something important and hence my remark.
Probably users that face the problem with the README.md files will have used the USE_MDFILE_AS_MAINPAGE setting or the \mainpage command

I checked the CGAL project and they have some README.md files but it does not look like it influences the output.
I think there might be cases but I have to think about it.

[equetzal]

If I can add my grain of salt, I think open source projects should not be driven by retro-compatibility and should follow what is best for the project. Having retro-compatibility in mind always makes the codebase more complicated to preserve old behaviors and in reality it's very hard to know how many impacted users you might have with a breaking change without feedback or telemetry. So, I would opt to also make it the default value, but it's up to the project lead. :D

A possible intermediate solution could be:

• If doxygen get's it's upgrading an older doxyfile, then the value set for this new IMPLICIT_DIR_DOCS flag is deactivated.
• If doxygen is creating a new config file, by default have the IMPLICIT_DIR_DOCS flag activated.

[equetzal]

@albert-github @doxygen, following up on this, I've been using the feature, and I definitely feel that the results are more consistent and match with what the user would expect to happen.

I want to add on this that something interesting I noticed is that I was expecting to be different the how readme titles are managed. So, suppose I have a directory like:

.
├── cfgmgr
│   └── README.md
└── mbs
    └── README.md

Then, my readme files would look something like:

# ProjectName Configuration Manager
This shalala directory does X, Y and Z. 

# ProjectName Modular Build System (MBS)
This component is in charge of X and Y for doing Z and shalala

So, in this scenario the title is really important to explain the naming of the directory acronym, usually showing the name of the component as the main title. However, doxygen renders the markdown file by removing the title from the detailed description of the directory, resulting in pages that look like:

mbs Directory Reference

Directories

└── mbs
└── README.md

Detailed Description

This component is in charge of X and Y for doing Z and shalala

Then the title is completely loss. I would expect for the directory readmes for this not to be the behavior and instead to preserve the title, even if it's not the bigger size-letter, but to stay present on the doc.

[albert-github]

@equetzal

With the limited info I cannot reproduce your problem from #11179 (comment), please show your settings and put the code in a compressed tar or zip file, so:

• Can you please attach a, small, self contained example (source+configuration file in a, compressed, tar or zip file!) that allows us to reproduce the problem?
  Please don't add external links as they might not be persistent (also references to GitHub repositories are considered non persistent).