Allow collections to add RST files

[felixfontein]

The RST files have to be in docs/docsite/rst/ (and subdirs), need to use labels of the form ansible_collections.{namespace}.{name}.docsite. (RST files not satisfying that are ignored), and there needs to be a index.yml which allows to add sections to the main collection page and allows to say which files show up in the toctree for that section (also allows to directly reference things).

Example branches:

• ansible-collections/community.docker@main...felixfontein:scenario-guide
• https://github.com/felixfontein/ansible-tools/compare/docs

These result in:

• https://ansible.fontein.de/collections/community/docker/ (note the scenario guide!)
• https://ansible.fontein.de/collections/felixfontein/tools/ (note the first three sections on filters and tests, which are undocumentable plugin types and thus are usually not shown at all!)

There's a lint tool for collection owners (antsibull-lint collection-docs /path/to/collection), it checks RST labels and also runs rstcheck.

The PR also contains a commit which ensures that "empty" collections (which have no documentable plugins) are shown in the index, and will obtain docs. Example: https://ansible.fontein.de/collections/felixfontein/acme/ (that collection only has roles; I'll add docsite RST files for these eventually, currently I only have MD files it now has role docs as well)

[briantist]

I don't feel qualified to actually review this but it looks great. I could definitely see myself using this for c.hv!

[felixfontein]

(I've just rebased so this PR includes all other docs building PRs that were merged today. Makes it easier to test for me :) )

[felixfontein]

As discussed at the docs meeting, I've renamed index.yml to extra-docs.yml, and removed the currently not used entries code.

I've also updated the example ansible-collections/community.docker@main...felixfontein:scenario-guide, which (still) results in https://ansible.fontein.de/collections/community/docker/.

ready_for_review

[abadger]

I think we should use a name other than docsite for the names of these functions and variables. We're already using docsite to refer to building docs.ansible.com as a whole. How about one of the following:

• extra_docs
• manual_docs
• static_docs

[abadger]

I note that this is synchronous code rather than async code. The majority of time is going to be spent reading and writing the actual page data rather than the extra-docs.yml file so it's not the end of the world but it is a point where we could be doing things in parallel. I assume you coded it as synchronous for easy reuse with antsibull-lint?

[felixfontein]

The main reason is that the IO part of this function is done by load_yaml_file, which is not async either. It is also used to load routing data for collections, so eventually it should be converted to async, but once that's done adjusting the calling code is also not so hard, so I would defer adjusting this function until then. Is that ok for you?

[felixfontein]

I think we should use a name other than docsite for the names of these functions and variables. We're already using docsite to refer to building docs.ansible.com as a whole. How about one of the following:

* extra_docs

I renamed docsite to this in a3aa61d.

[abadger]

Everything looks good! Talked with felixfontein earlier and he will squash this manually so we can look at the saving-content-in-memory implementation later if we need to, then go ahead and merge it.

Thanks @felixfontein !

[felixfontein]

@abadger @briantist thanks a lot for reviewing this!
@acozine @samccann @lmalivert thanks a lot for discussing this! :)