Building multi-repo docs

[xiongchiamiov]

At reddit, we have a primary open-source codebase with many plugins, most of which are also open-source, but a few of which are private. Currently, our documentation is scattered across the Github wikis for each repo, which makes browsing or searching untenable. Plus, incorporating changes to docs in the same commits as changes to the relevant code helps keep things up-to-date. So, I've been looking into alternative documentation solutions.

One option I'm considering is modifying mkdocs to support includes. So, when specifying the project layout, you might have something like this:

Foo repo

pages:
- ['index.md', 'Foo']

Bar repo

pages:
- ['index.md', 'Bar']
- @include(github.com/example/foo, 'foo')

which would end up looking as if we had dropped Foo right in:

index.html
foo-index.html

In addition to unifying all the different open-source repos, this would also allow all open-source documentation to be organized into one repo, which could then be included in an over-arching private repo that contains internal documentation as well.

Is this something you would find useful for mkdocs, or (if I move ahead with it) should I just keep it isolated in a fork?

[tomchristie]

Best to keep it in a fork for now - we're trying to keep the set of features nailed down while we work towards a really nicely polished 1.0. Thanks for the well explained request, tho.

[d0ugal]

Agree with @tomchristie. This is quite a specific usecase, it might be interesting to see how a plugin/extension API would make this possible. There is a work in progress patch to add one in #237.

At the moment it wont have all the hooks you need and it doesn't currently have a stable API. However, using it would be one option that allows you to extend without doing a full fork. Also, note that it is worth reading the history of that PR before diving in, it isn't likely to merge soon.

[meganhkelley]

@xiongchiamiov what did you end up doing to solve this (combining documentation content from multiple repositories)?

@tomchristie @d0ugal has MkDocs implemented this functionality or anything similar in the past 5 years?

[waylan]

We have added a Plugin API, which might be used to provide some sort of functionality similar to what is requested here. A list of known plugin can be found in the wiki, although I don't see anything like this. That said, there certainly have been discussions elsewhere in various issues about how to use a plugin to add external files, most recently in #1933, where I give some general pointers.

[ptbrowne]

For information, we had the same problem at Cozy and built a custom solution.

1. You can specify all the docs that should be pulled
2. Before building with mkdocs, every outside doc gets pulled via git
3. The mkdocs config is regenerated

Every outside doc can maintain a toc.yml listing the pages they want to have inside the main doc

It's pretty rough and not packaged as a plugin but the idea is there.

• The file specifying the outside docs : https://github.com/cozy/cozy.github.io/blob/dev/OUTSIDE_DOCS
• The file doing the heavy lifting of fetching and generating the config.yml : https://github.com/cozy/cozy.github.io/blob/dev/generate_config.py

[kmjungersen]

Thanks for sharing @ptbrowne. I'm interested in doing something similar as well!

[jdoiro3]

@xiongchiamiov, not sure if you're still looking for a plugin for this but I just started (this weekend) mkdocs-multirepo-plugin to add this feature.

Details

The plugin introduces the !import statement in your config's nav section. You can now use the import statement to add a documentation section, where the docs are pulled from the source repo.

nav:
  - Home: 'index.md'
  - MicroService: '!import https://github.com/{user}/{repo name}@{branch}'

If you'd prefer MkDocs to build the site nav based on the directory structure, you can define your other repos within the plugins section.

plugins:
  - multirepo:
      cleanup: True # (optional) tells multirepo to cleanup the temporary directory where other repo docs are imported to
      folder_name: multirepo_docs # (optional) tells multirepo what the temp directory should be called
      repos:
        - section: Backstage
          import_url: 'https://github.com/backstage/backstage'
        - section: Monorepo
          import_url: 'https://github.com/backstage/mkdocs-monorepo-plugin'
        - section: 'Techdocs-cli'
          import_url: 'https://github.com/backstage/techdocs-cli@main'

[kirannhegde]

Hello @jdoiro3

Isn't the plugin available on pypi server?

Also, it is my understanding that the plugin only references the already built documentation in these other repositories. It does not build the documentation in each of these other repositories. Could you kindly confirm?

Regards,
Kiran Hegde

[jdoiro3]

@kirannhegde, the plugin is not yet available on pypi. I was planning on write robust testing for the plugin before doing this but have been busy with work/school.

See my comment on another issue here for your other question.