Strategy for including docs from repository root

[c-mart]

The project that I work on has a few of the most important docs (like README.md and contributing.md) in the root directory of the git repository, and a bunch of other docs in a docs/ directory. I want to use all of these as source files for our MkDocs site.

I apologize for creating the N+1th issue about this, but I have a combination of:

• Inability to move our root-level docs to a subdirectory, because we're using GitLab instead of GitHub. Some projects move README.md to docs/, and make a symlink at README.md targeting /docs/README.md. GitHub's web UI traverses this symlink (showing people the contents of the target), but GitLab's web UI does not, so it would break any existing links to (e.g.) https://gitlab.com/exosphere/exosphere/-/blob/master/README.md, and show a mostly empty page when people browse to https://gitlab.com/exosphere/exosphere.
• README.md contains relative links to other docs, like docs/run-exosphere.md. I want these links to continue working in GitLab/GitHub's Web UI, but also work on the site generated by MkDocs.

Noting my solution to help others. Feel free to close this issue.

What Worked

In your repository, create a mkdocs/ folder structure with symlinks like this:

.
├── docs/              # Most of your docs live here
│   └── foobar.md      
├── mkdocs/
│   ├── mkdocs.yml
│   └── src/
│       ├── docs/      # This is a symlink to `../../docs`
│       └── README.md  # This is a symlink to `../../README.md`
└── README.md

Now, the structure of mkdocs/src/ mirrors the structure of the repo. You can set your docs_dir to src.

That's it! No extensions needed. Links inside README.md targeting docs/foobar.md continue to work on the MkDocs site, and on the repo site (GitLab/GitHub).

The only case where this may break is when developing on a Windows machine (which doesn't respect symlinks) -- fortunately not a problem for us. If you have developers on Windows, you could use this overall structure with snippets extension (per below) instead of symlinks.

Things that Didn't Work

mkdocs.yml in repo root, and docs_dir set to repo root (.)

This breaks mkdocs build for two reasons:

Config value 'docs_dir': The 'docs_dir' should not be the parent directory of the config file. Use a child directory instead so that the 'docs_dir' is a sibling of the config file.

Config value 'site_dir': The 'site_dir' should not be within the 'docs_dir' as this leads to the build directory being copied into itself and duplicate nested files in the 'site_dir'.

Since there's no higher-level directory in which to place mkdocs.yml, this is a dead end.

mkdocs.yml in repo root, docs_dir set to docs/, with nav containing relative paths to (e.g.) ../README.md

This breaks mkdocs build and mkdocs serve:

a relative path to '../README.md' is included in the 'nav' configuration, which is not found in the documentation files

So nav: cannot specify relative paths to files outside the docs_dir.

mkdocs.yml in repo root, docs_dir set to docs/, with docs/ containing symlinks targeting (e.g.) ../README.md (per #1279)

This was promising: MkDocs renders README.md. The problem is that README.md contains several links to pages inside the docs/ directory, e.g.:

[How to run Exosphere](docs/run-exosphere.md)

These links need to work on both the repo site (GitLab/GitHub) and on our MkDocs site. Alas, a link target of docs/run-exosphere.md doesn't work on the MkDocs site, and a target of run-exosphere.md doesn't work on the repo site. @cruisercoder also got stuck on this here.

mkdocs.yml in repo root, docs_dir set to docs/, with snippets extension to embed README.md as a snippet in docs/README.md

Per #2831, you can pip install pymdown-extensions and do this in mkdocs.yml:

markdown_extensions:
  - pymdownx.snippets

Then make a docs/README.md with contents --8<-- "README.md".

This renders the README.md but suffers the same issue as the symlinks option above. It renders the page, but links on the page are either broken on the docs site (if targeting docs/run-exosphere.md), or broken on the repo site (if targeting run-exosphere.md).

[oprypin]

mkdocs.yml in repo root, and docs_dir set to repo root (.)

https://github.com/oprypin/mkdocs-same-dir

[joined]

I faced more or less the same problem described in this discussion today, tried using this plugin and also the work in progress branch from #3450 but did not succeed.
The structure of my repository is the same described above. The issue I faced is that when setting docs_dir: ., mkdocs serve will try to include all files from the root folder, which contains also node_modules and a bunch of other stuff. I tried to use mkdocs serve --clean along with

exclude_docs: |
    *
    !/docs/*
    !/README.md

to prevent automatic inclusion of everything else, but then the site is broken (at least locally) because the assets cannot be found.
Moreover, it seems like exclude_docs has no efffect on the file watcher that triggers a re-build of the docs, which means that when editing any file in my repository a re-build is triggered.
And for some reason the docs build takes forever (8 seconds compared to 0.2s before the changes).

[athackst]

I opened #3526 do handle the re-build issue.

[athackst]

@oprypin How open would mkdocs be to support this natively instead of in a plugin? I made a plugin that was also initially made to address this issue (https://github.com/athackst/mkdocs-simple-plugin). It would be great if mkdocs could just natively handle this.

I think now is the right time for this due to some of the improvements that have been made to mkdocs.
(1) We now have decoupling the mkdocs.yml location from build (now we can load config from anywhere)
(2) on_files(..) and watch() and unwatch() functions help deal with edge cases plugins might encounter.

I prototyped this change for mkdocs. I'm happy to share/create a pull request. It boils down to removing a couple of validations and adding a check within get_files to skip files within the site_dir.

[oprypin]

@athackst Sure! If you have a code change and you can also briefly describe what you tried in order to see that it works reliably, we can consider it.

[athackst]

Opened #3450 for you to take a look