Paths are relative to you, not the config file

[d0ugal]

You can do mkdocs build --config-file=path/to/mkdocs.yml but any configuration values in that file are relative to your current working directory.

This seems wrong to me, but it's only something I've ran into when trying to automate MkDocs builds (for integration testing). I'm not sure if we want to change this, or what people really expect. The common use case is that you are in the same directory as the config file anyway, so that doen't matter.

Something like this should be changed or at least decided for a 1.0 release.

Thoughts?

[waylan]

I think that is a reasonable compromise. Not sure when I would want to point to a config file in a different location anyway. If I had multiple config files for one project, they would most likley be side-by-side in the root of the project anyway:

./docs/
./default.yml
./experimental.yml

Or I suppose the various config files could be in a subdir:

./docs/
./mkdocs_cfg/default.yml
./mkdocs_cfg/experimental.yml

Either way, I would be calling the mkdocs command from the project root. Therefore, it seems logical that all paths in the config would be relative to the cwd. Besides, if I rearranged my file structure from one to the other, I would not want to have to edit each file to alter the paths.

When automating doc builds (in a tox config or from a make file) one may need to cd into the project root before building, but that is nothing unusual (and good practice anyway).

Maybe the commands could accept a --project_root option which defaults to the cwd. That might address the few edge cases where someone is not calling a command from the project root. But is seems unnecessary to me.

[d0ugal]

Okay, fair point. I mostly wanted to raise this and have some sort of record of it. I think I agree, so, closing this for now :)

[jimporter]

Not sure when I would want to point to a config file in a different location anyway.

The case I care about (from issue #676) is when performing out-of-tree builds. I treat my build system as a toolbox of commands, and being able to run make doc (from the build dir, of course) is nice. The main reason I use out-of-tree builds is to make it easy to keep the build files for multiple build configs around (e.g. a clang build vs a gcc build).

While MkDocs itself is unlikely to benefit from from out-of-tree builds, it means that for my projects, I'm either forced to run mkdocs directly[1] or handle cding into the source dir to build the docs. Neither is particularly convenient, and I think the stated benefit (being able to put your mkdocs.yml file in a subdir without changing the relative paths) is of dubious merit. In fact, not knowing about this behavior, I'd probably end up changing the relative paths and then have to change them back when I found out that that's not what MkDocs expected.

[1] Which is more typing, since I prefer to use --clean and activate my MkDocs virtualenv in-line.

[d0ugal]

Yeah, so it is interesting that you hit this when doing automation like I did. I think we should try and fix this, I can't really see how the paths being relative to the user makes sense. Personally, I found it really easy to work around, I don't think it is that hard, but the fact that the config essentially changes as you move around the file-system is weird.

I think for this to work well, we would need to join each config option with the path to the mkdocs yaml when we do all the config processing in https://github.com/mkdocs/mkdocs/blob/master/mkdocs/config/config_options.py

[tomchristie]

Just agreeing that I think this is a valid issue

[waylan]

So, rather than:

cd /path/to/project && mkdocs build --clean

You'd prefer to use the following line in your automation scripts:

mkdocs build --clean --config-file /path/to/project/mkdocs.yml

Where the paths in the config file are presumably relative to the config file.

Sorry, but it is not clear to me how this is better. For that matter, it is not clear how the current way is better either (even if it is a few less characters of typing). What is clear is that it is different, which means that many existing users will need to change there config files if a change is made. And for what benefit?

[facelessuser]

I don't see why using cd isn't an acceptable solution. Instead of recalculating all paths inside mkdocs for this specific situation (which may or may not be a lot more work), just realizing it needs to be in the cwd and use cd should be enough. Just about any scripting language you would use to automate this stuff can change its cwd, IMO that should be enough, but I guess if the effort was put in to make it work the other way, that would be cool too, but I am okay with needing cd.

[d0ugal]

A very simple solution would be for MkDocs to to change it's cwd to the config file.

os.chdir(os.path.dirname(config_path))

This is a nasty nasty hack tho'

[tomchristie]

Def don't do that. ^ :)

[jimporter]

@waylan: Well, given the build system I use, it's more like:

command('doc', cmd=['mkdocs', 'build', '--clean', source_file('mkdocs.yml')])

vs

srcdir = ... # Not actually sure what I'd do here.
command('doc', cmd="cd '{}' && mkdocs build --clean".format(shlex.quote(srcdir)))

Even the above isn't really perfect, since it'd break if the user was running the build system under Python 2.x. It's also something that took me a while to figure out, since I wasn't even thinking about the current-directory issue. (Granted, some of this is the fault of the build system making it somewhat difficult to cd into the srcdir.)

I agree that it's not the world's biggest issue, but it's a rough corner that makes things less convenient, when the only real benefit I can see is that it keeps the code somewhat simpler. I could probably patch the build system to do this in a slightly cleaner way if this bug were wontfixed, but I still think it would be a nice fix for MkDocs. Maybe it's more of a philosophical argument, but I've never liked it when programs unnecessarily depend on the current directory. It usually just ends up confusing me when I run into an error (I rarely think about what the cwd will be when I'm writing an automation script).

[facelessuser]

This seems like a bit of a hack tho'

I've seen this done before, and I guess I've been guilty of doing it myself. I'm not sure I would call it a hack if nothing else cares about the actual current working directory though. I really see no problem with controlling your current working directory if you have a good reason to. It would be completely transparent to the user, and it seems like Mkdocs doesn't care about the actual cwd. You could always restore the actual cwd when Mkdocs is done. If you aren't changing the cwd, you are just going to have to pass around Mkdocs calculated working directory, and then make everything relative to that. Seems like a lot more work to me when os.chdir(os.path.dirname(config_path)) would essentially solve all your problems.

Now if you needed to track the actual cwd and Mkdoc's cwd directory for some reason, then yeah, this is not an ideal solution and it should probably be handled differently by passing around your paths.

Meh, that's just my thoughts.

[jimporter]

You could always restore the actual cwd when Mkdocs is done.

Not sure if that would even be necessary, since a child process can't change its parent's cwd (which is why cd is an intrinsic shell function, not a real executable).

[facelessuser]

If Mkdocs was a library, then it would, but Mkdocs isn't really a lib, so your probably right.

[waylan]

A very simple solution would be for MkDocs to to change it's cwd to the config file.

That would not help at all. For people who have a setup in which the cwd is not the directory in which the config file resides, this would break their setup the same way as making all paths relative to the config file's path. So, yeah, it would be a hack. Don't do that.

[jimporter]

this would break their setup the same way as making all paths relative to the config file's path.

I think that's the point. Since MkDocs isn't 1.0 yet, you can't really expect the config file's semantics to remain 100% set in stone.

[facelessuser]

That would not help at all. For people who have a setup in which the cwd is not the directory in which the config file resides, this would break their setup the same way as making all paths relative to the config file's path.

Ahh, then yeah, if that is a concern, then it is a problem. I'm not sure of all the ways it is being used and ultimately what is planned for final usage in 1.0.

I think that's the point. Since MkDocs isn't 1.0 yet, you can't really expect the config file's semantics to remain 100% set in stone.

True, but it depends on Mkdoc's goals, which I am unaware of in regards to this.

[d0ugal]

If Mkdocs was a library, then it would, but Mkdocs isn't really a lib, so your probably right.

While not a library, we tread into that territory once we have a plugin API. So, yeah, I don't think we want to change the cwd.

[jimporter]

I don't think it's actually that hard to detect if the semantics change would break your build, assuming we care enough to detect it. Some observations:

• This will only change behavior if your mkdocs.yml is not in your cwd.
• If your mkdocs.yml is in another directory, there are exactly two base directories you need to check: your cwd and the directory containing your mkdocs.yml.
• The docs_dir is probably unique enough that you're unlikely to have a directory of that name in both your cwd and your mkdocs.yml dir.

Therefore, you should be able to get pretty darn close to automatically detecting all cases where this would change. Just check both the cwd and the mkdocs.yml dir for the existence of the docs_dir:

• If it's in the mkdocs.yml dir, then hooray, you're done!
• If it's in the cwd, issue a warning (and build anyway?).
• If it's in both (which is pretty unlikely), error out, and maybe add a --force option to use the new semantics. That way, you're not completely out of luck, but you've still learned about the new behavior.

Eventually, say at 1.0, you can drop all of this logic and always use the mkdocs.yml dir.

[d0ugal]

There are a lot of MkDocs users. In fact, according to this comment, over 2000 projects are hosted on Read The Docs that use MkDocs.

The slight irony of using ReadTheDocs as an example is that users have no control over the CWD in that example. So the documentation should build consistently on RTD's and however you build it elsewhere. This wont happen if the user requires something specific like calling from the parent.

I do believe that RTD is a large proportion of ours users, many people come to us as they want to use RTD but don't want to learn Sphinx/reStructuredText.

However, if the paths in the config file are changed from being relative to the cwd to being relative to the config file's path (an easy enough change to make), there will be no way to detect whether users have updated their config files, adjust accordingly (avoiding a failure to build) and issue a warning.

I can see a couple of options.

• If we can't find anything specified in the config, look for them relative to CWD and build with that and display a warning.
• Add a config setting called paths_relative_to_config because I can't think of a name yet, that defaults to False in the next release but with a warning stating that the default will be changing in 1.0 and the setting will be removed. Users then have time to upgrade as they see fit. This is mostly going to be annoying for users that otherwise wouldn't need to do anything - as they would need to add a setting and remove it later.
• ... and I started to write about detecting it if the config isn't in your CWD but @jimporter beat me.

Specifically, a number of users have placed their config file inside their docs file. However, they call the mkdocs command from the parent dir. Every one of those users projects would break if this change was made.

This sounds like how lots of users do it on ReadTheDocs, they don't call from the parent tho' - often the root of the git repo is the docs_dir. So the config is within the docs_dir, that caused an annoying bug :)

If that issue did't exist, I would actually prefer the change myself. It is certainly less surprising.

I believe that one of the biggest goals of MkDocs is to have sensible defaults and be as unsurprising as possible. If we don't change these things early on, then they will only get harder to change later.

[trel]

All things being equal, I'd vote for detecting as much non-standardness as possible, complaining loudly if any is found, and enforcing as minimal-as-possible set of different-ways-to-do-a-thing headed into 1.0.

If we don't change these things early on, then they will only get harder to change later.

Hear hear! Now is the moment.

[waylan]

Alright then. Sounds like the consensus is that the breaking change is worth it.

Please, just don't use the paths_relative_to_config config setting (or any better named config setting). If we can either fall back to the cwd or detect if the config file is not in the cwd and adapt and issue a warning that would be a sensible middle ground.

[d0ugal]

Please, just don't use the paths_relative_to_config config setting (or any better named config setting)

I won't. If that is required to make it happen I might be swayed against the change.

[tomchristie]

Alternative: remove --config-file. Because why?

[waylan]

Alternative: remove --config-file. Because why?

You may have a point. But I suspect you will get more complaints from people by making that change than anything else. And it will be a never-ending feature request.

[jimporter]

Alternative: remove --config-file. Because why?

The downside of this is that it's more complex for my case: I want to build documentation from my build dir, and then put the HTML in a subdirectory of the build dir. This makes cleaning a build trivial: rm -rf build. Without --config-file (and this issue), I'd have to both cd into the srcdir, and set the --site-dir. With them, I could just set --config-file.

[petri]

A strong +1 for making the paths relative to config - or at least add whatever extra option to enable that. I sure as hell don't want to cd back and forth. Cding to the docs dir prior to mkdocs build may sound reasonable when you compare just the command lines to run mkdocs, but that is an insufficient comparison. I would think anyone spending a lot of time in unix shells doing varying tasks in various apps will understand that. Having to juggle between dirs is a pain, even with push & popd.

I am using mkdocs to generate API documentation for a Python package. In my case, the mkdocs sources are using Jinja templating and are kept within the package. So I run a sort of preprocessor (Jinja etc) to generate "static" mkdocs sources outside the package tree. Including mkdocs.yml. To work around the current situation, I have to hardcode paths in the preprocessor step to be able to properly run the final mkdocs build step in the end. Which feels insane.

At some point I will just use mkdocs as a library, so I hope I will be able to sidestep the issue, but not there yet...

[waylan]

Ironically (as I've been the voice against this change), I just came up against a need for this feature. If you are deploying to user/organization GitHub pages, you need this. User/org pages do not commit to the gh-pages branch, but the master branch of a separate repo from the project. Currently the mkdocs gh-deploy command won't work. You need to use ghp-import directly like this:

cd project-repo
mkdocs build
cd ../pages-repo
ghp-import -b master -p ../project-repo/site 

If paths in the config were relative to the config file rather than the CWD, then this would work:

cd pages-repo
mkdocs gh-deploy -f ../project-repo/mkdocs.yml -b master

But as long as paths are relative to the CWD, that will never work. And as you need to run the command from the repo you are committing to, cding into the project is not an option as that takes you out of the repo.

I think this is the use-case that pulls me over. We now have a reason to make the change which overcomes my previous objections.

[jimporter]

@waylan That's a really good point, and something I should take into account for mike as well.

[nikramakrishnan]

Is this available yet? I am on version 0.17.4 and it seems like paths are still relative to my cwd.

My use case:

• My cwd is the root of the repository.
• The docs are situated in docs/reference/markdown and the config file is docs/reference/mkdocs.yml.
• docs_dir: 'markdown' in mkdocs.yml
• Run mkdocs build -f 'docs/reference/mkdocs.yml' from root (this is supposed execute programmatically).

This gives me ERROR - Config value: 'docs_dir'. Error: The path markdown isn't an existing directory.

However from this discussion and the merged pull request it seems that this should work.

[waylan]

This will be available in the next major release: 1.0.

[nikramakrishnan]

I was confused because the docs at https://www.mkdocs.org/user-guide/configuration/#docs_dir say otherwise:

This can either be a relative directory, in which case it is resolved relative to the directory containing your configuration file.

If this is not available yet, I think the documentation should either explicitly say that as such, or only current behavior should be mentioned.