Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Environment variables in config file #1954

Closed
majkinetor opened this issue Jan 6, 2020 · 12 comments · Fixed by #2267
Closed

Environment variables in config file #1954

majkinetor opened this issue Jan 6, 2020 · 12 comments · Fixed by #2267
Milestone

Comments

@majkinetor
Copy link

Different environments may have different configurations such site_url.

It would be great to be able to set any attribute via env var. Some plugins implement this on their own but OTB solution would be very needed.

This is more general then #1827

Current solution looks funky:

site_name: !!python/object/apply:os.getenv ["CI_PROJECT_PATH"]  
repo_url: !!python/object/apply:os.getenv ["CI_PROJECT_URL"]
repo_name: !!python/object/apply:os.getenv ["CI_PROJECT_PATH"]

Something like following is more readable

site_name: ${CI_PROJECT_PATH}

@maciejmatczak
Copy link

I am having similar issues when dealing with different OSes on same sources.

My path used for a shared theme is different whether I am on Linux (origin) or Windows (dev machine with nfs mapped disks).

Different targets/environments might work. Maybe having multiple mkdocs.yml files?

@majkinetor
Copy link
Author

BTW, #1827 doesnt work always, just for strings, which makes this proposal even more needed.

I asked for help about this to no avail:
https://stackoverflow.com/questions/59680815/setting-non-string-variable-using-pyyaml

@waylan
Copy link
Member

waylan commented Feb 5, 2020

Maybe having multiple mkdocs.yml files?

This is how MkDocs was built to handle this sort of thing. Using the -f/--config-file flag you can pass in a different config file. Note that the file does not need to be named mkdocs.yml. That is simply the default value for the --config-file flag. You could have any number of different config files. Then you can set a environment variable which points to the appropriate config file and pass that env variable to the flag.

mkdocs build --config-file "$CONFIG_FILE"

@majkinetor
Copy link
Author

That is OK, but not DRY, and will significantly slow you down (i.e. adding nav section on 4 environments ? right). There need to either be base environment that can be overriden or env vars for meaningfull and efficent work on it.

@maciejmatczak
Copy link

Using the -f/--config-file flag you can pass in a different config file

Oops, I omit the nested command helps, I was checking only the top level -h. Thanks, that helps!

I came across this:
https://stackoverflow.com/questions/528281/how-can-i-include-a-yaml-file-inside-another
which mentions: https://github.com/tanbro/pyyaml-include

It might be worth throwing some custom Yaml loader commands here. With such inheritance one could use base file and target wise prod/dev which would include the first one. One command for include, one for env variables (with .env support?). This would be really handy (having both).

@waylan
Copy link
Member

waylan commented Feb 5, 2020

https://github.com/tanbro/pyyaml-include

I wasn't aware that that lib existed. We get requests to add support for "include" to our config files with some regularity. Our answer has always been that we don't support it because YAML doesn't support it. And we have no interest in building and maintaining a one-off solution. However, if a third party lib exists which we simply need to call, then we may be more open to adding support. A PR would be given serious consideration.

@majkinetor
Copy link
Author

majkinetor commented Feb 5, 2020

Config path makes sense if it can be overriden. Take docker-compose for example, you can add multiple config files and each next may contain overrides (and has env vars nevertheless). Things mentioned by @maciejmatczak are also great!

I think env var is probably trivial to support and will be most flexible but having both is the best.

I wasn't aware that that lib existed. We get requests to add support for "include" to our config files with some regularity.

This always comes up. Gitlab recently added it in CI/CD yaml config as it was coming up for years.

@g13013
Copy link

g13013 commented Mar 31, 2020

Guys docs are meant to be built, it means, multi site, multi context, multi environments, please make it dynamic to be more DevOps friendly. I would add that a DotEnv support is also a very big improvement. just like docker-compose do.

@majkinetor
Copy link
Author

majkinetor commented Mar 31, 2020

Exactly.

Multi context would require some more work besides env vars and I do happen to have that on almost 100% of the cases - different subset of NAV section per environment.

Example:

  1. Staging environment contains all types of docs - functional, technical, user manual etc. It is installed internally on stakeholder infrastructure.
  2. Production env has only user manual, which is only 1 part of the complete docs project. It is publically available which means that end users shouldn't see internal IT stuff of the project. It is good to keep them together as stuff between different consumer groups are constantly reused.

Currently I mark with comment lines sections that should be added to specific environments and then build servers removes stuff and constructs new mkdocs.yml that differs only in NAV section.

While that is defnitelly part of different issue, I think that NAV section should be moved out of the config and if there is motivation for this, I can do detailed spec with examples and benefits versus current approach.

@waylan
Copy link
Member

waylan commented Nov 9, 2020

FYI I recently posted a proposal at #2218 which discusses various options for addressing complex configs. That discussion does not specifically address environment variables (which is the subject of this issue), but some of the other things mentioned in comments here are directly addressed there. Please leave any feedback at that issue.

Regarding environment variables, I expect we will be adding something for that separately from #2218. Look for a PR in the near future.

@waylan waylan added this to the 1.2 milestone Nov 9, 2020
@waylan
Copy link
Member

waylan commented Dec 21, 2020

I recently created waylan/pyyaml-env-tag, which provides for a nice and clean way to include the values of environment variables in YAML files with simple types being preserved. It should be relatively simple to incorporate that into MkDocs' Yaml loader. However, before I do, will this sufficiently address the concerns raised here?

@majkinetor
Copy link
Author

Look OK!

waylan added a commit to waylan/mkdocs that referenced this issue Dec 23, 2020
Environment variables can be assigned as values in the configuration
file using the !ENV tag. Resolves mkdocs#1954.

The behavior is defined in the third-party package pyyaml_env_tag:
https://github.com/waylan/pyyaml-env-tag
waylan added a commit to waylan/mkdocs that referenced this issue Dec 23, 2020
Environment variables can be assigned as values in the configuration
file using the !ENV tag. Resolves mkdocs#1954.

The behavior is defined in the third-party package pyyaml_env_tag:
https://github.com/waylan/pyyaml-env-tag
waylan added a commit that referenced this issue Dec 23, 2020
Environment variables can be assigned as values in the configuration
file using the !ENV tag. Resolves #1954.

The behavior is defined in the third-party package pyyaml_env_tag:
https://github.com/waylan/pyyaml-env-tag
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants