Configure via env vars, not sed #115
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@puckel Anything I can do to get this merged? (other than inventing a magic machine that gives everyone more time) |
|
Hi @ashb |
If any of these had spaces in them they could have caused interesting behaviour. As a safety measure lets just quote them.
Airfow will look at "AIRFLOW__${section}__${setting}" in preference to
anything in the config file, so using this we can 1) avoid having to run
sed, and 2) means we don't have to change the config file at run time
ashb
force-pushed
the
env-vars-not-sed
branch
from
41a887d
to
48e7b77
Compare
|
Hi @puckel I've finally gotten around to adding some docs to the readme about configuration via environment. Let me know if you'd like any more detail. |
|
Thanks @ashb I'll release it at the same time with 1.9.0 |
|
Great, 1.9.0 should be soon - RC8 has been accepted by the Airfow team so we're just waiting on Apache governance process now. (Not sure if you follow along on the mailing list or not.) |
|
I'm following the mailing-list too. I'll release as soon as possible when 1.9.0 is out |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR changes the entrypoint.sh script so that it doesn't need to change airflow.cfg to customize the settings.
The main reason we want this is that we have a repo which is our AIRFLOW_HOME, it contains a
dags/folder, and custom plugins inplugins/, some library functions/python modules and anairflow.cfg.Because we have more than just the dags folder in our repo I'd prefer to not have to manually volume mount each thing individually and would prefer to just
docker run -v .:/usr/local/airflow. Which works, except that our config is no longer default so some of the sed commands fail, and others end up changing the file which is tracked in git. All just a bit messy.So since Airflow has built in feature to read config settings from environment variables we can just export some vars. No sed needed, no changes I need to revert every time in git (and an added bonus no changing to the base docker image at run time)