Document env variable format

[tredmon]

Please do not report security vulnerabilities here. The Responsible Disclosure Program details the procedure for disclosing security issues.

Thank you in advance for helping us to improve this library! Your attention to detail here is greatly appreciated and will help us respond as quickly as possible. For general support or usage questions, use the Auth0 Community or Auth0 Support. Finally, to avoid duplicates, please search existing Issues before submitting one here.

By submitting an Issue to this repository, you agree to the terms within the Auth0 Code of Conduct.

Describe the problem you'd like to have solved

A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

The documentation is sparse on the config.json file, but is completely lacking when it comes to the format of environment variables that override said file. Furthermore, the format is not intuitive.

For example, I would think the format of the variable AUTH0_EXCLUDED_RULES would either be a JSON array or comma-delimited list, but it seems to be neither.

$ AUTH0_DOMAIN=some-valid-domain.auth0.com AUTH0_CLIENT_ID=some-valid-client-id AUTH0_CLIENT_SECRET=some-valid-client-secret AUTH0_EXCLUDED_RULES='["my-rule-to-exclude-1","my-rule-to-exclude-2"]' a0deploy import -i config/dir
2019-10-11T20:12:41.006Z - info: Processing directory config
2019-10-11T20:12:41.010Z - info: Getting access token for some-valid-client-id/some-valid-domain.auth0.com
2019-10-11T20:12:41.055Z - error: Problem running command import
2019-10-11T20:12:41.055Z - error: Schema validation failed loading [
    {
        "keyword": "type",
        "dataPath": ".exclude.rules",
        "schemaPath": "#/properties/exclude/properties/rules/type",
        "params": {
            "type": "array"
        },
        "message": "should be array"
    }
]
$ AUTH0_DOMAIN=some-valid-domain.auth0.com AUTH0_CLIENT_ID=some-valid-client-id AUTH0_CLIENT_SECRET=some-valid-client-secret AUTH0_EXCLUDED_RULES='my-rule-to-exclude-1,my-rule-to-exclude-2' a0deploy import -i config/dir
2019-10-11T20:12:41.006Z - info: Processing directory config
2019-10-11T20:12:41.010Z - info: Getting access token for some-valid-client-id/some-valid-domain.auth0.com
2019-10-11T20:12:41.055Z - error: Problem running command import
2019-10-11T20:12:41.055Z - error: Schema validation failed loading [
    {
        "keyword": "type",
        "dataPath": ".exclude.rules",
        "schemaPath": "#/properties/exclude/properties/rules/type",
        "params": {
            "type": "array"
        },
        "message": "should be array"
    }
]

Describe the ideal solution

A clear and concise description of what you want to happen.

The ideal situation would be clear documentation for the format of the environment variables.

Alternatives and current work-arounds

A clear and concise description of any alternatives you've considered or any work-arounds that are currently in place.

I considered the workaround of creating a temporary config.json file with values from the environment variables. This fails due to #172

set -e
jq -n --arg AUTH0_DOMAIN "$AUTH0_DOMAIN" --arg AUTH0_CLIENT_ID "$AUTH0_CLIENT_ID" --arg AUTH0_CLIENT_SECRET "$AUTH0_CLIENT_SECRET" --argjson AUTH0_EXCLUDED_RULES "$AUTH0_EXCLUDED_RULES" '{$AUTH0_DOMAIN,$AUTH0_CLIENT_ID,$AUTH0_CLIENT_SECRET,$AUTH0_EXCLUDED_RULES}' > /tmp/a0deploy-config.json
export FAILED=0
a0deploy import --no-env -c /tmp/a0deploy-config.json -i config/dir || export FAILED=1
rm /tmp/a0deploy-config.json
[[ $FAILED -eq 0 ]]

Current workaround that also works around the --no-env issue:

set -e
printf -v CONFIG_VAR_AUTH0_DOMAIN %s "${CONFIG_VAR_PREFIX}AUTH0_DOMAIN"
printf -v CONFIG_VAR_AUTH0_CLIENT_ID %s "${CONFIG_VAR_PREFIX}AUTH0_CLIENT_ID"
printf -v CONFIG_VAR_AUTH0_CLIENT_SECRET %s "${CONFIG_VAR_PREFIX}AUTH0_CLIENT_SECRET"
printf -v CONFIG_VAR_AUTH0_ALLOW_DELETE %s "${CONFIG_VAR_PREFIX}AUTH0_ALLOW_DELETE"
printf -v CONFIG_VAR_AUTH0_KEYWORD_REPLACE_MAPPINGS %s "${CONFIG_VAR_PREFIX}AUTH0_KEYWORD_REPLACE_MAPPINGS"
printf -v CONFIG_VAR_INCLUDED_PROPS %s "${CONFIG_VAR_PREFIX}INCLUDED_PROPS"
printf -v CONFIG_VAR_EXCLUDED_PROPS %s "${CONFIG_VAR_PREFIX}EXCLUDED_PROPS"
printf -v CONFIG_VAR_AUTH0_EXCLUDED_RULES %s "${CONFIG_VAR_PREFIX}AUTH0_EXCLUDED_RULES"
printf -v CONFIG_VAR_AUTH0_EXCLUDED_CLIENTS %s "${CONFIG_VAR_PREFIX}AUTH0_EXCLUDED_CLIENTS"
printf -v CONFIG_VAR_AUTH0_EXCLUDED_RESOURCE_SERVERS %s "${CONFIG_VAR_PREFIX}AUTH0_EXCLUDED_RESOURCE_SERVERS"
jq -ne \
    --arg AUTH0_DOMAIN "${!CONFIG_VAR_AUTH0_DOMAIN}" \
    --arg AUTH0_CLIENT_ID "${!CONFIG_VAR_AUTH0_CLIENT_ID}" \
    --arg AUTH0_CLIENT_SECRET "${!CONFIG_VAR_AUTH0_CLIENT_SECRET}" \
    --arg AUTH0_ALLOW_DELETE "${!CONFIG_VAR_AUTH0_ALLOW_DELETE}" \
    --argjson AUTH0_KEYWORD_REPLACE_MAPPINGS "${!CONFIG_VAR_AUTH0_KEYWORD_REPLACE_MAPPINGS}" \
    --argjson INCLUDED_PROPS "${!CONFIG_VAR_INCLUDED_PROPS}" \
    --argjson EXCLUDED_PROPS "${!CONFIG_VAR_EXCLUDED_PROPS}" \
    --argjson AUTH0_EXCLUDED_RULES "${!CONFIG_VAR_AUTH0_EXCLUDED_RULES}" \
    --argjson AUTH0_EXCLUDED_CLIENTS "${!CONFIG_VAR_AUTH0_EXCLUDED_CLIENTS}" \
    --argjson AUTH0_EXCLUDED_RESOURCE_SERVERS "${!CONFIG_VAR_AUTH0_EXCLUDED_RESOURCE_SERVERS}" \
    '{$AUTH0_DOMAIN,$AUTH0_CLIENT_ID,$AUTH0_CLIENT_SECRET,$AUTH0_ALLOW_DELETE,$AUTH0_KEYWORD_REPLACE_MAPPINGS,$INCLUDED_PROPS,$EXCLUDED_PROPS,$AUTH0_EXCLUDED_RULES,$AUTH0_EXCLUDED_CLIENTS,$AUTH0_EXCLUDED_RESOURCE_SERVERS}' \
    >/tmp/a0deploy-config.json
export FAILED=0
a0deploy import -i config -c /tmp/a0deploy-config.json || export FAILED=1
rm /tmp/a0deploy-config.json
[[ $FAILED -eq 0 ]]

Additional context

Add any other context or screenshots about the feature request here.

[mendhak]

Hi Auth0 would really appreciate some help with this, some examples would do. I'm specifically trying to figure out how to set an array in an environment variable and supply it using the @@NOTATION@@

[mikestopcontinues]

@mendhak Did you figure out @@Notation@@ arrays? I'm looking myself, and I'm afraid to test import, because there's also no way to dry-run changes. 😓

[mendhak]

@mikestopcontinues I never did figure it out. I also opened a support ticket with Auth0 about this, but they said it wasn't possible (which means the documentation is wrong?) and they'd take it as feedback. That was a back in 2019. I've now got large config.xxxx.json files... 😅

Yeah there's no way to dry run so I manage like so:

I've got a staging branch, and I 'test' it out by deploying to dev and staging tenants first. Also all development and non-production testing happens against those tenants.

Then I do a pull request from staging to prod branch. The best thing to do here is to have small, frequent changes into prod and deploy those. The smaller the changes the less fearsome it is, as it'll be small and manageable.

[mikestopcontinues]

@mendhak After a bunch more trial and error, I was able to get it working, but only using the config.json files, not via JSON-encoded env vars:

config.tenant.js:

{
  "AUTH0_KEYWORD_REPLACE_MAPPINGS": {
    "ARRAY": ["val"]
  }
}

Some file:

{
  "allowed_logout_urls": @@ARRAY@@
}

[dan-jackson-github]

@mendhak @mikestopcontinues if I have understood your problem correctly, it is the same one I think I have just solved.

If you want to save an array value via an environment variable then use the ## notation to denote your variable and set the variable value like this:

["http://domain1","http://domain2"]. Double quotes surrounding each array element, but no double quotes surrounding the entire array. Also make sure there are no spaces between array elements, this threw an error for me.

In my specific example I was setting the web_origins arrary in my client.json. It look like this:

an extract of my json looked like this:

"app_type": "spa",
"grant_types": [
"authorization_code",
"implicit",
"refresh_token"
],
"web_origins": ##WEB_CLIENT_CONNECTION_WEB_ORIGINS##,
"custom_login_page_on": true

We use Azure Devops so we just set the environment variable from a variable in an associated group. Hope that helps.

[willvedd]

Hi folks, there's a lot covered in this thread; a bit too much for me to break down. However I do hear a general cry for env var documentation. Fortunately, I'm in the process of updating our docs as we speak. Below is the initial draft for the environment variables section. Please give a read through and let me know if it answers your questions and if not, how I could clarify them. Thanks!

Environment variables

By default, the Deploy CLI ingests environment variables, providing the ability to pass credentials and other configurations to the tool without needing to publish to the config.json file. Environment variables can either be used to augment the config.json file or replace it altogether depending on the project needs.

Non-primitive configuration values like AUTH0_KEYWORD_MAPPING and AUTH0_EXCLUDED can also be passed in through environment variables so long as these values are properly serialized JSON.

To disable the consumption of environment variables for either the import or export commands, pass the --env=false argument.

Examples:

# Deploying configuration for YAML formats without a config.json file
AUTH0_DOMAIN=<YOUR_AUTH0_DOMAIN> \
AUTH0_CLIENT_ID=<YOUR_CLIENT_ID> \
AUTH0_CLIENT_SECRET=<YOUR_CLIENT_SECRET> \
a0deploy import --input_file=local/tenant.yaml

# Disable environment variable ingestion
a0deploy export -c=config.json --format=yaml --output_folder=local --env=false

# Non-primitive configuration values
AUTH0_EXCLUDED='["actions","organizations"]'\
AUTH0_KEYWORD_REPLACE_MAPPINGS='{"ENVIRONMENT":"dev"}'\
a0deploy export -c=config.json --format=yaml --output_folder=local

[willvedd]

I just merged in project documentation including a section about environment variables. I believe that it satisfies the original request of documenting the "env var format".

[wouterSeyen]

Hi, @willvedd
I stumbled on this thread while trying to supply an array env variable to be used for replace mappings.
I only want to supply one specific mapping from an environment variable. The rest is defined in the config.json files.
From the documentation, I understand that the AUTH0_KEYWORD_REPLACE_MAPPINGS is merged with the current environment variables. However when trying to set an environment variable to contain an array, by serializing the JSON array, it does not get correctly replaced and keeps on giving following error:

 error: Schema validation failed loading [
    {
        "keyword": "type",
        "dataPath": ".resourceServers[0].scopes",
        "schemaPath": "#/properties/resourceServers/items/properties/scopes/type",
        "params": {
            "type": "array"
        },
        "message": "should be array"
    }
]

[willvedd]

@wouterSeyen please refer to the documentation regarding keyword replacement. Specifically, observe the @@ALLOWED_ORIGINS@@ example which may clear-up some confusion.

Otherwise, I'm not sure what the specific issue is; you would need to provide a sample of your work for me to help troubleshoot. If you think you've found a bug or a unique issue, please open a new Github issue.

[wouterSeyen]

@willvedd, thank you for the fast reply. I've created #687 for this issue.