-
Notifications
You must be signed in to change notification settings - Fork 128
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Multi-Line Environment Variables Validation Errors #616
Comments
I agree that the injection is a bit simplistic and we should be doing something more. For the specific case, something like this may get work: _format_version: "1.1"
consumers:
- custom_id: consumer_1
jwt_secrets:
- algorithm: RS256
key: consumer_1_iss_key
rsa_public_key: |
${{ env "DECK_RSA_PUBLIC_KEY" }}
secret: consumer_1_secret
username: consumer_1_user_name
|
Unfortunately that still fails with the same error, since the template is still injecting multiple lines into the file and the subsequent lines aren't indented.
@hbagdi What do you think about first unmarshaling the YAML file into a temporary map immediately after reading in the |
Either that, or the template funcs that are passed in need to be a lot more powerful so that they can enable the yaml (and JSON) formatting necessary |
That is my current stance since I expect to not have a lot of such cases. |
The one downside that I do know of to doing things raw within the template is that it requires a lot of extra overhead... for example, in this case, you not only need to "yaml-ize" the input, but also to ensure that it is indented the correct indentation level, and ensure that each line (since it is a multi-line input) is indented the correct indentation level as well. It's adds some complexity to know the levels and to know how it will be rendered... which is made even more complicated because of the fact that there's no render command to only render what the actual config will look like pre-validation... |
A workaround for this would be to use the JSON format for the configuration file, instead of YAML (please note the backticks around the variable name):
|
@GGabriele I just tried running with the JSON format, it caused a schema violation error. $ cat kong.json
{
"_format_version": "1.1",
"consumers": [
{
"custom_id": "consumer_1",
"username": "consumer_1_user_name",
"jwt_secrets": [
{
"algorithm": "RS256",
"key": "consumer_1_iss_key",
"rsa_public_key": "${{ env `DECK_RSA_PUBLIC_KEY` }}",
"secret": "consumer_1_secret"
}
]
}
]
}
```shell
$ DECK_RSA_PUBLIC_KEY=$(cat ./auth0_rsa_public.key) deck --state "./kong.json" sync
creating consumer consumer_1_user_name
creating jwt-auth consumer_1_iss_key for consumer faeb4f73-45aa-45b5-aa33-d1b10fd4bc9a
Summary:
Created: 1
Updated: 0
Deleted: 0
Error: 1 errors occurred:
while processing event: {Create} jwt-auth consumer_1_iss_key for consumer faeb4f73-45aa-45b5-aa33-d1b10fd4bc9a failed: HTTP status 400 (message: "2 schema violations (failed conditional validation given value of field 'algorithm'; rsa_public_key: invalid key)") |
@probably-not I see what's wrong, my env variable was well formatted but getting it directly from a file is more problematic. Can you try something like this?
|
The workaround, with using the json format on the state files, is working in my case. Thank you! |
Good to hear! I'll try it out in my pipeline to see if it works well. |
Thank you both for the patience on this! |
@GGabriele Looks like this works in our pipeline, the only issue I found is that the tools (jq, tr) are not available in the base deck Docker image, so we had to create our own image and add in tools manually. If we can add these tools to the deck Dockerfile that would be a big help. |
@probably-not I just tried with the latest
Just opened a PR to add Thanks for confirming this works well with you! |
Excellent, thanks 👍 |
For anyone else coming here having the same problems I had (passing a multi-line public key into the declarative config as an environment variable), I was able to make it work in YAML like this: Replace newlines with "\n" in the public key and assign that to the environment variable:
And then in the declarative config YAML:
The above usage takes advantage of the fact that double quotes in YAML convert "\n" to newlines. I'm also using the backticks for quoting the environment variable name -- apparently this is a Go Template thing. Anyway... there you have it. Worked for me. |
In most companies, security policies dictate that we don't store secrets within a simple Git repository, and instead, inject the data via env variables to any CI/CD process that runs and requires the secrets.
While decK allows env variable injection, the injection is done in a simple way using Go Templates, without any regards to formatting of the file format. Subsequently, if the secret is a multi-line secret (such as a private or public RSA key necessary in certain plugins), then this breaks the file formatting and will cause errors in the file validation.
As an example, with a state file that looks like this:
Running
decK validate
fails with the following error:The text was updated successfully, but these errors were encountered: