Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: allow interpolation of cluster generator values #9254

Conversation

blakepettersson
Copy link
Member

@blakepettersson blakepettersson commented Apr 29, 2022

Allow the interpolation of values found in the cluster generator.
This allows interpolation of {{name}}, {{server}},
{{metadata.labels.*}} and {{metadata.annotations.*}}. See
argoproj/applicationset#371.

This interpolation could potentially be extended to the list and
duck-type generators if desired.

Note on DCO:

If the DCO action in the integration test fails, one or more of your commits are not signed off. Please click on the Details link next to the DCO action for instructions on how to resolve this.

Checklist:

  • Either (a) I've created an enhancement proposal and discussed it with the community, (b) this is a bug fix, or (c) this does not need to be in the release notes.
  • The title of the PR states what changed and the related issues number (used for the release note).
  • I've included "Closes [ISSUE #]" or "Fixes [ISSUE #]" in the description to automatically close the associated issue.
  • I've updated both the CLI and UI to expose my feature, or I plan to submit a second PR with them.
  • Does this PR require documentation updates?
  • I've updated documentation as required by this PR.
  • Optional. My organization is added to USERS.md.
  • I have signed off all my commits as required by DCO
  • I have written unit and/or e2e tests for my change. PRs without these are unlikely to be merged.
  • My build is green (troubleshooting builds).

@codecov
Copy link

codecov bot commented Apr 29, 2022

Codecov Report

Merging #9254 (28f0846) into master (9d4c940) will increase coverage by 0.00%.
The diff coverage is 60.00%.

@@           Coverage Diff           @@
##           master    #9254   +/-   ##
=======================================
  Coverage   45.78%   45.79%           
=======================================
  Files         227      227           
  Lines       26904    26920   +16     
=======================================
+ Hits        12318    12327    +9     
- Misses      12908    12911    +3     
- Partials     1678     1682    +4     
Impacted Files Coverage Δ
applicationset/utils/utils.go 78.40% <ø> (ø)
applicationset/generators/cluster.go 73.75% <60.00%> (-4.38%) ⬇️
util/settings/settings.go 48.16% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9d4c940...28f0846. Read the comment docs.

Copy link
Collaborator

@crenshaw-dev crenshaw-dev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is super interesting, thanks for writing it up! Could you add some docs about how it would be used?

@blakepettersson blakepettersson force-pushed the feature/enable-interpolation-for-cluster-generator-values branch from 5bcf40c to 1c16a4a Compare June 9, 2022 16:08
@blakepettersson
Copy link
Member Author

This is super interesting, thanks for writing it up! Could you add some docs about how it would be used?

Thanks for finding it useful 😄 I wrote some basic usage docs in Generators-Clusters.md, let me know if that is enough or if I need to complement with something else!

@blakepettersson blakepettersson force-pushed the feature/enable-interpolation-for-cluster-generator-values branch from 1c16a4a to 9b746b6 Compare June 9, 2022 18:35
@blakepettersson
Copy link
Member Author

@crenshaw-dev is there anything else I should do on this PR?

@crenshaw-dev
Copy link
Collaborator

@blakepettersson ah! No, besides pinging me to remind me to review. Thanks. :-)

if err != nil {
return err
}
params[fmt.Sprintf("values.%s", key)] = replacedTmplStr
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a little worried about setting directly to params. At this point, AppSet is admin-only, but in the future I'd like it to see it made available to lower-privileged users. In that case, I'd be concerned about a billion-laughs attack.

The attack would work like this:

values:
  lol1: lol
  lol2: '{{values.lol1}}{{values.lol1}}{{values.lol1}}{{values.lol1}}{{values.lol1}}{{values.lol1}}{{values.lol1}}{{values.lol1}}{{values.lol1}}{{values.lol1}}'
  lol3: '{{values.lol2}}{{values.lol2}}{{values.lol2}}{{values.lol2}}{{values.lol2}}{{values.lol2}}{{values.lol2}}{{values.lol2}}{{values.lol2}}{{values.lol2}}'
  # ...etc. through lol9

There are probably other/easier ways to attack the AppSet controller, but I'd rather not add another one.

What if we just keep track of an interpolatedParams map which isn't merged back with params until all the other interpolation is finished?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if we just keep track of an interpolatedParams map which isn't merged back with params until all the other interpolation is finished?

Ah, good catch! I updated the PR to do just that, and added a test case for it as well.

@blakepettersson blakepettersson force-pushed the feature/enable-interpolation-for-cluster-generator-values branch from 3f64a40 to c9a163e Compare June 24, 2022 15:54
Allow the interpolation of `values` found in the cluster generator.
This allows interpolation of `{{name}}`, `{{server}}`,
`{{metadata.labels.*}}` and `{{metadata.annotations.*}}`. See
argoproj/applicationset#371.

This interpolation could potentially be extended to the list and
duck-type generators if desired.

Signed-off-by: Blake Pettersson <blake.pettersson@gmail.com>
Add a basic example of how values interpolation can be used with the
cluster generator.

Signed-off-by: Blake Pettersson <blake.pettersson@gmail.com>
The previous implementation was vulnerable to a billion-laughs attack,
where someone could interpolate values based upon other values,
something like:

```yaml

values:
  lol1: lol
  lol2: '{{values.lol1}}{{values.lol1}}' #
  lol3: '{{values.lol2}}{{values.lol2}}{{values.lol2}}{{values.lol2}}'
```

To counteract that, instead of directly manipulating the `params` map,
we create a map to keep track of the interpolated values, and only
template the values which have been previously whitelisted. Once we go
through all the values, we then merge the interpolated values map back
to the `params` map.

Signed-off-by: Blake Pettersson <blake.pettersson@gmail.com>
@blakepettersson blakepettersson force-pushed the feature/enable-interpolation-for-cluster-generator-values branch from c9a163e to 28f0846 Compare June 24, 2022 16:54
Copy link
Collaborator

@crenshaw-dev crenshaw-dev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thanks for your patience on the review!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants