Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Render Single Component #122

Merged
merged 12 commits into from
Jul 2, 2020
Merged

Render Single Component #122

merged 12 commits into from
Jul 2, 2020

Conversation

srueg
Copy link
Contributor

@srueg srueg commented Jun 24, 2020

Render a single component by faking the required parts of the inventory. Additional classes can be specified with the --values/-f flag (multiple possible).
For the compilation to succeed, a user is responsible to provide all required parameters (which are not covered in the defaults.yml).

This changes the CLI structure: There's two new command groups: catalog and component. The previous compile and clean commands are moved to the catalog group and the new-component was moved together with the compile-component command into the component group:

commodore component new
commodore component compile
commodore catalog compile

This first implementation doesn't support secret refs. This means all used secret refs in the defaults.yml must be overridden with concrete values in the provided classes.
Vault secret refs are supported and get revealed to the path they reference (e.g. ?{vaultkv:${customer:name}/${cluster:name}/backup-cluster-objects/password} becomes t-silent-test-1234/c-green-test-1234/backup-cluster-objects/password)

Closes #74

@srueg srueg force-pushed the render-single-component branch 2 times, most recently from d6164d8 to cad627d Compare June 24, 2020 15:41
@srueg
Copy link
Contributor Author

srueg commented Jun 24, 2020

Not done yet / Caveats:

  • The argocd.libjsonnet library is being faked with an empty object, therefore the Argo CD application manifest won't be rendered

  • Other component libs can't be used

    • Provide additional search path to fake libs
  • No jsonnet lib dependencies are created, therefore the kube.libjsonnet is missing (which breaks many components)

    • Try to reuse current dependency management
  • Parameters defined in target.yml can't be overridden

    • Instead of faking the cluster facts to the target directly, create a class to include, in order to allow overriding of facts etc.
  • Output is rendered to /tmp

    • Specify output with flag --output/-o

@srueg srueg force-pushed the render-single-component branch 5 times, most recently from dd4e6b5 to 45d27d5 Compare June 26, 2020 08:26
@srueg
Copy link
Contributor Author

srueg commented Jun 26, 2020

I fixed the remaining open points and was even able to find a solution to support vault references: By creating a fake VaultBackend, references are "revealed" to the path in Vault they specify.

@srueg srueg force-pushed the render-single-component branch 7 times, most recently from 6eaf8b1 to b21ef94 Compare June 29, 2020 16:52
@srueg srueg marked this pull request as ready for review June 29, 2020 16:52
@srueg srueg requested review from simu, corvus-ch and tobru June 29, 2020 16:52
Copy link
Member

@simu simu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall. Feel free to do the _relsymlink refactoring or not.

commodore/component/compile.py Outdated Show resolved Hide resolved
commodore/component/compile.py Outdated Show resolved Hide resolved
commodore/component/compile.py Outdated Show resolved Hide resolved
@srueg srueg force-pushed the render-single-component branch 2 times, most recently from 2f36a30 to d5ed0f7 Compare June 30, 2020 14:05
Copy link
Contributor

@corvus-ch corvus-ch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR has fallen into classic scope creep. From what I can see, I would have made at least three PRs out of this.

  1. Remove the build component test container image build
  2. Restructure
  3. Add the functionality

Why? Because this results in smaller commits with a more narrow scope and thus are easier to review. It also allows to separate discussions from each other. I might question one thing but agree with others. If in a separate PR, I can approve the one and discuss on the other.

Yes I am aware of the fact that some changes rely on each other. But combining them is what in german we call a Mogelpackung which might work in politics but should not be done in software engineering.

@srueg srueg force-pushed the render-single-component branch 4 times, most recently from 89c4156 to 04e49e9 Compare July 2, 2020 06:47
@srueg
Copy link
Contributor Author

srueg commented Jul 2, 2020

I moved as many changes into own PRs as possible. This last one still includes the change of command structure and the actual component compile functionality since they're too closely intertwined.
PR #130 needs to be merged first since it switches to using the Kapitan library instead of running the binary.

@srueg srueg changed the base branch from master to use-kapitan-lib July 2, 2020 07:16
@srueg srueg requested review from simu and corvus-ch July 2, 2020 07:17
Makefile Show resolved Hide resolved
Base automatically changed from use-kapitan-lib to master July 2, 2020 07:36
@srueg srueg merged commit cc02802 into master Jul 2, 2020
@srueg srueg deleted the render-single-component branch July 2, 2020 13:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support rendering manifests for a single component
3 participants