-
Notifications
You must be signed in to change notification settings - Fork 228
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bulk package creation #3347
Comments
My initial package set proposal (google internal) last year was based on renaming Config Sync's RootSync/RepoSync to ClusterPackageSet and PackageSet. I don't know if this 100% overlaps with what you're suggesting here, but the idea was to effectively use GitOps (or "OCI Ops", I guess) as the way to externalize the packages. Config Sync already enables nesting of packages, by having an RSync point to a repo that includes another RSync. This follows the externalized sub-package pattern, as opposed to kpt's current internalized (aka vendored) sub-package pattern. Before I joined the Config Sync team, I did ask for kpt to implement externalized sub-packages, but I'm not sure it really needs it if you're using Config Sync. This raises a number of questions:
I think we need a vision for where this all leads in a multi-cluster world in order to work backwards to figure out what we need at the single-cluster level. Package and PackageSet is a really good name. I don't want to waste it by evolving organically and then wish we'd named it something else so we could use the name at a higher level. |
This is about generation of packages rather than deployment of packages. |
Sure, but Config Sync already enables generation of package variants using kustomize and helm. It could also do the same with kpt's function pipeline, similar to how porch executes kpt functions. ArgoCD Application Sets similarly render a template into multiple variants remotely, before applying them. I'm not saying that all hydration should happen server-side. Enviornmental varients, for example, often need to be client-side, because they're going to be applied to multiple clusters/fleets. But if you add in an admin cluster and another layer of abstraction, you can push that variant generation work to be server-side again with a multi-cluster package orchestration controller. |
Porch is server side. The "package set" proposal I was referring to was Morgante's. |
We've talked about "package sets", which could generate multiple deployment packages from a blueprint.
The idea would be similar to ArgoCD Application Sets:
https://argo-cd.readthedocs.io/en/stable/user-guide/application-set/
https://argo-cd.readthedocs.io/en/stable/operator-manual/applicationset/
https://www.youtube.com/watch?v=GcvHKc2IHi8
But without literal templates, of course.
The concept of generators is a good one. However, with package orchestration, we'd fully render the configuration specialized for each deployment target, as well as generating the sync spec for the target.
To do this, we need to find the input context for each variant, and inject it into each deployment package, in addition to finding the target coordinates. As with application sets, there could be several supported sources and patterns of input data and methods of target discovery. Like the ArgoCD ApplicationSet matrix generator, inputs and targets could be independent sources.
Managing multiple sources of truth is tricky. We'd want to allow direct edits to the deployment packages.
We'd also want to be able to layer multiple layers of specialization and sources of inputs, such as an off-the-shelf blueprint customized for a specific use, then specialized for an environment (dev, prod), and specialized for a specific cluster.
AIUI, Puppet supports multiple sources of specialization via a combination of Hiera (hierarchical config values) and Facter (target-specific details).
The text was updated successfully, but these errors were encountered: