Bulk package creation

[bgrant0607]

We've talked about "package sets", which could generate multiple deployment packages from a blueprint.

The idea would be similar to ArgoCD Application Sets:
https://argo-cd.readthedocs.io/en/stable/user-guide/application-set/
https://argo-cd.readthedocs.io/en/stable/operator-manual/applicationset/
https://www.youtube.com/watch?v=GcvHKc2IHi8

But without literal templates, of course.

The concept of generators is a good one. However, with package orchestration, we'd fully render the configuration specialized for each deployment target, as well as generating the sync spec for the target.

To do this, we need to find the input context for each variant, and inject it into each deployment package, in addition to finding the target coordinates. As with application sets, there could be several supported sources and patterns of input data and methods of target discovery. Like the ArgoCD ApplicationSet matrix generator, inputs and targets could be independent sources.

Managing multiple sources of truth is tricky. We'd want to allow direct edits to the deployment packages.

We'd also want to be able to layer multiple layers of specialization and sources of inputs, such as an off-the-shelf blueprint customized for a specific use, then specialized for an environment (dev, prod), and specialized for a specific cluster.

AIUI, Puppet supports multiple sources of specialization via a combination of Hiera (hierarchical config values) and Facter (target-specific details).

[karlkfi]

My initial package set proposal (google internal) last year was based on renaming Config Sync's RootSync/RepoSync to ClusterPackageSet and PackageSet. I don't know if this 100% overlaps with what you're suggesting here, but the idea was to effectively use GitOps (or "OCI Ops", I guess) as the way to externalize the packages.

Config Sync already enables nesting of packages, by having an RSync point to a repo that includes another RSync. This follows the externalized sub-package pattern, as opposed to kpt's current internalized (aka vendored) sub-package pattern.

Before I joined the Config Sync team, I did ask for kpt to implement externalized sub-packages, but I'm not sure it really needs it if you're using Config Sync.

This raises a number of questions:

• Where does kpt end and Config Sync begin?
• Does kpt need to duplicate this functionality in order to satisfy users that want to execute kpt from their own CI/CD?
• If kpt can push packages as OCI images, how do you then refer to those OCI images as packages?
• Do we need to make a PackageSet controller or would that just be re-implementing Config Sync?
• Does Config Sync need to be able to execute kpt functions as part of the deployment pipeline, like it can do today for helm/kustomize?
• Is a PackageSet analogous to a ReplicaSet (all instances identical with different names)?
• Is a PackageSet analogous to a Deployment (orchestrating lifecycle management, upgrades, and rollouts)?
• Is a PackageSet analogous to a StatefulSet (unique variants with different input variable permutations)?
• Is a PackageSet analogous to a CronJob (re-appied periodically, aggregating success signals)?
• Is a PackageSet a single-cluster or multi-cluster abstraction?
• If PackageSet handles single-cluster rollouts, should it also handle multi-cluster rollouts, or is that another layer on top?
• If we have a PackageSet, do we need a ClusterPackageSet (cluster-scoped vs namespace-scoped)?
• Are PackageSet also composable?
• Do we also need a FleetPackageSet?

I think we need a vision for where this all leads in a multi-cluster world in order to work backwards to figure out what we need at the single-cluster level. Package and PackageSet is a really good name. I don't want to waste it by evolving organically and then wish we'd named it something else so we could use the name at a higher level.

[bgrant0607]

This is about generation of packages rather than deployment of packages.

[karlkfi]

This is about generation of packages rather than deployment of packages.

Sure, but Config Sync already enables generation of package variants using kustomize and helm. It could also do the same with kpt's function pipeline, similar to how porch executes kpt functions.

ArgoCD Application Sets similarly render a template into multiple variants remotely, before applying them.

I'm not saying that all hydration should happen server-side. Enviornmental varients, for example, often need to be client-side, because they're going to be applied to multiple clusters/fleets. But if you add in an admin cluster and another layer of abstraction, you can push that variant generation work to be server-side again with a multi-cluster package orchestration controller.

[bgrant0607]

Porch is server side.

The "package set" proposal I was referring to was Morgante's.