Describe how you compare with Kubernetes WG-Multi Tenancy implementation of Virtual Clusters

[craigbox]

How does your implementation compare with https://github.com/kubernetes-sigs/multi-tenancy/tree/master/incubator/virtualcluster?

Are you aware of, or working with, this team?

[LukasGentele]

@craigbox Yes, @FabianKramm and I had a nice chat with the SIG multi-tenancy in December. At that time vcluster was still directly bundled with Loft and not open-source which made it more tricky for us to go deeper into the details and see how we can contribute. Part of the decision behind open-sourcing vcluster was actually to facilitate the conversation with the SIG multi-tenancy and others in the CNCF ecosystem who are eager to make virtual clusters work.

As for the differences between vcluster and the design of the multi-tenancy group, take a look at our design principles for vcluster: https://www.vcluster.com/docs/architecture/basics#design-principles

The key differences are in:

Principle 1 - Control plane: vcluster is based on the more lightweight k3s distro while the SIGs virtual clusters are focused on regular k8s AFAIK. That may be a good way to join forces actually because both options are great but for different use cases and dual support would be awesome for users. Sometimes users may want lightweight clusters but sometimes, they may want the full k8s capabilities.

Principle 4 - Provisioning: vcluster makes no assumption about how to it is being deployed while the SIGs approach is integrating it into cluster-api AFAIK. As a result, vcluster does not offer any CRDs for creating vcluster or a control plane for managing them. We see that as out-of-scope and just provide a helm chart as a deployment option while the SIG is offering CRDs through their cluster-api integration.

Principle 5 - Admin privileges: vcluster is basically just a StatefulSet in a namespace, so you don't need any changes to the underlying cluster which may require admin privileges. If you have access to a single namespace in a multi-tenant cluster today, you can launch a vcluster.

Principle 6 - Single namespace: This may be the most significant difference in design. If you have X namespace in vcluster, they will still be mapped and contained into a single namespace of the underlying host cluster. The SIGs design would create X namespaces in the underlying cluster AFAIK. Both can have advantages and disadvantages for certain use cases. Supporting a dual model would actually be super interesting, so users could choose based on the use cases.

After launching vcluster this week, I hope we can set up another call with the SIG to discuss how to work together on making vcluster to a popular choice for solid k8s multi-tenancy. I need to follow-up with Fei and Adrian on Slack about setting up a call.

[craigbox]

Thanks. I just thought it was very interesting that your launch happened the same week that https://kubernetes.io/blog/2021/04/15/three-tenancy-models-for-kubernetes/ was published 🙂

Both are covered in this week's news: https://kubernetespodcast.com/episode/147-service-level-objectives-nobl9/

[LukasGentele]

That was coincidence :D The blog post beat us by a couple of days though. Our original goal was actually launching in March already but I'm sure you know how hard it is sometimes to ship on time ;)

Great to hear. Thanks for the honorable mention in the podcast!

[olljanat]

One big difference is in area of persistent volumes?

SIG virtualcluster readme contains comment:

VirtualCluster does not support tenant PersistentVolumes. All PVs and Storageclasses are provided by the super cluster.

But IIUC https://www.vcluster.com/docs/architecture/storage it should not be issue on here?

[matskiv]

Seems like the question was answered, so I'll close this issue.
If you have follow-up questions - don't hesitate to open another issue or to reach out to us in the #vcluster channel in our Slack.