New Tenant v1beta1 version

[prometherion]

New features have been implemented in the current version, most of them are available using annotations that are just a workaround (or rather, a Technology Preview) for the said features.

We have to start releasing the new version that will provide real specification keys for the upcoming and planned features.

• Assign Priority Classes to tenants
• Require tenants to use imagePullPolicy as Always
• Add applications to act as tenant owner
• Make configurable the Resource Quota enforcement at tenant level

Luckily, we had a discussion on #202 and from the reconciliation PoV it's pretty easy, not the same for webhooks but we can work on that.

@MaxFedotov do you already have some rough ideas on the new API version structure?

[MaxFedotov]

And also we should not forgot about:

The following Tenant annotations allow a sort of RBAC on the operations of the nodes:
capsule.clastix.io/enable-node-listing: allows listing of nodes and node retrieval
capsule.clastix.io/enable-node-update: allows the update of the node (cordoning and uncording, node tainting)
capsule.clastix.io/enable-node-deletion: allows deletion of the node

The following Tenant annotations allow a sort of RBAC on the Storage Class operations:
capsule.clastix.io/enable-storageclass-listing: allows listing of Storage Class and Storage Classes retrieval
capsule.clastix.io/enable-storageclass-update: allows the update of the Storage Class
capsule.clastix.io/enable-storageclass-deletion: allows deletion of the Storage Class

The following Tenant annotations allow a sort of RBAC on the Ingress Class operations:
capsule.clastix.io/enable-ingressclass-listing: allows listing of Ingress Class and Ingress Classes retrieval
capsule.clastix.io/enable-ingressclass-update: allows the update of the Ingress Class
capsule.clastix.io/enable-ingressclass-deletion: allows deletion of the Ingress Class

maybe add some new field for capsule-proxy settings?

  proxySettings:
  - kind: nodes
    operations: ["LIST", "UPDATE", "DELETE"] //or allowed?
  - kind: storageclasses
    operations: ["LIST"]
  - kind: ingressclasses
    operations: ["LIST"]

so in the result (not including #50, which needs some discussion) it could be smth like:

package v1alpha2

import (
	corev1 "k8s.io/api/core/v1"
	networkingv1 "k8s.io/api/networking/v1"
	rbacv1 "k8s.io/api/rbac/v1"
	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

// TenantSpec defines the desired state of Tenant
type TenantSpec struct {
	Owner []OwnerSpec `json:"owner"` // or should it be Owners?

	//+kubebuilder:validation:Minimum=1
	NamespaceQuota         *int32                           `json:"namespaceQuota,omitempty"`
	NamespacesMetadata     AdditionalMetadata               `json:"namespacesMetadata,omitempty"`
	ServicesMetadata       AdditionalMetadata               `json:"servicesMetadata,omitempty"`
	StorageClasses         *AllowedListSpec                 `json:"storageClasses,omitempty"`
	IngressClasses         *AllowedListSpec                 `json:"ingressClasses,omitempty"`
	IngressHostnames       *AllowedListSpec                 `json:"ingressHostnames,omitempty"`
	ContainerRegistries    *AllowedListSpec                 `json:"containerRegistries,omitempty"`
	NodeSelector           map[string]string                `json:"nodeSelector,omitempty"`
	NetworkPolicies        []networkingv1.NetworkPolicySpec `json:"networkPolicies,omitempty"`
	LimitRanges            []corev1.LimitRangeSpec          `json:"limitRanges,omitempty"`
	ResourceQuota          []corev1.ResourceQuotaSpec       `json:"resourceQuotas,omitempty"`
	AdditionalRoleBindings []AdditionalRoleBindings         `json:"additionalRoleBindings,omitempty"`
	ExternalServiceIPs     *ExternalServiceIPs              `json:"externalServiceIPs,omitempty"`
        ImagePullPolicy        []ImagePullPolicy                `json:"allowedImagePullPolicies,omitempty"`
        PriorityClasses        *AllowedListSpec                 `json:"priorityClasses,omitempty"`
        ProxySettings          []ProxySettings                  `json:"proxySettings,omitempty"`  // or proxyRbac?
}

type AdditionalMetadata struct {
	AdditionalLabels      map[string]string `json:"additionalLabels,omitempty"`
	AdditionalAnnotations map[string]string `json:"additionalAnnotations,omitempty"`
}

// +kubebuilder:validation:Pattern="^([0-9]{1,3}.){3}[0-9]{1,3}(/([0-9]|[1-2][0-9]|3[0-2]))?$"
type AllowedIP string

type ExternalServiceIPs struct {
	Allowed []AllowedIP `json:"allowed"`
}

// +kubebuilder:validation:Enum=Always;Never;IfNotPresent
type ImagePullPolicy string

func (i ImagePullPolicy) String() string {
	return string(i)
}

type AdditionalRoleBindings struct {
	ClusterRoleName string `json:"clusterRoleName"`
	// kubebuilder:validation:Minimum=1
	Subjects []rbacv1.Subject `json:"subjects"`
}

// OwnerSpec defines tenant owner name and kind
type OwnerSpec struct {
	Kind OwnerKind   `json:"kind"`
	*AllowedListSpec
}

// +kubebuilder:validation:Enum=User;Group;ServiceAccount
type OwnerKind string

func (k OwnerKind) String() string {
	return string(k)
}

type ProxySettings struct {
	Kind       ProxyServiceKind `json:"kind"`
	Operations []ProxyOperation `json:"operations"`
}

// +kubebuilder:validation:Enum=List;Update;Delete
type ProxyOperation string 


func (p ProxyOperation) String() string {
	return string(k)
}

// +kubebuilder:validation:Enum=Nodes;Storageclasses;Ingressclasses
type ProxyServiceKind string 


func (p ProxyServiceKind) String() string {
	return string(k)
}

// TenantStatus defines the observed state of Tenant
type TenantStatus struct {
	Size       uint     `json:"size"`
	Namespaces []string `json:"namespaces,omitempty"`
}

[prometherion]

What a stunning job you did, @MaxFedotov! 🚀

Rather than ProxySettings, I was thinking that we could further decorate the OwnerSpec as follows.

// OwnerSpec defines tenant owner name and kind
type OwnerSpec struct {
	Kind OwnerKind   `json:"kind"`
	*AllowedListSpec
        ProxyOperations []ProxySettings
}

With this, we could have additional fine-grained permissions on users.

WDYT?

[MaxFedotov]

@prometherion great idea :)

[prometherion]

The new version has been updated to v1beta1 since we need this according to the deprecation policy of Kubernetes.

Error: failed to install CRD crds/tenant-crd.yaml: CustomResourceDefinition.apiextensions.k8s.io "tenants.capsule.clastix.io" is invalid: spec.conversion.conversionReviewVersions: Invalid value: []string{"v1alpha1", "v1alpha2"}: must include at least one of v1, v1beta1