Add more modifiable deployment fields to the Overrieders of the EdgeApplication

[WillardHu]

What would you like to be added/modified:

Add more modifiable deployment fields to the Overrieders of the EdgeApplication.

Why is this needed:

Currently, we can only override image and replicas of deployment differently for node groups in EdgeApplication.

In practice, we need to override more deployment fields for node groups, like: env, command, args, resources, and more ...

[Onion-of-dreamed]

/assign

[Shelley-BaoYue]

/milestone v1.16

[WillardHu]

/assign @tangming1996

[tangming1996]

In order to support EdgeApplication to support more differentiated configuration fields, the basic design scheme is presented here.

The four newly added differential configuration fields only use the replace and remove operation modes, and discard the add operation. The specific api is defined as follows:

// Overriders represents the override rules that would apply on resources.
type Overriders struct {
	// Replicas will override the replicas field of deployment
	// +optional
	Replicas *int `json:"replicas,omitempty"`
	// ImageOverriders represents the rules dedicated to handling image overrides.
	// +optional
	ImageOverriders []ImageOverrider `json:"imageOverriders,omitempty"`
	// Env will override the env field of the container
	// +optional
	Env EnvOverrider `json:"env,omitempty"`
	// Command will override the command field of the container
	// +optional
	Command CommandOverrider `json:"command,omitempty"`
	// Args will override the args field of the container
	// +optional
	Args ArgsOverrider `json:"args,omitempty"`
	// Resources will override the resources field of the container
	// +optional
	Resources ResourcesOverrider `json:"resources,omitempty"`
}

// EnvOverrider represents the rules dedicated to handling env overrides.
type EnvOverrider struct {
	// Path indicates the path of target field.
	//
	// Defaults to nil, in that case, the system will automatically detect env fields if the resource type is
	// Pod, ReplicaSet, Deployment or StatefulSet by following rule:
	//   - Pod: /spec/containers/<N>/env
	//   - ReplicaSet: /spec/template/spec/containers/<N>/env
	//   - Deployment: /spec/template/spec/containers/<N>/env
	//   - StatefulSet: /spec/template/spec/containers/<N>/env
	// In addition, all env will be processed if the resource object has more than one containers.
	//
	// If not nil, only env matches the filters will be processed.
	// +optional
	Path string `json:"path,omitempty"`

	// Operator represents the operator which will apply on the env.
	// +kubebuilder:validation:Enum=add;remove;replace
	// +required
	Operator OverriderOperator `json:"operator"`

	// Value to be applied to env.
	// Must not be empty when operator is 'add' or 'replace'.
	// Defaults to empty and ignored when operator is 'remove'.
	// +optional
	Value []corev1.EnvVar `json:"value,omitempty"`
}

// CommandOverrider represents the rules dedicated to handling command overrides.
type CommandOverrider struct {
	// Path indicates the path of target field.
	//
	// Defaults to nil, in that case, the system will automatically detect command fields if the resource type is
	// Pod, ReplicaSet, Deployment or StatefulSet by following rule:
	//   - Pod: /spec/containers/<N>/command
	//   - ReplicaSet: /spec/template/spec/containers/<N>/command
	//   - Deployment: /spec/template/spec/containers/<N>/command
	//   - StatefulSet: /spec/template/spec/containers/<N>/command
	// In addition, all command will be processed if the resource object has more than one containers.
	//
	// If not nil, only command matches the filters will be processed.
	// +optional
	Path string `json:"path,omitempty"`

	// Operator represents the operator which will apply on the command.
	// +kubebuilder:validation:Enum=add;remove;replace
	// +required
	Operator OverriderOperator `json:"operator"`

	// Value to be applied to command.
	// Must not be empty when operator is 'add' or 'replace'.
	// Defaults to empty and ignored when operator is 'remove'.
	// +optional
	Value []string `json:"value,omitempty"`
}

// ArgsOverrider represents the rules dedicated to handling args overrides.
type ArgsOverrider struct {
	// Path indicates the path of target field.
	//
	// Defaults to nil, in that case, the system will automatically detect args fields if the resource type is
	// Pod, ReplicaSet, Deployment or StatefulSet by following rule:
	//   - Pod: /spec/containers/<N>/args
	//   - ReplicaSet: /spec/template/spec/containers/<N>/args
	//   - Deployment: /spec/template/spec/containers/<N>/args
	//   - StatefulSet: /spec/template/spec/containers/<N>/args
	// In addition, all args will be processed if the resource object has more than one containers.
	//
	// If not nil, only args matches the filters will be processed.
	// +optional
	Path string `json:"path,omitempty"`

	// Operator represents the operator which will apply on the args.
	// +kubebuilder:validation:Enum=add;remove;replace
	// +required
	Operator OverriderOperator `json:"operator"`

	// Value to be applied to args.
	// Must not be empty when operator is 'add' or 'replace'.
	// Defaults to empty and ignored when operator is 'remove'.
	// +optional
	Value []string `json:"value,omitempty"`
}

// ResourcesOverrider represents the rules dedicated to handling resources overrides.
type ResourcesOverrider struct {
	// Path indicates the path of target field.
	//
	// Defaults to nil, in that case, the system will automatically detect resources fields if the resource type is
	// Pod, ReplicaSet, Deployment or StatefulSet by following rule:
	//   - Pod: /spec/containers/<N>/resources
	//   - ReplicaSet: /spec/template/spec/containers/<N>/resources
	//   - Deployment: /spec/template/spec/containers/<N>/resources
	//   - StatefulSet: /spec/template/spec/containers/<N>/resources
	// In addition, all resources will be processed if the resource object has more than one containers.
	//
	// If not nil, only resources matches the filters will be processed.
	// +optional
	Path string `json:"path,omitempty"`

	// Operator represents the operator which will apply on the resources.
	// +kubebuilder:validation:Enum=add;remove;replace
	// +required
	Operator OverriderOperator `json:"operator"`

	// Value to be applied to resources.
	// Must not be empty when operator is 'add' or 'replace'.
	// Defaults to empty and ignored when operator is 'remove'.
	// +optional
	Value corev1.ResourceRequirements `json:"value,omitempty"`
}

Concrete realization in kubeedge/cloud/pkg/controllermanager/edgeapplication/overridemanager directory add four files, as follows:

├── envoverrider.go
├── resourceoverrider.go
├── argoverrider.go
├── commandoverrider.go

Then, the Override interface is implemented separately to realize the differentiated configuration of env, command, args, and resoruces

type Overrider interface {
	ApplyOverrides(rawObjs *unstructured.Unstructured, overrideInfo OverriderInfo) error
}

[WillardHu]

/unassign @Onion-of-dreamed