Use CUE to describe a Definition

[hongchaodeng]

Background

Currently, the Definition object is a pure YAML file but blended with CUE raw string:

https://github.com/oam-dev/kubevela/blob/b124873fd73b8c1ca02a351fc8d4dbb2ee00fca1/charts/vela-core/templates/defwithtemplate/webservice.yaml#L2-L19

This is not very user-friendly. We shouldn't let user write such YAML.

Proposal

We should use CUE to describe a Definition, and then provide CLI tool to parse them into k8s object.

Here's an example. In user's IDE, a definition would be a folder:

definitions/
|
|-- webservice/
    |
    |-- kube.cue # Optional. We can still infer from folder name.
    |-- schema.cue

kube.cue would look like:

metadata: kube.metadata & {
  name: "webservice"
}

spec: kube.definitionSpec & {
  workload:
    apiVersion: "apps/v1"
    kind: "Deployment"
}

schema.cue would look like:

output: {
	apiVersion: "apps/v1"
	kind:       "Deployment"
	spec: {
		selector: matchLabels: {
			"app.oam.dev/component": context.name
			if parameter.addRevisionLabel {
				"app.oam.dev/appRevision": context.appRevision
			}
		}
...
}

parameter: {
	// +usage=Which image would you like to use for your service
	// +short=i
	image: string

	// +usage=Commands to run in the container
	cmd?: [...string]
...
}

Then provide CLI or script to blend them together to generate definition object.

[zzxwill]

• Seems the way how our built-in templates are assembled is more friendly

├── cue
│   ├── annotations.cue
│   ├── cpuscaler.cue
│   ├── expose.cue
│   ├── ingress.cue
│   ├── labels.cue
│   ├── scaler.cue
│   ├── service-binding.cue
│   ├── sidecar.cue
│   ├── task.cue
│   ├── webservice.cue
│   └── worker.cue
├── definitions
│   ├── annotations.yaml
│   ├── cpuscaler.yaml
│   ├── expose.yaml
│   ├── ingress.yaml
│   ├── labels.yaml
│   ├── scaler.yaml
│   ├── service-binding.yaml
│   ├── sidecar.yaml
│   ├── task.yaml
│   ├── webservice.yaml
│   └── worker.yaml

Everyone is familiar with the yaml section.

• Should definitions like helm or Terraform use CUE?

This is a ComponentDefinition of Terraform OSS, currently it has nothing to do with CUE.

apiVersion: core.oam.dev/v1alpha2
kind: ComponentDefinition
metadata:
  name: alibaba-oss
  annotations:
    definition.oam.dev/description: Terraform configuration for Alibaba Cloud OSS object
    type: terraform
spec:
  workload:
    definition:
      apiVersion: terraform.core.oam.dev/v1beta1
      kind: Configuration
  schematic:
    terraform:
      configuration: |
        resource "alicloud_oss_bucket" "bucket-acl" {
          bucket = var.bucket
          acl = var.acl
        }

        output "BUCKET_NAME" {
          value = "${alicloud_oss_bucket.bucket-acl.bucket}.${alicloud_oss_bucket.bucket-acl.extranet_endpoint}"
        }

        variable "bucket" {
          description = "OSS bucket name"
          default = "vela-website"
          type = string
        }

        variable "acl" {
          description = "OSS bucket ACL, supported 'private', 'public-read', 'public-read-write'"
          default = "private"
          type = string
        }

[leejanee]

It is good idea that Use cue to manage the definition through the command line

Solution 1:
The ‘definition’ sub-command should be supported, for example

// get definition from vela
vela definition get componentDefinition/webservice  --format=cue 

// generate definition
vela definition apply ./

// validate...and output the rendered result
vela definition validate --input-file=./params.cue 

Solution 2:
Users can extend the capabilities of vela command line by customizing tasks

vela task run --name=comp
// In the 'webservice' directory

 webservice/
    |
    |-- meta.cue 
    |-- schema.cue

---tasks/comp.cue---
import ( 
    "vela/op" 
 )

loadmeta: op.#Load & {
           source: "./meta.cue"
           format: "CUE"
           out: {}
 }
loadSchema: op.#Load & {
           source: "./schema.cue"
           format: "Text"
           out: string
}
validate: op.#Script & {
           vars: loadmeta.out.#validate
           script:  loadSchema.out
           out: loadmeta.out.#validate.except
}
export: op.#Apply & {
          apiVersion: "core.oam.dev/v1beta1"
          kind: "ComponentDefinition" 
          metadata:  name: loadmeta.out.name
          spec: workload: definition:  loadmeta.out.spec.workload
          schematic: 
            cue: 
                template: "\(loadSchema.out)"
}


---schema.cue---
context: _

output: {
	apiVersion: "apps/v1"
	kind:       "Deployment"
	spec: {
		selector: matchLabels: {
			"app.oam.dev/component": context.name
			if parameter.addRevisionLabel {
				"app.oam.dev/appRevision": context.appRevision
			}
		}
...
}

parameter: {
	// +usage=Which image would you like to use for your service
	// +short=i
	image: string

	// +usage=Commands to run in the container
	cmd?: [...string]
...
}

----./meta.cue----
 #metadata:  {
    name: "webservice"
    workload: {
         apiVersion: "apps/v1"
         kind: "Deployment"
   }
}

#validate: {
  context: {...}
  paramter: {...}
  except: {...}
}

[resouer]

@wonderflow , @zzxwill follow up comments regarding to your thoughts on Terraform components:

Should definitions like helm or Terraform use CUE?

Yes, exactly. All components, traits, workflow steps should be CUE modules, and I don't even think a K8s entity wrapper is needed for X-Definitions (could be fixed later of course).

Also, I don't think embedding HCL in ComponentDefinition is a reasonable approach for long run. The Terraform component should also be a generic definition like what I proposed for Helm component: it should reference Terraform modules from registry or GitHub repo. Users should be able to compose, debug, test their Terraform modules like how they do today, and when they want to deploy the whole unit (i.e. the full app + infra), they go to KubeVela. They should not be exposed to another layer of complex like defining ComponentDefinition for given cloud service.

I noticed this issue is still focusing on web service etc - they will become niche features in upcoming v1.1~1.2 releases, more like toy demos to show how to glue raw k8s resources together w/o helm or kustomize.

[hongchaodeng]

There have been some offline discussion I had with @Somefive . Let me sum it up here:

• Previously we want to unify them in a single format, and we think CUE can do that. But over time we found users have difficulty in understanding CUE well enough to do anything. We think that using CUE is a bad way to unify it.
• There are two parts in a Definition: CUE glue code and the metadata.
• The CUE compartment is flexible for any user input, and we couldn't predefine it. We should keep this part as is and let user provide text or file input.
• The metadata part is static. For this part, we can provide a better interface for users. For example, in UI, we can provide forms; in CLI, we can use flags or questions.