Skip to content

Commit

Permalink
Introducing a new Task specification (#859)
Browse files Browse the repository at this point in the history 
We introduce a new `Task` specification, which will allow us to better encapsulate different kinds of tasks (e.g. Helm, Pipe-tasks, etc) in the future. Here is how the API changed:
*Before:* tasks were a list of resources, which were applied in step with an optional `delete: true` signaling, that resources have to be deleted.
```
# operator.yaml
  ...
  tasks:
    deploy:
      resources:
        - deploy.yaml
    remove:
      reources:
        - deploy.yaml

  plans:
    deploy:
      strategy: serial
      phases:
        - name: phase
          strategy: serial
          steps:
            - name: step
              tasks:
                - deploy
    remove:
      strategy: serial
      phases:
        - name: phase
          strategy: serial
          steps:
            - name: step
              tasks:
                - remove
              delete: true
```
The new task definition requires specifying tasks `name`, `kind` and `spec` fields. Here is the same example using new `Apply` and `Delete` tasks kinds:
```
  tasks:
    - Name: deploy
      Kind: Apply
      Spec:
        resources:
          - deploy.yaml
    - Name: uninstall
      Kind: Delete
      Spec:
        resources:
          - deploy.yaml
```
Its usage from the step is almost the same, merely `delete: true` has been removed.

Additionally, plan execution logic was reworked, It now respects phases strategy and propagates fatal and transient execution errors differently (see [this comment](https://github.com/kudobuilder/kudo/pull/859/files#diff-bec59313038b1b7f55ce3d37d3445a32R39) for more information).

Fixes: #839

Co-authored-by: Aleksey Dukhovniy <adukhovniy@mesosphere.io>
  • Loading branch information
@gerred @zen-dog
2 people authored and zen-dog committed Oct 19, 2019
1 parent 5f0439d commit 8827667
Show file tree
Hide file tree
Showing 43 changed files with 1,925 additions and 596 deletions.
71 changes: 38 additions & 33 deletions config/crds/kudo_v1alpha1_operatorversion.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,50 +3,47 @@ kind: CustomResourceDefinition
metadata:
creationTimestamp: null
labels:
app: kudo-manager
controller-tools.k8s.io: "1.0"
name: operatorversions.kudo.dev
spec:
group: kudo.dev
names:
kind: OperatorVersion
plural: operatorversions
singular: operatorversion
scope: Namespaced
validation:
openAPIV3Schema:
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds'
type: string
metadata:
meta:
type: object
spec:
properties:
connectionString:
description: ConnectionString defines a templated string that can be
used to connect to an instance of the Operator.
description: ConnectionString defines a mustached string that can be
used to connect to an instance of the Operator
type: string
crdVersion:
type: string
dependencies:
description: Dependencies a list of all dependencies of the operator.
items:
properties:
referenceName:
description: Name specifies the name of the dependency. Referenced
via defaults.config.
type: string
version:
crdVersion:
description: 'Version captures the requirements for what versions
of the above object are allowed. Example: ^3.1.4'
of the above object are allowed Example: ^3.1.4'
type: string
referenceName:
description: Name specifies the name of the dependency. Referenced
via this in defaults.config
type: string
required:
- referenceName
- version
- crdVersion
type: object
type: array
operator:
Expand All @@ -56,55 +53,63 @@ spec:
properties:
default:
description: Default is a default value if no parameter is provided
by the instance.
by the instance
type: string
description:
description: Description captures a longer description of how
the parameter will be used.
the variable will be used
type: string
displayName:
description: DisplayName can be used by UI's.
description: Human friendly crdVersion of the parameter name
type: string
name:
description: 'Name is the string that should be used in the templated
file for example, if `name: COUNT` then using the variable in
a spec like: spec: replicas: {{COUNT}}'
description: 'Name is the string that should be used in the template
file for example, if `name: COUNT` then using the variable `.Params.COUNT`'
type: string
required:
description: Required specifies if the parameter is required to
be provided by all instances, or whether a default can suffice.
be provided by all instances, or whether a default can suffice
type: boolean
trigger:
description: Trigger identifies the plan that gets executed when
this parameter changes in the Instance object. Default is `update`
if a plan with that name exists, otherwise it's `deploy`
if present, or `deploy` if not present
type: string
type: object
type: array
plans:
description: Plans maps a plan name to a plan.
description: Plans specify a map a plans that specify how to
type: object
tasks:
type: object
description: List of all tasks available in this OperatorVersions
items:
properties:
kind:
type: string
name:
type: string
spec:
type: object
type: object
type: array
templates:
description: Yaml captures a templated yaml list of elements that define
the application operator instance.
description: List of go templates YAML files that define the application
operator instance
type: object
upgradableFrom:
description: UpgradableFrom lists all OperatorVersions that can upgrade
to this OperatorVersion.
to this OperatorVersion
items:
type: object
type: array
version:
type: string
type: object
status:
type: object
type: object
version: v1alpha1
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
storedVersions: []
8 changes: 5 additions & 3 deletions config/samples/first-operator/operator.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,11 @@ maintainers:
email: <your@email.com>
url: https://kudo.dev
tasks:
app:
resources:
- deployment.yaml
- name: app
kind: Apply
spec:
resources:
- deployment.yaml
plans:
deploy:
strategy: serial
Expand Down
2 changes: 2 additions & 0 deletions go.mod
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ require (
github.com/Masterminds/semver v1.4.2
github.com/Microsoft/go-winio v0.4.14 // indirect
github.com/containerd/containerd v1.2.9 // indirect
github.com/davecgh/go-spew v1.1.1
github.com/docker/distribution v2.7.1+incompatible // indirect
github.com/docker/docker v1.4.2-0.20190916154449-92cc603036dd
github.com/docker/go-connections v0.4.0 // indirect
Expand Down Expand Up @@ -50,6 +51,7 @@ require (
go.uber.org/atomic v1.4.0 // indirect
go.uber.org/zap v1.10.0 // indirect
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3
golang.org/x/net v0.0.0-20190620200207-3b0461eec859
golang.org/x/oauth2 v0.0.0-20190402181905-9f3314589c9a // indirect
golang.org/x/sys v0.0.0-20190911201528-7ad0cfa0b7b5 // indirect
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4
Expand Down
2 changes: 1 addition & 1 deletion pkg/apis/kudo/v1alpha1/instance_types.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ type AggregatedStatus struct {

// PlanStatus is representing status of a plan
//
// These are valid states and trainsitions
// These are valid states and transitions
//
// +----------------+
// | Never executed |
Expand Down
55 changes: 40 additions & 15 deletions pkg/apis/kudo/v1alpha1/operatorversion_types.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@ type OperatorVersionSpec struct {
Version string `json:"version,omitempty"`

// Yaml captures a templated yaml list of elements that define the application operator instance.
Templates map[string]string `json:"templates,omitempty"`
Tasks map[string]TaskSpec `json:"tasks,omitempty"`
Templates map[string]string `json:"templates,omitempty"`
Tasks []Task `json:"tasks,omitempty"`

Parameters []Parameter `json:"parameters,omitempty"`

Expand All @@ -50,12 +50,14 @@ type OperatorVersionSpec struct {
// Ordering specifies how the subitems in this plan/phase should be rolled out.
type Ordering string

// Serial specifies that the plans or objects should be created in order. The first should be healthy before
// continuing on.
const Serial Ordering = "serial"
const (
// Serial specifies that the plans or objects should be created in order. The first should be healthy before
// continuing on.
Serial Ordering = "serial"

// Parallel specifies that the plan or objects in the phase can all be launched at the same time.
const Parallel Ordering = "parallel"
// Parallel specifies that the plan or objects in the phase can all be launched at the same time.
Parallel Ordering = "parallel"
)

// Plan specifies a series of Phases that need to be completed.
type Plan struct {
Expand Down Expand Up @@ -95,11 +97,6 @@ type Parameter struct {

}

// TaskSpec is a struct containing lists of Kustomize resources.
type TaskSpec struct {
Resources []string `json:"resources"`
}

// Phase specifies a list of steps that contain Kubernetes objects.
type Phase struct {
Name string `json:"name" validate:"required"` // makes field mandatory and checks if set and non empty
Expand All @@ -111,14 +108,42 @@ type Phase struct {

// Step defines a specific set of operations that occur.
type Step struct {
Name string `json:"name" validate:"required"` // makes field mandatory and checks if set and non empty
Tasks []string `json:"tasks" validate:"required,gt=0,dive,required"` // makes field mandatory and checks if non empty
Delete bool `json:"delete,omitempty"` // no checks needed
Name string `json:"name" validate:"required"` // makes field mandatory and checks if set and non empty
Tasks []string `json:"tasks" validate:"required,gt=0,dive"` // makes field mandatory and checks if non empty
Delete bool `json:"delete,omitempty"` // no checks needed

// Objects will be serialized for each instance as the params and defaults are provided.
Objects []runtime.Object `json:"-"` // no checks needed
}

// Task is a global, polymorphic implementation of all publicly available tasks
type Task struct {
Name string `json:"name" validate:"required"`
Kind string `json:"kind" validate:"required"`
Spec TaskSpec `json:"spec" validate:"required"`
}

// TaskSpec embeds all possible task specs. This allows us to avoid writing custom un/marshallers that would only parse
// certain fields depending on the task Kind. The downside of this approach is, that embedded types can not have fields
// with the same json names as it would become ambiguous for the default parser. We might revisit this approach in the
// future should this become an issue.
type TaskSpec struct {
ResourceTaskSpec
DummyTaskSpec
}

// ResourceTaskSpec is referencing a list of resources
type ResourceTaskSpec struct {
Resources []string `json:"resources"`
}

// DummyTaskSpec can succeed of fail on demand and is very useful for testing operators
type DummyTaskSpec struct {
WantErr bool `json:"wantErr"`
Fatal bool `json:"fatal"`
Done bool `json:"done"`
}

// OperatorVersionStatus defines the observed state of OperatorVersion.
type OperatorVersionStatus struct {
// INSERT ADDITIONAL STATUS FIELD - define observed state of cluster
Expand Down
67 changes: 59 additions & 8 deletions pkg/apis/kudo/v1alpha1/zz_generated.deepcopy.go
View file

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 8827667

Please sign in to comment.