TEP-0061, Allow custom task to be embedded.

docs/pipelines.md

@@ -1179,6 +1179,31 @@ If the `taskRef` specifies a name, the custom task controller should look up the
 If the `taskRef` does not specify a name, the custom task controller might support
 some default behavior for executing unnamed tasks.
 
+### Specifying a Custom Task Spec in-line (or embedded)
+
+```yaml
+spec:
+  tasks:
+    - name: run-custom-task
+      taskSpec:
+        apiVersion: example.dev/v1alpha1
+        kind: Example
+          spec:
+            field1: value1
+            field2: value2
+```
+
+If the custom task controller supports the in-line or embedded task spec, this will create a `Run` of a custom task of
+type `Example` in the `example.dev` API group with the version `v1alpha1`.
+
+If the `taskSpec` is not supported, the custom task controller should produce proper validation errors.
+
+Please take a look at the
+[developer guide for custom controllers supporting `taskSpec`.](runs.md#developer-guide-for-custom-controllers-supporting-spec)
+
+`taskSpec` support for `pipelineRun` was designed and discussed in
+[TEP-0061](https://github.com/tektoncd/community/blob/main/teps/0061-allow-custom-task-to-be-embedded-in-pipeline.md)
+
 ### Specifying parameters
 
 If a custom task supports [`parameters`](tasks.md#parameters), you can use the

docs/runs.md

@@ -47,8 +47,10 @@ A `Run` definition supports the following fields:
   - [`metadata`][kubernetes-overview] - Specifies the metadata that uniquely identifies the
     `Run`, such as a `name`.
   - [`spec`][kubernetes-overview] - Specifies the configuration for the `Run`.
-    - [`ref`](#specifying-the-target-custom-task) - Specifies the type and
+    - [`ref`](#1-specifying-the-target-custom-task-with-ref) - Specifies the type and
       (optionally) name of the custom task type to execute.
+    - [`spec`](#2-specifying-the-target-custom-task-by-embedding-its-spec) - Embed the custom task resource spec
+      directly in a `Run`.
 - Optional:
   - [`params`](#specifying-parameters) - Specifies the desired execution
     parameters for the custom task.
@@ -64,6 +66,19 @@ A `Run` definition supports the following fields:
 
 ### Specifying the target Custom Task
 
+A custom task resource's `Spec` may be directly embedded in the `Run` or it may
+be referred to by a `Ref`. But, not both at the same time.
+
+1. [Specifying the target Custom Task with ref](#1-specifying-the-target-custom-task-with-ref)
+   Referring a custom task (i.e. `Ref` ) promotes reuse of custom task definitions.
+
+2. [Specifying the target Custom Task by embedding its spec](#2-specifying-the-target-custom-task-by-embedding-its-spec)
+   Embedding a custom task (i.e. `Spec` ) helps in avoiding name collisions with other users within the same namespace.
+   Additionally, in a pipeline with multiple embedded custom tasks, the details of entire pipeline can be fetched in a
+   single API request.
+
+### 1. Specifying the target Custom Task with ref
+
 To specify the custom task type you want to execute in your `Run`, use the
 `ref` field as shown below:
 
@@ -99,6 +114,45 @@ In either case, if the named resource cannot be found, or if unnamed tasks are
 not supported, the custom task controller should update the `Run`'s status to
 indicate the error.
 
+### 2. Specifying the target Custom Task by embedding its spec
+
+To specify the custom task spec, it can be embedded directly into a
+`Run`'s spec as shown below:
+
+```yaml
+apiVersion: tekton.dev/v1alpha1
+kind: Run
+metadata:
+  name: embedded-run
+spec:
+  spec:
+    apiVersion: example.dev/v1alpha1
+    kind: Example
+    spec:
+      field1: value1
+      field2: value2
+```
+
+This initiates the execution of a `Run` of a custom task of type `Example`, in
+the `example.dev` API group,  with the version `v1alpha1`.
+
+#### Developer guide for custom controllers supporting `spec`.
+
+1. A custom controller may or may not support a `Spec`. In cases where it is
+   not supported the custom controller should respond with proper validation error.
+
+2. Validation of the fields of the custom task is delegated to the custom task controller. It is recommended to
+   implement validations as asynchronous
+   (i.e. at reconcile time), rather than part of the webhook. Using a webhook for validation is problematic because, it
+   is not possible to filter custom task resource objects before validation step, as a result each custom task resource
+   has to undergo validation by all the installed custom task controllers.
+
+3. A custom task may have an empty spec, but cannot have an empty
+    `ApiVersion` and `Kind`. Custom task controllers should handle
+   an empty spec, either with a default behaviour, in a case no default
+   behaviour is supported then, appropriate validation error should be
+   updated to the `Run`'s status.
+
 ### Specifying `Parameters`
 
 If a custom task supports [`parameters`](tasks.md#parameters), you can use the

internal/builder/v1beta1/pipeline_test.go

@@ -156,10 +156,10 @@ func TestPipeline(t *testing.T) {
 						Labels:      map[string]string{"label": "labelvalue"},
 						Annotations: map[string]string{"annotation": "annotationvalue"},
 					},
+
 					TaskSpec: getTaskSpec(),
 				},
-			},
-			},
+			}},
 			Workspaces: []v1beta1.PipelineWorkspaceDeclaration{{
 				Name: "workspace1",
 			}},

pkg/apis/pipeline/v1alpha1/run_types.go

@@ -20,9 +20,10 @@ import (
 	"fmt"
 
 	"github.com/tektoncd/pipeline/pkg/apis/pipeline"
-	v1beta1 "github.com/tektoncd/pipeline/pkg/apis/pipeline/v1beta1"
+	"github.com/tektoncd/pipeline/pkg/apis/pipeline/v1beta1"
 	runv1alpha1 "github.com/tektoncd/pipeline/pkg/apis/run/v1alpha1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/apimachinery/pkg/runtime"
 	"k8s.io/apimachinery/pkg/runtime/schema"
 	"knative.dev/pkg/apis"
 	duckv1 "knative.dev/pkg/apis/duck/v1"
@@ -36,11 +37,27 @@ var (
 	}
 )
 
+// EmbeddedRunSpec allows custom task definitions to be embedded
+type EmbeddedRunSpec struct {
+	runtime.TypeMeta `json:",inline"`
+
+	// +optional
+	Metadata v1beta1.PipelineTaskMetadata `json:"metadata,omitempty"`
+
+	// Spec is a specification of a custom task
+	// +optional
+	Spec runtime.RawExtension `json:"spec,omitempty"`
+}
+
 // RunSpec defines the desired state of Run
 type RunSpec struct {
 	// +optional
 	Ref *TaskRef `json:"ref,omitempty"`
 
+	// Spec is a specification of a custom task
+	// +optional
+	Spec *EmbeddedRunSpec `json:"spec,omitempty"`
+
 	// +optional
 	Params []v1beta1.Param `json:"params,omitempty"`
 

pkg/apis/pipeline/v1alpha1/run_validation.go

@@ -36,20 +36,30 @@ func (r *Run) Validate(ctx context.Context) *apis.FieldError {
 
 // Validate Run spec
 func (rs *RunSpec) Validate(ctx context.Context) *apis.FieldError {
+	// this covers the case rs.Ref == nil && rs.Spec == nil
 	if equality.Semantic.DeepEqual(rs, &RunSpec{}) {
 		return apis.ErrMissingField("spec")
 	}
 
-	if rs.Ref == nil {
-		return apis.ErrMissingField("spec.ref")
+	if rs.Ref != nil && rs.Spec != nil {
+		return apis.ErrMultipleOneOf("spec.ref", "spec.spec")
 	}
-	if rs.Ref.APIVersion == "" {
-		return apis.ErrMissingField("spec.ref.apiVersion")
+	if rs.Ref != nil {
+		if rs.Ref.APIVersion == "" {
+			return apis.ErrMissingField("spec.ref.apiVersion")
+		}
+		if rs.Ref.Kind == "" {
+			return apis.ErrMissingField("spec.ref.kind")
+		}
 	}
-	if rs.Ref.Kind == "" {
-		return apis.ErrMissingField("spec.ref.kind")
+	if rs.Spec != nil {
+		if rs.Spec.APIVersion == "" {
+			return apis.ErrMissingField("spec.spec.apiVersion")
+		}
+		if rs.Spec.Kind == "" {
+			return apis.ErrMissingField("spec.spec.kind")
+		}
 	}
-
 	if err := validateParameters("spec.params", rs.Params); err != nil {
 		return err
 	}

pkg/apis/pipeline/v1alpha1/run_validation_test.go

@@ -26,6 +26,7 @@ import (
 	"github.com/tektoncd/pipeline/test/diff"
 	corev1 "k8s.io/api/core/v1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/apimachinery/pkg/runtime"
 	"knative.dev/pkg/apis"
 )
 
@@ -43,18 +44,39 @@ func TestRun_Invalid(t *testing.T) {
 		},
 		want: apis.ErrMissingField("spec"),
 	}, {
-		name: "missing ref",
+		name: "Empty spec",
 		run: &v1alpha1.Run{
 			ObjectMeta: metav1.ObjectMeta{
 				Name: "temp",
 			},
 			Spec: v1alpha1.RunSpec{
-				Ref: nil,
+				Ref:  nil,
+				Spec: nil,
 			},
 		},
 		want: apis.ErrMissingField("spec"),
 	}, {
-		name: "missing apiVersion",
+		name: "Both Ref and Spec",
+		run: &v1alpha1.Run{
+			ObjectMeta: metav1.ObjectMeta{
+				Name: "temp",
+			},
+			Spec: v1alpha1.RunSpec{
+				Ref: &v1alpha1.TaskRef{
+					APIVersion: "apiVersion",
+					Kind:       "kind",
+				},
+				Spec: &v1alpha1.EmbeddedRunSpec{
+					TypeMeta: runtime.TypeMeta{
+						APIVersion: "apiVersion",
+						Kind:       "kind",
+					},
+				},
+			},
+		},
+		want: apis.ErrMultipleOneOf("spec.ref", "spec.spec"),
+	}, {
+		name: "missing apiVersion in Ref",
 		run: &v1alpha1.Run{
 			ObjectMeta: metav1.ObjectMeta{
 				Name: "temp",
@@ -67,7 +89,7 @@ func TestRun_Invalid(t *testing.T) {
 		},
 		want: apis.ErrMissingField("spec.ref.apiVersion"),
 	}, {
-		name: "missing kind",
+		name: "missing kind in Ref",
 		run: &v1alpha1.Run{
 			ObjectMeta: metav1.ObjectMeta{
 				Name: "temp",
@@ -80,6 +102,37 @@ func TestRun_Invalid(t *testing.T) {
 			},
 		},
 		want: apis.ErrMissingField("spec.ref.kind"),
+	}, {
+		name: "missing apiVersion in Spec",
+		run: &v1alpha1.Run{
+			ObjectMeta: metav1.ObjectMeta{
+				Name: "temp",
+			},
+			Spec: v1alpha1.RunSpec{
+				Spec: &v1alpha1.EmbeddedRunSpec{
+					TypeMeta: runtime.TypeMeta{
+						APIVersion: "",
+					},
+				},
+			},
+		},
+		want: apis.ErrMissingField("spec.spec.apiVersion"),
+	}, {
+		name: "missing kind in Spec",
+		run: &v1alpha1.Run{
+			ObjectMeta: metav1.ObjectMeta{
+				Name: "temp",
+			},
+			Spec: v1alpha1.RunSpec{
+				Spec: &v1alpha1.EmbeddedRunSpec{
+					TypeMeta: runtime.TypeMeta{
+						APIVersion: "apiVersion",
+						Kind:       "",
+					},
+				},
+			},
+		},
+		want: apis.ErrMissingField("spec.spec.kind"),
 	}, {
 		name: "non-unique params",
 		run: &v1alpha1.Run{
@@ -142,6 +195,21 @@ func TestRun_Valid(t *testing.T) {
 				},
 			},
 		},
+	}, {
+		name: "Spec with valid ApiVersion and Kind",
+		run: &v1alpha1.Run{
+			ObjectMeta: metav1.ObjectMeta{
+				Name: "temp",
+			},
+			Spec: v1alpha1.RunSpec{
+				Spec: &v1alpha1.EmbeddedRunSpec{
+					TypeMeta: runtime.TypeMeta{
+						APIVersion: "apiVersion",
+						Kind:       "kind",
+					},
+				},
+			},
+		},
 	}, {
 		name: "unique params",
 		run: &v1alpha1.Run{