adding metadata to taskSpec in PipelineTask #2826

pritidesai · 2020-06-17T02:10:04Z

Changes

Adding metadata to taskSpec within PipelineTask to allow specifying labels and annotations when a task is embedded using taskSpec. These labels and annotations are propagated to taskRun and then to the pods.

apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  name: pipelinerun-with-taskspec-to-echo-greetings
spec:
  pipelineSpec:
    tasks:
      - name: echo-greetings
         taskSpec:
           metadata:
             labels: [ …]
             annotations: [...]
         steps:
...

Metadata is already supported as part of Tasks and Pipelines while respective CRDs are created. But was not possible to specify with embedded resources.

These labels are commonly used to identify and filter pods for further actions (such as collecting pod metrics, and cleaning up completed pod with certain labels, etc) even being part of one single Pipeline.

  pipelineSpec:
    tasks:
    - name: echo
      metadata:
        labels:
          pipeline-sdk-type: kfp
      taskSpec:
       ...
    - name: echo2
      metadata:
        labels:
          pipeline-sdk-type: tfx
      taskSpec:
       ...

Closes #2767

Submitter Checklist

These are the criteria that every PR should meet, please check them off as you
review them:

Includes tests (if functionality changed/added)
Includes docs (if user facing)
Commit messages follow commit message best practices
Release notes block has been filled in or deleted (only if no user facing changes)

See the contribution guide for more details.

Double check this list of stuff that's easy to miss:

If you are adding a new binary/image to the cmd dir, please update
the release Task to build and release this image.

Reviewer Notes

If API changes are included, additive changes must be approved by at least two OWNERS and backwards incompatible changes must be approved by more than 50% of the OWNERS, and they must first be added in a backwards compatible way.

Release Notes

Pipeline authors can now specify metadata while embedding tasks (using taskSpec) into their pipeline.

tekton-robot · 2020-06-17T02:10:26Z

This PR cannot be merged: expecting exactly one kind/ label

Available kind/ labels are:

kind/bug: Categorizes issue or PR as related to a bug.
kind/flake: Categorizes issue or PR as related to a flakey test
kind/cleanup: Categorizes issue or PR as related to cleaning up code, process, or technical debt.
kind/design: Categorizes issue or PR as related to design.
kind/documentation: Categorizes issue or PR as related to documentation.
kind/feature: Categorizes issue or PR as related to a new feature.
kind/misc: Categorizes issue or PR as a miscellaneuous one.

tekton-robot · 2020-06-17T02:10:38Z

This PR cannot be merged: expecting exactly one kind/ label

Available kind/ labels are:

kind/bug: Categorizes issue or PR as related to a bug.
kind/flake: Categorizes issue or PR as related to a flakey test
kind/cleanup: Categorizes issue or PR as related to cleaning up code, process, or technical debt.
kind/design: Categorizes issue or PR as related to design.
kind/documentation: Categorizes issue or PR as related to documentation.
kind/feature: Categorizes issue or PR as related to a new feature.
kind/misc: Categorizes issue or PR as a miscellaneuous one.

pritidesai · 2020-06-17T02:11:28Z

/kind feature

tekton-robot · 2020-06-17T02:12:45Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
internal/builder/v1beta1/pipeline.go	82.0%	82.2%	0.2
pkg/apis/pipeline/v1beta1/pipeline_types.go	62.1%	67.6%	5.5
pkg/apis/pipeline/v1beta1/pipeline_validation.go	96.2%	96.3%	0.1
pkg/reconciler/pipelinerun/pipelinerun.go	83.2%	83.4%	0.3

tekton-robot · 2020-06-17T04:18:15Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
internal/builder/v1beta1/pipeline.go	82.0%	82.2%	0.2
pkg/apis/pipeline/v1beta1/pipeline_types.go	62.1%	67.6%	5.5
pkg/apis/pipeline/v1beta1/pipeline_validation.go	96.2%	96.3%	0.1
pkg/reconciler/pipelinerun/pipelinerun.go	83.2%	83.4%	0.3

jlpettersson · 2020-06-17T07:52:27Z

examples/v1beta1/pipelineruns/pipelinerun-with-pipelinespec-and-taskspec.yaml

@@ -6,6 +6,9 @@ spec:
  pipelineSpec:
    tasks:
      - name: echo-good-morning
+        metadata:


this is its own field now? so it will not necessarily be written "just above" taskSpec. If we could have the Task as a field, optional metadata would work out of the box, as with volumeClaimTemplate where I can declare a PersistentVolumeClaim with or without metadata, https://github.com/tektoncd/pipeline/blob/master/pkg/apis/pipeline/v1beta1/workspace_types.go#L61
I would prefer that.

@jlpettersson I think your proposal would help avoid adding k8s specific types in the PipelineTask, we can introduce a new field TaskTemplate or EmbeddedTask in PipelineTask which can of type *Task:

type PipelineTask struct { TaskTemplate *Task `json:"taskTemplate,omitempty"`

PipelineTask can have one of TaskRef, TaskSpec, or TaskTemplate.

I understand we are looking over to Custom Tasks which would help solve many use cases but this would solve the issue in short term and could be extended to custom tasks if needed. I am looking for feedback on this 🙏

@vdemeester @bobcatfish @jlpettersson thoughts?

I think something like that is the most proper solution, yes.
We haven't used "Template" for declarations in other places, but it may work. Other alternatives are:

Task *Task 'json:"task, omitempty"'

or

TaskDeclaration *Task 'json:"task, omitempty"'

Most correct description about what this is, is probably: inline task declaration

I don't think this is related to CustomTask at all, it will be the exact same problem. I think CustomTask recently has become the most misunderstood thing about Tekton.

jlpettersson · 2020-06-18T20:27:15Z

pkg/apis/pipeline/v1beta1/pipeline_types.go

@@ -93,6 +95,9 @@ type PipelineResult struct {
 // PipelineTask defines a task in a Pipeline, passing inputs from both
 // Params and from the output of previous tasks.
 type PipelineTask struct {
+	// +optional
+	metav1.ObjectMeta `json:"metadata,omitempty"`
+


If we had Task *v1beta1.Task instead of (or in addition to - for compatibility) the current TaskSpec *TaskSpec, then metadata would be included as optional - by design

...as how I understand from the volumeClaimTemplate.

but this does not work for TaskRef if we want a "custom" metadata - but if we want to use the metdata as-is, I think it should work?

thanks @jlpettersson, with taskRef, the referring task can have metadata which should be sufficient like we have today. My focus was to only add metadata with taskSpec as pipeline has no means of specifying task metadata with taskSpec.

bobcatfish · 2020-06-18T21:46:44Z

I think this might require a TEP - since it's an API change, and also this is introducing more k8s specific stuff into a PipelineTask - I'd at least like to discuss it before adding it

bobcatfish · 2020-06-18T21:48:07Z

looks like @vdemeester didnt feel it required a TEP (#2767 (comment)) that's ok with me, but could we at least discuss this at the API working group and talk about why we need it?

pritidesai · 2020-06-18T22:28:49Z

looks like @vdemeester didnt feel it required a TEP (#2767 (comment)) that's ok with me, but could we at least discuss this at the API working group and talk about why we need it?

yup, will add it for the discussion in the API group 🙏

Tomcli · 2020-06-22T17:31:29Z

Here is one of our use cases with this new API:
With the current API, we can have two tasks in our pipeline that use the same label key:

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  labels:
    pipelines.kubeflow.org/pipeline-sdk-type: kfp
  name: echo
spec:
  steps:
  - args:
    - echo hello world
    command:
    - sh
    - -c
    image: library/bash:4.4.23
    name: main
---
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  labels:
    pipelines.kubeflow.org/pipeline-sdk-type: tfx
  name: echo2
spec:
  steps:
  - args:
    - echo hello world
    command:
    - sh
    - -c
    image: library/bash:4.4.23
    name: main
---
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: sequential-pipeline
spec:
  tasks:
  - name: echo
    taskRef:
      name: echo
  - name: echo2
    runAfter:
    - echo
    taskRef:
      name: echo2
---
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  name: sequential-pipeline-run
spec:
  pipelineRef:
    name: sequential-pipeline

Here, we can filter these pods based on a label. e.g. kubectl get pod -l pipelines.kubeflow.org/pipeline-sdk-type=kfp. It is also useful with labelSelector when applying affinity on these tasks.

However, when we try to convert the above pipeline into a single pipelineRun kind, we don't have a way to express the labels or annotations on the task level. Therefore, we want to have the above API to represent the pipelineRun in this way.

apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  name: sequential-pipeline-run
spec:
  pipelineSpec:
    tasks:
    - name: echo
      metadata:
        labels:
          pipelines.kubeflow.org/pipeline-sdk-type: kfp
      taskSpec:
        steps:
        - args:
          - echo hello world
          command:
          - sh
          - -c
          image: library/bash:4.4.23
          name: main
    - name: echo2
      metadata:
        labels:
          pipelines.kubeflow.org/pipeline-sdk-type: tfx
      runAfter:
      - echo
      taskSpec:
        steps:
        - args:
          - echo hello world
          command:
          - sh
          - -c
          image: library/bash:4.4.23
          name: main

pritidesai · 2020-06-22T23:14:31Z

thanks @Tomcli for the example, it looks great, would also like to know:

How are we utilizing these pods after they are filtered based on the pipeline-sdk-type?
Is it possible to use any other type instead of metadata to specify labels and annotations like json?

I understand the expectation is accurate that the labels and annotations should propagate to each taskRun irrespective of how task definition is specified i.e. taskRef or taskSpec. The biggest drawback with this approach is:

PipelineTask level metadata would cause conflicts with the metadata coming from the task itself (while using taskRef). We could restrict it at the pipeline task level which would avoid any conflict but trying to understand the use cases to better design this.

Tomcli · 2020-06-22T23:46:58Z

thanks @Tomcli for the example, it looks great, would also like to know:

How are we utilizing these pods after they are filtered based on the pipeline-sdk-type?

Is it possible to use any other type instead of metadata to specify labels and annotations like json?

I understand the expectation is accurate that the labels and annotations should propagate to each taskRun irrespective of how task definition is specified i.e. taskRef or taskSpec. The biggest drawback with this approach is:

PipelineTask level metadata would cause conflicts with the metadata coming from the task itself (while using taskRef). We could restrict it at the pipeline task level which would avoid any conflict but trying to understand the use cases to better design this.

We have a metadata service that uses this label to filter out pods that need additional actions (such as collecting pod metrics or cleaning up completed pod with certain labels). K8S labels are useful for selecting objects like this and we want to use it in a single pipelineRun spec as well.
I'm not sure is there any other type that can reproduce the same behavior as k8s labels and annotations. If there's any alternative solution I can also give it a try.

I know there will be conflicts when using the taskRef. We only want to override the labels and annotations if the provided keys don't exist. It's similar to the conflicts we have for volumeClaimTemplate and podTemplate.

animeshsingh · 2020-06-23T00:23:22Z

PipelineTask level metadata would cause conflicts with the metadata coming from the task itself (while using taskRef)

This is similar to paradigms where a superset has "default" values, and unless a subset from that superset defines its own values which override the "default", the default gets applied to subset as well.

vdemeester

Nice 👯
One nit, and one question 😉

pkg/apis/pipeline/v1alpha1/pipeline_conversion.go

vdemeester · 2020-07-01T08:49:42Z

pkg/reconciler/pipelinerun/pipelinerun.go

+	// Propagate labels and annotations from PipelineRun and TaskSpec to TaskRun.
+	labels := getTaskrunLabels(pr, rprt.PipelineTask.Name)
+	annotations := getTaskrunAnnotations(pr)
+	if rprt.PipelineTask.TaskSpec != nil {


One question here : which labels should take precedence ? My gut is that the TaskRun labels should take precedence over the one defined in the Pipeline ; otherwise a Pipeline can define a label that override tekton.dev/Task or something and it could break some UX/UI/…. This code does the other way around (labels define in the Pipeline tasks take over the TaskRun labels).

the way its implemented, labels defined in pipeline would take precedence. You are right, it is possible that pipeline defined tekton.dev label would break and takes over the TaskRun labels. I will change the order here.

Should we apply the same precedence order in Conditions? I can create a separate PR for conditions.

Also, I think labels with tekton.dev/ should be reserved for the platform and we should not allow such labels, anything starting with tekton.dev/. What do you think? We could add this kind of validation in this PR as part of ValidateTaskSpecMetadata.

Should we apply the same precedence order in Conditions? I can create a separate PR for conditions.

I would think so indeed 😉

Also, I think labels with tekton.dev/ should be reserved for the platform and we should not allow such labels, anything starting with tekton.dev/. What do you think? We could add this kind of validation in this PR as part of ValidateTaskSpecMetadata.

Indeed, we should probably add that kind of validation to ValidateTaskSpecMetadata (for some reason I thought that was already the case 😜 )

Nope, in general Metadata is validated for valid Name (no special character . and max length of 63 characters). I have added validation on labels as part of taskSpec metadata to restrict using tekton.dev/Task. Not sure if we can block in general any label with tekton.dev in taskSpec metadata (git and docker credentials use tekton.dev but they belong to k8s Secrets) or should we rather restrict labels with tekton.dev/ from every Tekton resource i.e. Pipeline, Task, etc

I have updated the order so that TaskRun labels takes higher precedence.

tekton-robot · 2020-07-01T16:34:19Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
internal/builder/v1beta1/pipeline.go	82.1%	82.8%	0.6
pkg/apis/pipeline/v1beta1/pipeline_types.go	62.1%	64.7%	2.6
pkg/apis/pipeline/v1beta1/pipeline_validation.go	96.2%	96.3%	0.0
pkg/reconciler/pipelinerun/pipelinerun.go	85.9%	86.2%	0.3

tekton-robot · 2020-07-01T20:38:27Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
internal/builder/v1beta1/pipeline.go	82.1%	82.8%	0.6
pkg/apis/pipeline/v1beta1/pipeline_types.go	62.1%	70.0%	7.9
pkg/apis/pipeline/v1beta1/pipeline_validation.go	96.2%	96.3%	0.0
pkg/reconciler/pipelinerun/pipelinerun.go	86.0%	86.6%	0.7

tekton-robot · 2020-07-01T20:50:16Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
internal/builder/v1beta1/pipeline.go	82.1%	82.8%	0.6
pkg/apis/pipeline/v1beta1/pipeline_types.go	62.1%	70.0%	7.9
pkg/apis/pipeline/v1beta1/pipeline_validation.go	96.2%	96.3%	0.0
pkg/reconciler/pipelinerun/pipelinerun.go	86.0%	86.7%	0.7

tekton-robot · 2020-08-07T17:21:26Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
internal/builder/v1beta1/pipeline.go	81.2%	81.8%	0.6
pkg/apis/pipeline/v1beta1/pipeline_types.go	62.1%	60.0%	-2.1
pkg/reconciler/pipelinerun/pipelinerun.go	85.6%	86.3%	0.7

pritidesai · 2020-08-07T17:46:49Z

@vdemeester @imjasonh @bobcatfish @jlpettersson its ready for review, PTAL 🙏

pkg/apis/pipeline/v1beta1/pipeline_types.go

Adding metadata to TaskSpec in PipelineTask to allow specifying metadata. This metadata will be propogated to taskRun and then to the pods. ``` apiVersion: tekton.dev/v1beta1 kind: PipelineRun metadata: name: pipelinerun-with-taskspec-to-echo-greetings spec: pipelineSpec: tasks: - name: echo-greetings taskSpec: metadata: labels: [ …] steps: ... ``` Metadata is already supported as part of Tasks and Pipelines while respective CRDs are created. But was not possible to specify with embedded resources.

tekton-robot · 2020-08-07T18:31:49Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
internal/builder/v1beta1/pipeline.go	81.2%	81.8%	0.6
pkg/apis/pipeline/v1beta1/pipeline_types.go	62.1%	60.0%	-2.1
pkg/reconciler/pipelinerun/pipelinerun.go	85.6%	86.3%	0.7

imjasonh · 2020-08-08T01:04:28Z

/lgtm

🎉

vdemeester

/lgtm
/meow

tekton-robot · 2020-08-11T10:12:02Z

@vdemeester:

In response to this:

/lgtm
/meow

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

tekton-robot · 2020-08-11T10:12:08Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: vdemeester

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [vdemeester]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

pritidesai · 2020-08-18T23:39:24Z

/test check-pr-has-kind-label

pritidesai · 2020-08-19T01:53:57Z

/retest

vdemeester · 2020-08-19T08:21:20Z

/test pull-tekton-pipeline-integration-tests

vdemeester · 2020-08-19T08:37:52Z

2020/08/19 08:26:57 main.go:319: Something went wrong: failed to prepare test environment: --provider=gke boskos failed to acquire project: resources not found

Hum, boskos is in trouble 😅

vdemeester · 2020-08-19T08:38:03Z

/test pull-tekton-pipeline-integration-tests

pritidesai · 2020-08-19T16:33:28Z

one more try 🙃

/test pull-tekton-pipeline-integration-tests

vdemeester · 2020-08-20T08:54:44Z

/test pull-tekton-pipeline-integration-tests

tekton-robot requested review from a user and vdemeester June 17, 2020 02:10

tekton-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jun 17, 2020

tekton-robot added the kind/feature Categorizes issue or PR as related to a new feature. label Jun 17, 2020

pritidesai force-pushed the pt-metadata branch from aff0b65 to 564db51 Compare June 17, 2020 04:15

jlpettersson reviewed Jun 17, 2020

View reviewed changes

jlpettersson reviewed Jun 18, 2020

View reviewed changes

bobcatfish mentioned this pull request Jun 22, 2020

TEP 0002: Custom Tasks tektoncd/community#128

Merged

pritidesai force-pushed the pt-metadata branch from 564db51 to b42df6a Compare July 1, 2020 06:44

pritidesai changed the title ~~adding metadata to PipelineTask~~ adding metadata to taskSpec in PipelineTask Jul 1, 2020

vdemeester reviewed Jul 1, 2020

View reviewed changes

tekton-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 1, 2020

pritidesai force-pushed the pt-metadata branch from b42df6a to 28a4c1a Compare July 1, 2020 16:31

pritidesai force-pushed the pt-metadata branch from 28a4c1a to 7d67edd Compare July 1, 2020 20:33

pritidesai force-pushed the pt-metadata branch from 7d67edd to 4697722 Compare July 1, 2020 20:47

imjasonh reviewed Aug 7, 2020

View reviewed changes

pkg/apis/pipeline/v1beta1/pipeline_types.go Outdated Show resolved Hide resolved

pritidesai force-pushed the pt-metadata branch from a5c02ae to 9ed844d Compare August 7, 2020 18:28

tekton-robot assigned imjasonh Aug 8, 2020

tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Aug 8, 2020

vdemeester added kind/feature Categorizes issue or PR as related to a new feature. and removed kind/feature Categorizes issue or PR as related to a new feature. labels Aug 11, 2020

vdemeester approved these changes Aug 11, 2020

View reviewed changes

tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 11, 2020

vdemeester added kind/feature Categorizes issue or PR as related to a new feature. and removed kind/feature Categorizes issue or PR as related to a new feature. labels Aug 19, 2020

vdemeester added kind/feature Categorizes issue or PR as related to a new feature. and removed kind/feature Categorizes issue or PR as related to a new feature. labels Aug 20, 2020

tekton-robot merged commit 74cc03a into tektoncd:master Aug 20, 2020

AlanGreene mentioned this pull request Feb 23, 2021

TEP-0047: Pipeline Task Display Name tektoncd/community#319

Merged

pritidesai mentioned this pull request Nov 9, 2021

restricting tekton.dev labels in metadata #4366

Open

lbernick mentioned this pull request Apr 6, 2022

Owning our *Container* spec #4737

Closed

adding metadata to taskSpec in PipelineTask #2826

adding metadata to taskSpec in PipelineTask #2826

Conversation

pritidesai commented Jun 17, 2020 • edited

Changes

Submitter Checklist

Reviewer Notes

Release Notes

tekton-robot commented Jun 17, 2020

tekton-robot commented Jun 17, 2020

pritidesai commented Jun 17, 2020

tekton-robot commented Jun 17, 2020

tekton-robot commented Jun 17, 2020

jlpettersson Jun 17, 2020 • edited

Choose a reason for hiding this comment

pritidesai Jun 25, 2020 • edited

Choose a reason for hiding this comment

jlpettersson Jun 25, 2020 • edited

Choose a reason for hiding this comment

jlpettersson Jun 18, 2020

Choose a reason for hiding this comment

jlpettersson Jun 18, 2020

Choose a reason for hiding this comment

pritidesai Jun 23, 2020 • edited

Choose a reason for hiding this comment

bobcatfish commented Jun 18, 2020

bobcatfish commented Jun 18, 2020

pritidesai commented Jun 18, 2020 • edited

Tomcli commented Jun 22, 2020

pritidesai commented Jun 22, 2020 • edited

Tomcli commented Jun 22, 2020 • edited

animeshsingh commented Jun 23, 2020

vdemeester left a comment

Choose a reason for hiding this comment

vdemeester Jul 1, 2020

Choose a reason for hiding this comment

pritidesai Jul 1, 2020

Choose a reason for hiding this comment

vdemeester Jul 1, 2020

Choose a reason for hiding this comment

pritidesai Jul 2, 2020 • edited

Choose a reason for hiding this comment

pritidesai Jul 2, 2020

Choose a reason for hiding this comment

tekton-robot commented Jul 1, 2020

tekton-robot commented Jul 1, 2020

tekton-robot commented Jul 1, 2020

tekton-robot commented Aug 7, 2020

pritidesai commented Aug 7, 2020

tekton-robot commented Aug 7, 2020

imjasonh commented Aug 8, 2020

vdemeester left a comment

Choose a reason for hiding this comment

tekton-robot commented Aug 11, 2020

tekton-robot commented Aug 11, 2020

pritidesai commented Aug 18, 2020

pritidesai commented Aug 19, 2020

vdemeester commented Aug 19, 2020

vdemeester commented Aug 19, 2020

vdemeester commented Aug 19, 2020

pritidesai commented Aug 19, 2020

vdemeester commented Aug 20, 2020

pritidesai commented Jun 17, 2020 •

edited

jlpettersson Jun 17, 2020 •

edited

pritidesai Jun 25, 2020 •

edited

jlpettersson Jun 25, 2020 •

edited

pritidesai Jun 23, 2020 •

edited

pritidesai commented Jun 18, 2020 •

edited

pritidesai commented Jun 22, 2020 •

edited

Tomcli commented Jun 22, 2020 •

edited

pritidesai Jul 2, 2020 •

edited