WIP - implementing finally at the pipeline level 🤞🏼💥😱 #2437

pritidesai · 2020-04-18T07:31:36Z

Changes

We can now specify a list of tasks that needs to be executed just before
pipeline exits (either after finishing all non-final tasks successfully or after
a single failure)

Most common final tasks could be (1) report test results, (2) cleanup cluster resources, etc

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: pipeline-with-final-tasks
spec:
  tasks:
    - name: pre-work
      taskRef:
        Name: some-pre-work
    - name: unit-test
      taskRef:
        Name: run-unit-test
      runAfter:
        - pre-work
    - name: integration-test
      taskRef:
        Name: run-integration-test
      runAfter:
        - unit-test
  finally:
    - name: cleanup-test
      taskRef:
        Name: cleanup-cluster
    - name: report-results
      taskRef:
        Name: report-test-results

Design doc: https://docs.google.com/document/d/1lxpYQHppiWOxsn4arqbwAFDo4T0-LCqpNa6p-TJdHrw/edit#

Part of #1684
Fixes #2446

Depends on: #2504

Submitter Checklist

These are the criteria that every PR should meet, please check them off as you
review them:

Includes tests (if functionality changed/added)
Includes docs (if user facing)
Commit messages follow commit message best practices

See the contribution guide for more details.

Double check this list of stuff that's easy to miss:

If you are adding a new binary/image to the cmd dir, please update
the release Task to build and release this image.

Reviewer Notes

If API changes are included, additive changes must be approved by at least two OWNERS and backwards incompatible changes must be approved by more than 50% of the OWNERS, and they must first be added in a backwards compatible way.

Release Notes

Users can now specify Tasks within a Pipeline that will always execute, even if Tasks fail, via the new `finally` clause

tekton-robot · 2020-04-18T07:31:40Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign vdemeester
You can assign the PR to them by writing /assign @vdemeester in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

tekton-robot · 2020-04-18T07:36:37Z

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1alpha1/pipeline_conversion.go	96.3%	64.3%	-32.0
pkg/apis/pipeline/v1alpha1/pipeline_validation.go	96.9%	90.1%	-6.8
test/builder/pipeline.go	83.0%	77.6%	-5.4

bobcatfish · 2020-04-20T14:43:03Z

Hey @pritidesai ! Very excited to see this moving along!! :D :D

I'm going to update the PR description to include the release notes, design doc and related issue.

Is it possible to include the docs on how to use this in the PR? This makes reviewing the PR waaaay easier, otherwise one has to read the code to understand how this is intended to work

pkg/apis/pipeline/v1beta1/pipeline_types.go

linux-foundation-easycla · 2020-04-21T01:34:56Z

The committers are authorized under a signed CLA.

✅ Priti Desai (0795040)

tekton-robot · 2020-04-21T21:36:16Z

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1alpha1/pipeline_conversion.go	96.3%	93.8%	-2.5
pkg/apis/pipeline/v1alpha1/pipeline_validation.go	96.9%	90.0%	-6.9
pkg/apis/pipeline/v1beta1/pipeline_defaults.go	33.3%	28.6%	-4.8
pkg/apis/pipeline/v1beta1/pipeline_validation.go	92.2%	89.8%	-2.4

tekton-robot · 2020-04-21T23:15:17Z

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1alpha1/pipeline_conversion.go	96.3%	93.8%	-2.5
pkg/apis/pipeline/v1alpha1/pipeline_validation.go	96.9%	90.0%	-6.9
pkg/apis/pipeline/v1beta1/pipeline_defaults.go	33.3%	28.6%	-4.8
pkg/apis/pipeline/v1beta1/pipeline_validation.go	92.2%	89.8%	-2.4

tekton-robot · 2020-04-22T00:45:48Z

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1alpha1/pipeline_conversion.go	96.3%	93.8%	-2.5
pkg/apis/pipeline/v1alpha1/pipeline_validation.go	96.9%	90.0%	-6.9
pkg/apis/pipeline/v1beta1/pipeline_defaults.go	33.3%	28.6%	-4.8
pkg/apis/pipeline/v1beta1/pipeline_validation.go	92.2%	89.8%	-2.4

tekton-robot · 2020-04-23T01:43:52Z

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1alpha1/pipeline_conversion.go	96.4%	93.9%	-2.5
pkg/apis/pipeline/v1alpha1/pipeline_validation.go	96.9%	90.0%	-6.9
pkg/apis/pipeline/v1beta1/pipeline_defaults.go	33.3%	28.6%	-4.8
pkg/apis/pipeline/v1beta1/pipeline_validation.go	92.2%	89.8%	-2.4

tekton-robot · 2020-04-29T04:29:07Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1alpha1/pipeline_conversion.go	96.4%	95.5%	-1.0
pkg/apis/pipeline/v1alpha1/pipeline_validation.go	96.9%	95.4%	-1.5
pkg/apis/pipeline/v1beta1/pipeline_defaults.go	33.3%	28.6%	-4.8
pkg/apis/pipeline/v1beta1/pipeline_validation.go	91.7%	91.0%	-0.6
pkg/reconciler/pipelinerun/pipelinerun.go	72.4%	65.2%	-7.2
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	90.2%	89.5%	-0.8
pkg/reconciler/pipelinerun/resources/resultrefresolution.go	75.6%	75.9%	0.3
test/builder/pipeline.go	83.6%	80.4%	-3.2

pritidesai · 2020-04-29T04:43:44Z

/test pull-tekton-pipeline-build-tests

pritidesai · 2020-04-29T04:45:30Z

this is ready for review now, thanks 🙏

imjasonh · 2020-04-29T12:11:30Z

/hold

There's still some ongoing discussion in the design doc that I'd like to see resolved before we merge this.

vdemeester

Few comments 👼

docs/pipelines.md

tekton-robot · 2020-05-01T06:39:42Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1alpha1/pipeline_conversion.go	96.4%	95.5%	-1.0
pkg/apis/pipeline/v1alpha1/pipeline_validation.go	96.9%	95.4%	-1.5
pkg/apis/pipeline/v1beta1/pipeline_defaults.go	33.3%	28.6%	-4.8
pkg/apis/pipeline/v1beta1/pipeline_validation.go	91.7%	91.0%	-0.6
pkg/reconciler/pipelinerun/pipelinerun.go	72.4%	65.2%	-7.2
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	90.2%	89.6%	-0.6
pkg/reconciler/pipelinerun/resources/resultrefresolution.go	75.6%	75.9%	0.3
test/builder/pipeline.go	83.6%	80.4%	-3.2

We can now specify a list of tasks needs to be executed just before pipeline exits (either after finishing all non-final tasks successfully or after a single failure) Most useful for tasks such as report test results, cleanup cluster resources, etc ``` apiVersion: tekton.dev/v1beta1 kind: Pipeline metadata: name: pipeline-with-final-tasks spec: tasks: - name: pre-work taskRef: Name: some-pre-work - name: unit-test taskRef: Name: run-unit-test runAfter: - pre-work - name: integration-test taskRef: Name: run-integration-test runAfter: - unit-test finally: - name: cleanup-test taskRef: Name: cleanup-cluster - name: report-results taskRef: Name: report-test-results ```

tekton-robot · 2020-05-01T07:03:05Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1alpha1/pipeline_conversion.go	96.4%	95.5%	-1.0
pkg/apis/pipeline/v1alpha1/pipeline_validation.go	96.9%	95.4%	-1.5
pkg/apis/pipeline/v1beta1/pipeline_defaults.go	33.3%	28.6%	-4.8
pkg/apis/pipeline/v1beta1/pipeline_validation.go	91.7%	91.0%	-0.6
pkg/reconciler/pipelinerun/pipelinerun.go	72.4%	65.2%	-7.2
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	90.2%	89.6%	-0.7
pkg/reconciler/pipelinerun/resources/resultrefresolution.go	75.6%	75.9%	0.3
test/builder/pipeline.go	83.6%	80.4%	-3.2

pritidesai · 2020-05-01T09:53:12Z

Few comments 👼

thanks @vdemeester I have addressed all of your comments, let me know if any more changes needed 🙏

docs/pipelines.md

bobcatfish

This is looking great! I think we're all almost totally agreed about the feature itself so most of my comments are about the implementation.

Main themes:

I'm not sure about allowing "from" in finally tasks, feels a bit confusing to me when we don't allow results (e.g. imagine a git resource that is "from" a task that builds - if the build task executes, the data on disc for the resource may be different than if it doesnt)
I think its worth trying to avoid duplicating the validation logic as much as possible, since it leads to some nasty bugs, for example we ran into that with results (Fix 3 bugs with results #2471) - we're still duplicating some logic there so it might not be completely avoidable, but if we can avoid it its worth the effort
If we get the structure of PipelineRunState right, i bet we can avoid some of the duplicated code in the reconciler
Better to add functions in separate packages than add more functions to the reconciler
Is it possible for the logic that determines what runs next to become aware of finally? If the logic that returns the next tasks to run can handle Finally tasks, i bet we need a lot less special casing for finally

It looks like the coverage generally went down, and we want it to generally go up or stay the same so it might be worth digging into that a bit once we're more sure about the details of the implementation:

docs/pipelines.md

bobcatfish · 2020-05-05T17:54:46Z

docs/pipelines.md

+| one or more `PipelineTask` skipped and rest successful | all final tasks successful | `true` | `Completed` |
+| one or more `PipelineTask` skipped and rest successful | one or more failure of final tasks | `false` | `Failed` |
+| single failure of `PipelineTask` | all final tasks successful | `false` | `failed` |
+| single failure of `PipelineTask` | one or more failure of final tasks | `false` | `failed` |


nice! 🎉 thanks for making this explicit :D

examples/v1beta1/pipelineruns/pipelinerun-with-final-tasks.yaml

pkg/reconciler/pipelinerun/pipelinerun.go

bobcatfish · 2020-05-05T18:43:11Z

I was just also thinking about ways to make incremental progress on this - I wonder if it would work to have a PR to add the types and their validation before adding the functionality.

pritidesai · 2020-05-20T07:15:14Z

I have split this PR into two separate wip PRs:

(1) adding finally type and its validation - PR #2650
(2) implementing this new feature - PR #2661

will close this PR once all the comments from this PR are addressed in new PRs.

ghost · 2020-05-21T12:52:14Z

/kind feature

pritidesai · 2020-06-04T06:27:26Z

I have addressed all the comments from this PR. Please review the two (new) separate PRs:

Closing this PR 👋

tekton-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 18, 2020

googlebot added the cla: yes Trying to make the CLA bot happy with ppl from different companies work on one commit label Apr 18, 2020

tekton-robot requested review from afrittoli and imjasonh April 18, 2020 07:31

tekton-robot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Apr 18, 2020

bobcatfish reviewed Apr 20, 2020

View reviewed changes

pkg/apis/pipeline/v1beta1/pipeline_types.go Outdated Show resolved Hide resolved

chhsia0 mentioned this pull request Apr 20, 2020

Add a field to Step that allows it to ignore failed prior Steps *within the same Task* #1559

Closed

pritidesai force-pushed the finally-4-16 branch from 9279df2 to 0795040 Compare April 21, 2020 01:34

tekton-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Apr 21, 2020

pritidesai force-pushed the finally-4-16 branch from 0795040 to 326bb89 Compare April 21, 2020 06:24

tekton-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Apr 21, 2020

pritidesai force-pushed the finally-4-16 branch 5 times, most recently from 06d4d8c to d6602c6 Compare April 21, 2020 21:33

pritidesai force-pushed the finally-4-16 branch from d6602c6 to 1f7213d Compare April 21, 2020 23:13

pritidesai force-pushed the finally-4-16 branch from 1f7213d to 5e2de42 Compare April 22, 2020 00:43

pritidesai force-pushed the finally-4-16 branch from 5e2de42 to 7aa21eb Compare April 23, 2020 01:41

pritidesai force-pushed the finally-4-16 branch from 7aa21eb to 7e9a69e Compare April 25, 2020 07:34

pritidesai changed the title ~~wip - implementing finally at the pipeline level 🤞🏼~~ implementing finally at the pipeline level 🤞🏼 Apr 29, 2020

tekton-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 29, 2020

tekton-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 29, 2020

vdemeester requested changes Apr 29, 2020

View reviewed changes

docs/pipelines.md Show resolved Hide resolved

docs/pipelines.md Outdated Show resolved Hide resolved

docs/pipelines.md Outdated Show resolved Hide resolved

docs/pipelines.md Outdated Show resolved Hide resolved

docs/pipelines.md Outdated Show resolved Hide resolved

pritidesai force-pushed the finally-4-16 branch from b29538c to fa968dc Compare May 1, 2020 06:36

pritidesai changed the title ~~implementing finally at the pipeline level 🤞🏼~~ implementing finally at the pipeline level 🤞🏼💥😱 May 1, 2020

pritidesai force-pushed the finally-4-16 branch from fa968dc to 6347772 Compare May 1, 2020 06:59

afrittoli reviewed May 1, 2020

View reviewed changes

docs/pipelines.md Show resolved Hide resolved

afrittoli reviewed May 1, 2020

View reviewed changes

docs/pipelines.md Show resolved Hide resolved

bobcatfish reviewed May 5, 2020

View reviewed changes

pritidesai mentioned this pull request May 6, 2020

Design and implement task results in final tasks #2557

Closed

fenglixa mentioned this pull request May 19, 2020

Support Exit handler kubeflow/kfp-tekton#85

Closed

tekton-robot added the kind/feature Categorizes issue or PR as related to a new feature. label May 21, 2020

pritidesai changed the title ~~implementing finally at the pipeline level 🤞🏼💥😱~~ hold - implementing finally at the pipeline level 🤞🏼💥😱 Jun 1, 2020

pritidesai changed the title ~~hold - implementing finally at the pipeline level 🤞🏼💥😱~~ WIP - implementing finally at the pipeline level 🤞🏼💥😱 Jun 1, 2020

tekton-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 1, 2020

pritidesai closed this Jun 4, 2020

pritidesai mentioned this pull request Jun 13, 2020

implementing pipeline level finally #2661

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP - implementing finally at the pipeline level 🤞🏼💥😱 #2437

WIP - implementing finally at the pipeline level 🤞🏼💥😱 #2437

pritidesai commented Apr 18, 2020 •

edited

Loading

tekton-robot commented Apr 18, 2020

tekton-robot commented Apr 18, 2020

bobcatfish commented Apr 20, 2020

linux-foundation-easycla bot commented Apr 21, 2020 •

edited

Loading

tekton-robot commented Apr 21, 2020

tekton-robot commented Apr 21, 2020

tekton-robot commented Apr 22, 2020

tekton-robot commented Apr 23, 2020

tekton-robot commented Apr 29, 2020

pritidesai commented Apr 29, 2020

pritidesai commented Apr 29, 2020

imjasonh commented Apr 29, 2020

vdemeester left a comment

tekton-robot commented May 1, 2020

tekton-robot commented May 1, 2020

pritidesai commented May 1, 2020

bobcatfish left a comment

bobcatfish May 5, 2020

bobcatfish commented May 5, 2020

pritidesai commented May 20, 2020 •

edited

Loading

ghost commented May 21, 2020

pritidesai commented Jun 4, 2020

WIP - implementing finally at the pipeline level 🤞🏼💥😱 #2437

WIP - implementing finally at the pipeline level 🤞🏼💥😱 #2437

Conversation

pritidesai commented Apr 18, 2020 • edited Loading

Changes

Submitter Checklist

Reviewer Notes

Release Notes

tekton-robot commented Apr 18, 2020

tekton-robot commented Apr 18, 2020

bobcatfish commented Apr 20, 2020

linux-foundation-easycla bot commented Apr 21, 2020 • edited Loading

tekton-robot commented Apr 21, 2020

tekton-robot commented Apr 21, 2020

tekton-robot commented Apr 22, 2020

tekton-robot commented Apr 23, 2020

tekton-robot commented Apr 29, 2020

pritidesai commented Apr 29, 2020

pritidesai commented Apr 29, 2020

imjasonh commented Apr 29, 2020

vdemeester left a comment

Choose a reason for hiding this comment

tekton-robot commented May 1, 2020

tekton-robot commented May 1, 2020

pritidesai commented May 1, 2020

bobcatfish left a comment

Choose a reason for hiding this comment

bobcatfish May 5, 2020

Choose a reason for hiding this comment

bobcatfish commented May 5, 2020

pritidesai commented May 20, 2020 • edited Loading

ghost commented May 21, 2020

pritidesai commented Jun 4, 2020

pritidesai commented Apr 18, 2020 •

edited

Loading

linux-foundation-easycla bot commented Apr 21, 2020 •

edited

Loading

pritidesai commented May 20, 2020 •

edited

Loading