Add a Timeouts optional field to pipelinerun #3843

souleb · 2021-03-18T17:06:26Z

This PR is an implementation of TEP #326.

Fixes issue #2989.

Changes

Timeouts is a dict of timeout fields:

timeouts:
  pipeline: "0h0m60s"
  tasks: "0h0m30s"
  finally: "0h0m20s"

All three subfields are optional

Validates timeouts if:
- timeouts.pipeline >= timeouts.tasks + timeouts.finally
- all fields much be strictly positive

It is still possible to use timeout, but it is invalid to have both timeout and timeouts. The timeout behavior is unchanged.

⚠️ tkn does not process this new field

Have been tested with:

apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  name: hello-world-pipeline-run-with-timeout
spec:
  timeout: "0h0m60s"
  pipelineSpec:
    tasks:
    - name: dagtask
      timeout: "0h0m30s"
      taskSpec:
        steps:
          - name: hello
            image: ubuntu
            script: |
              echo "Hello World!"
              sleep 10
    finally:
    - name: finallytask
      params:
        - name: echoStatus
          value: "$(tasks.dagtask.status)"
      taskSpec:
        params:
          - name: echoStatus
        steps: 
          - name: verify-status
            image: ubuntu
            script: |
              if [ $(params.echoStatus) == "Succeeded" ]
              then
                echo " Hello World echoed successfully"
              fi

---
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  name: hello-world-pipeline-run-with-taskstimeout
spec:
  timeouts: 
    tasks: "0h0m30s"
  pipelineSpec:
    tasks:
    - name: dagtask
      taskSpec:
        steps:
          - name: hello
            image: ubuntu
            script: |
              echo "Hello World!"
              sleep 30
    finally:
    - name: finallytask
      params:
        - name: echoStatus
          value: "$(tasks.dagtask.status)"
      taskSpec:
        params:
          - name: echoStatus
        steps:
          - name: verify-status
            image: ubuntu
            script: |
              if [ $(params.echoStatus) == "Succeeded" ]
              then
                echo " Hello World echoed successfully"
              fi


---
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  name: hello-world-pipeline-run-with-timeout-and-taskstimeout
spec:
  timeouts: 
    pipeline: "0h0m40s"
    tasks: "0h0m30s"
  pipelineSpec:
    tasks:
    - name: dagtask
      taskSpec:
        steps:
          - name: hello
            image: ubuntu
            script: |
              echo "Hello World!"
              sleep 30
    finally:
    - name: finallytask
      params:
        - name: echoStatus
          value: "$(tasks.dagtask.status)"
      taskSpec:
        params:
          - name: echoStatus
        steps:
          - name: verify-status
            image: ubuntu
            script: |
              if [ $(params.echoStatus) == "Succeeded" ]
              then
                echo " Hello World echoed successfully"
              fi

---
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  name: pipeline-run-with-timeout-and-taskstimeout-finally-timeout
spec:
  timeouts: 
    pipeline: "0h0m60s"
    tasks: "0h0m30s"
    finally: "0h0m4s"
  pipelineSpec:
    tasks:
    - name: dagtask
      taskSpec:
        steps:
          - name: hello
            image: ubuntu
            script: |
              echo "Hello World!"
              sleep 30
    finally:
    - name: finallytask
      params:
        - name: echoStatus
          value: "$(tasks.dagtask.status)"
      taskSpec:
        params:
          - name: echoStatus
        steps:
          - name: verify-status
            image: ubuntu
            script: |
              if [ $(params.echoStatus) == "Succeeded" ]
              then
                echo " Hello World echoed successfully"
              fi


---
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  name: hello-world-pipeline-run-with-timeout-and-finallytimeout
spec:
  timeouts: 
    pipeline: "0h0m40s"
    finally: "0h0m30s"
  pipelineSpec:
    tasks:
    - name: dagtask
      taskSpec:
        steps:
          - name: hello
            image: ubuntu
            script: |
              echo "Hello World!"
              sleep 30
    finally:
    - name: finallytask
      params:
        - name: echoStatus
          value: "$(tasks.dagtask.status)"
      taskSpec:
        params:
          - name: echoStatus
        steps:
          - name: verify-status
            image: ubuntu
            script: |
              if [ $(params.echoStatus) == "Succeeded" ]
              then
                echo " Hello World echoed successfully"
              fi

/kind feature

Submitter Checklist

These are the criteria that every PR should meet, please check them off as you
review them:

Includes tests (if functionality changed/added)
Includes docs (if user facing)
Commit messages follow commit message best practices
Release notes block has been filled in or deleted (only if no user facing changes)

See the contribution guide for more details.

Double check this list of stuff that's easy to miss:

If you are adding a new binary/image to the cmd dir, please update
the release Task to build and release this image.

Reviewer Notes

If API changes are included, additive changes must be approved by at least two OWNERS and backwards incompatible changes must be approved by more than 50% of the OWNERS, and they must first be added in a backwards compatible way.

Release Notes

API changes
- Added field Timeouts to PipelineRun spec. It is a dict with the following sub-fields
  - pipeline, to control the pipeline failure timeout
  - tasks, to control the pipeline tasks failure timeout
  - finally, , to control the pipeline finally tasks failure timeout
Changes in behavior
- When supplied, a timeouts field combination permits deciding which part of the pipeline runtime is allocated to tasks and finally tasks.

tekton-robot · 2021-03-18T17:06:51Z

Hi @souleb. Thanks for your PR.

I'm waiting for a tektoncd member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

souleb · 2021-03-18T17:07:05Z

cc @jerop @pritidesai

pritidesai · 2021-03-18T18:13:12Z

/ok-to-test

tekton-robot · 2021-03-18T18:15:16Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
internal/builder/v1beta1/pipeline.go	85.6%	85.8%	0.2
pkg/apis/pipeline/v1beta1/pipelinerun_validation.go	100.0%	97.6%	-2.4
pkg/reconciler/pipelinerun/pipelinerun.go	83.8%	84.3%	0.5
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	92.1%	92.2%	0.0

ghost

Thanks for this @souleb ! Looking good so far - I did a partial review and will grab some more time to look again tomorrow.

pkg/apis/pipeline/v1beta1/pipelinerun_types.go

pkg/apis/pipeline/v1beta1/pipelinerun_validation.go

ghost · 2021-03-18T18:19:03Z

pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go

@@ -219,6 +219,11 @@ func (t *ResolvedPipelineRunTask) parentTasksSkip(facts *PipelineRunFacts) bool
 	return false
 }

+// IsFinalTask returns true if a task is a finally task
+func (t *ResolvedPipelineRunTask) IsFinalTask(facts *PipelineRunFacts) bool {


We could make facts.isFinalTask public instead of wrapping it in a new function on the RPRT type. wdyt?

(I'm assuming this isn't prevented for some other reason reason I'm missing?)

It just seemed to me more natural to put it that way because I was expecting to find the function closed to IsFinallySkipped. I like the idea of collocating those receiver function as they seem related. But i'm not against changing the scope of facts.isFinalTask.

Nope, there is no specific reason preventing it from going public i.e facts.isFinalTask can be made public.

I like the idea of collocating those receiver function as they seem related.

I see this a bit differently - IsFinallySkipped is answering a question about the whole pipeline graph which requires the graph be fully "resolved", so makes sense on the ResolvedPipelineRunTasks type. IsFinalTask answers a question about a "fact" of an individual task of the pipeline, so makes sense to me to live on the PipelineRunFacts type. (Edit to add: now that I read this again I'm actually not sure there's much distinction here after all :D)

Given that these two types (RPRT and PipelineRunFacts) are used so closely together in our code I don't feel super strongly either way. I just got a bit confused when I first read it because the two funcs are named so similarly and use the same objects 😅

pkg/reconciler/pipelinerun/pipelinerun_test.go

tekton-robot · 2021-03-18T21:28:23Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
internal/builder/v1beta1/pipeline.go	85.6%	85.8%	0.2
pkg/apis/pipeline/v1beta1/pipelinerun_validation.go	100.0%	97.6%	-2.4
pkg/reconciler/pipelinerun/pipelinerun.go	83.8%	84.3%	0.5
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	92.1%	92.2%	0.0

ghost · 2021-03-22T12:47:03Z

I'm going to add a hold on this just so to ensure we get the TEP merged before this goes in.

/hold

Edit to add: also to wait until the feature gate work is in place.

ghost · 2021-03-22T12:53:47Z

@souleb it would also be great to add an example YAML as part of this PR which shows the new field's usage too. examples/v1beta1/pipelineruns/no-ci/pipeline-timeout.yaml is another similar example which serves that purpose.

ghost · 2021-03-22T12:57:37Z

One more thing that would be useful (sorry for the message spam!) would be to include some integration tests as well. These are quite a bit of work but it's good to have a fully-constructed pipeline run that tests timeout behaviour as part of our e2e test suite. See test/timeout_test.go for an example that you could base your own test on.

tekton-robot · 2021-03-22T13:31:42Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
internal/builder/v1beta1/pipeline.go	85.6%	85.8%	0.2
pkg/apis/pipeline/v1beta1/pipelinerun_validation.go	100.0%	97.6%	-2.4
pkg/reconciler/pipelinerun/pipelinerun.go	83.8%	84.3%	0.5
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	92.1%	92.2%	0.0

souleb · 2021-03-22T13:37:49Z

One more thing that would be useful (sorry for the message spam!) would be to include some integration tests as well. These are quite a bit of work but it's good to have a fully-constructed pipeline run that tests timeout behaviour as part of our e2e test suite. See test/timeout_test.go for an example that you could base your own test on.

Sure, will do that. Just I wanted to wait for the TEP to be approved beforehand.

tekton-robot · 2021-04-07T21:42:57Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
internal/builder/v1beta1/pipeline.go	85.6%	85.8%	0.2
pkg/apis/pipeline/v1beta1/pipelinerun_validation.go	100.0%	97.6%	-2.4
pkg/reconciler/pipelinerun/pipelinerun.go	83.8%	84.3%	0.5
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	92.1%	92.2%	0.0

tekton-robot · 2021-04-22T23:57:04Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/pipelinerun_defaults.go	100.0%	92.9%	-7.1
pkg/apis/pipeline/v1beta1/pipelinerun_types.go	75.8%	73.5%	-2.2
pkg/apis/pipeline/v1beta1/pipelinerun_validation.go	100.0%	98.2%	-1.8
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	92.1%	92.2%	0.0

tekton-robot · 2021-05-27T08:12:55Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
internal/builder/v1beta1/pipeline.go	85.6%	86.5%	0.9
pkg/apis/pipeline/v1beta1/pipelinerun_defaults.go	100.0%	92.9%	-7.1
pkg/apis/pipeline/v1beta1/pipelinerun_types.go	75.8%	73.5%	-2.2
pkg/apis/pipeline/v1beta1/pipelinerun_validation.go	100.0%	98.2%	-1.8
pkg/reconciler/pipelinerun/pipelinerun.go	83.8%	82.9%	-0.9
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	92.2%	92.3%	0.0

souleb · 2021-05-27T09:46:31Z

@pritidesai will you have some time to review this PR?

pritidesai · 2021-05-27T17:19:24Z

@pritidesai will you have some time to review this PR?

Thank you @souleb for all the changes and your patience 🙇‍♀️ Yes I will spend some time today to review this PR and continue early next week after long weekend. Thanks a bunch again for all your efforts 🙏

vdemeester · 2021-06-01T14:53:11Z

/retest

vdemeester

/lgtm

tekton-robot · 2021-06-01T14:55:49Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sbwsg, vdemeester

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [sbwsg,vdemeester]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

jerop · 2021-06-01T16:39:37Z

/test pull-tekton-pipeline-integration-tests

pritidesai · 2021-06-04T18:14:38Z

@souleb please rebase this PR, its ready to merge. Thanks again for all the hard work. 🙏

TaskTimeout is used to timeout all dag tasks, finally tasks excluded Validates a TasksTimeout if: - TasksTimeout > 0 - Timeout is specified and TasksTimeout <= Timeout - Timeout not specified and TasksTimeout <= default Timeout Add a builder function for taskTimeout Defines 2 functions to get taskrun timeout One for dag tasks and one specifically for finally tasks

i.e. timeouts: pipeline: "0h0m60s" tasks: "0h0m30s" finally: "0h0m20s" All three subfields are optional

tekton-robot · 2021-06-04T21:19:51Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
internal/builder/v1beta1/pipeline.go	85.6%	86.5%	0.9
pkg/apis/pipeline/v1beta1/pipelinerun_defaults.go	100.0%	92.9%	-7.1
pkg/apis/pipeline/v1beta1/pipelinerun_types.go	75.8%	73.5%	-2.2
pkg/apis/pipeline/v1beta1/pipelinerun_validation.go	100.0%	98.3%	-1.7
pkg/reconciler/pipelinerun/pipelinerun.go	83.8%	82.9%	-0.9
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	91.9%	91.9%	0.0

pritidesai · 2021-06-05T04:40:41Z

/retest

souleb · 2021-06-07T14:46:21Z

@vdemeester can reset the lgtm? The bot unset it following my rebase.

vdemeester · 2021-06-07T14:54:33Z

/lgtm

vdemeester · 2021-06-07T15:29:45Z

/retest

vdemeester · 2021-06-07T16:15:53Z

/retest

🤔

tektoncd/pipeline#3843 was merged back in June. Signed-off-by: Andrew Bayer <andrew.bayer@gmail.com>

tekton-robot added the release-note Denotes a PR that will be considered when it comes time to generate release notes. label Mar 18, 2021

tekton-robot requested review from bobcatfish and dlorenc March 18, 2021 17:06

tekton-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Mar 18, 2021

tekton-robot added the kind/feature Categorizes issue or PR as related to a new feature. label Mar 18, 2021

tekton-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Mar 18, 2021

ghost reviewed Mar 18, 2021

View reviewed changes

souleb force-pushed the finally-exec-post-timeout branch from b7f3bf9 to c4a61fe Compare March 18, 2021 21:25

jerop self-assigned this Mar 21, 2021

tekton-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 22, 2021

souleb force-pushed the finally-exec-post-timeout branch from c4a61fe to 9d5768a Compare March 22, 2021 13:29

souleb force-pushed the finally-exec-post-timeout branch from 9d5768a to f903552 Compare April 7, 2021 21:40

souleb changed the title ~~Add a TasksTimeout optional field to pipelinerun~~ Add a Timeouts optional field to pipelinerun Apr 22, 2021

tekton-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Apr 22, 2021

souleb force-pushed the finally-exec-post-timeout branch from a334074 to 11be900 Compare April 23, 2021 00:25

souleb force-pushed the finally-exec-post-timeout branch from 5e894ee to 4fc27b2 Compare May 27, 2021 08:10

vdemeester approved these changes Jun 1, 2021

View reviewed changes

tekton-robot assigned vdemeester Jun 1, 2021

tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Jun 1, 2021

tekton-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 2, 2021

souleb added 3 commits June 4, 2021 23:03

Add a new api field Timeouts, a dict of timeout fields:

8ab988a

i.e. timeouts: pipeline: "0h0m60s" tasks: "0h0m30s" finally: "0h0m20s" All three subfields are optional

Puts the new timeouts field under feature gates

a965afd

souleb force-pushed the finally-exec-post-timeout branch from 4fc27b2 to a965afd Compare June 4, 2021 21:16

tekton-robot removed lgtm Indicates that a PR is ready to be merged. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Jun 4, 2021

tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Jun 7, 2021

tekton-robot merged commit b6cb0a0 into tektoncd:main Jun 7, 2021

abayer added a commit to abayer/community that referenced this pull request Dec 14, 2021

TEP-0046: Mark as implemented

a620b58

tektoncd/pipeline#3843 was merged back in June. Signed-off-by: Andrew Bayer <andrew.bayer@gmail.com>

abayer mentioned this pull request Dec 14, 2021

TEP-0046: Mark as implemented tektoncd/community#577

Merged

tekton-robot pushed a commit to tektoncd/community that referenced this pull request Dec 14, 2021

TEP-0046: Mark as implemented

aec8cb0

tektoncd/pipeline#3843 was merged back in June. Signed-off-by: Andrew Bayer <andrew.bayer@gmail.com>

lbernick pushed a commit to lbernick/community that referenced this pull request Dec 15, 2021

TEP-0046: Mark as implemented

617ff5a

tektoncd/pipeline#3843 was merged back in June. Signed-off-by: Andrew Bayer <andrew.bayer@gmail.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a Timeouts optional field to pipelinerun #3843

Add a Timeouts optional field to pipelinerun #3843

souleb commented Mar 18, 2021 •

edited

tekton-robot commented Mar 18, 2021

souleb commented Mar 18, 2021

pritidesai commented Mar 18, 2021

tekton-robot commented Mar 18, 2021

ghost left a comment

ghost Mar 18, 2021

ghost Mar 18, 2021

souleb Mar 18, 2021

pritidesai Mar 19, 2021

ghost Mar 22, 2021 •

edited by ghost

tekton-robot commented Mar 18, 2021

ghost commented Mar 22, 2021 •

edited by ghost

ghost commented Mar 22, 2021

ghost commented Mar 22, 2021

tekton-robot commented Mar 22, 2021

souleb commented Mar 22, 2021

tekton-robot commented Apr 7, 2021

tekton-robot commented Apr 22, 2021

tekton-robot commented May 27, 2021

souleb commented May 27, 2021

pritidesai commented May 27, 2021

vdemeester commented Jun 1, 2021

vdemeester left a comment

tekton-robot commented Jun 1, 2021

jerop commented Jun 1, 2021

pritidesai commented Jun 4, 2021

tekton-robot commented Jun 4, 2021

pritidesai commented Jun 5, 2021

souleb commented Jun 7, 2021

vdemeester commented Jun 7, 2021

vdemeester commented Jun 7, 2021

vdemeester commented Jun 7, 2021

Add a Timeouts optional field to pipelinerun #3843

Add a Timeouts optional field to pipelinerun #3843

Conversation

souleb commented Mar 18, 2021 • edited

Changes

Submitter Checklist

Reviewer Notes

Release Notes

tekton-robot commented Mar 18, 2021

souleb commented Mar 18, 2021

pritidesai commented Mar 18, 2021

tekton-robot commented Mar 18, 2021

ghost left a comment

Choose a reason for hiding this comment

ghost Mar 18, 2021

Choose a reason for hiding this comment

ghost Mar 18, 2021

Choose a reason for hiding this comment

souleb Mar 18, 2021

Choose a reason for hiding this comment

pritidesai Mar 19, 2021

Choose a reason for hiding this comment

ghost Mar 22, 2021 • edited by ghost

Choose a reason for hiding this comment

tekton-robot commented Mar 18, 2021

ghost commented Mar 22, 2021 • edited by ghost

ghost commented Mar 22, 2021

ghost commented Mar 22, 2021

tekton-robot commented Mar 22, 2021

souleb commented Mar 22, 2021

tekton-robot commented Apr 7, 2021

tekton-robot commented Apr 22, 2021

tekton-robot commented May 27, 2021

souleb commented May 27, 2021

pritidesai commented May 27, 2021

vdemeester commented Jun 1, 2021

vdemeester left a comment

Choose a reason for hiding this comment

tekton-robot commented Jun 1, 2021

jerop commented Jun 1, 2021

pritidesai commented Jun 4, 2021

tekton-robot commented Jun 4, 2021

pritidesai commented Jun 5, 2021

souleb commented Jun 7, 2021

vdemeester commented Jun 7, 2021

vdemeester commented Jun 7, 2021

vdemeester commented Jun 7, 2021

souleb commented Mar 18, 2021 •

edited

ghost Mar 22, 2021 •

edited by ghost

ghost commented Mar 22, 2021 •

edited by ghost