Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement When Expressions #3117

Closed
wants to merge 1 commit into from
Closed

Implement When Expressions #3117

wants to merge 1 commit into from

Conversation

jerop
Copy link
Member

@jerop jerop commented Aug 19, 2020

Adding WhenExpressions used to efficiently specify guarded execution
of Tasks, without spinning up new pods. We use WhenExpressions to
avoid adding an opinionated and complex expression language to the Tekton
API to ensure Tekton can be supported by as many systems as possible.
Further details about the design are in Conditions Beta TEP.

The components of WhenExpressions are Input, Operator
and Values:

  • Input is the input for the Guard checking which can be static
    inputs or variables, such as Parameters or Results.
  • Operator represents an Input's relationship to a set of Values.
    Operators we will use in WhenExpressions are In and NotIn.
  • Values is an array of string values. The Values array must be
    non-empty. It can contain static values or variables (Parameters or
    Results).

The declared WhenExpressions are evaluated before the Task is run.
If all the WhenExpressions evaluate to True, the Task is run.
If any of the WhenExpressions evaluate to False, the Task is
skipped.

Some key parts of this PR are:

  • Expressions package, which can be reused beyond WhenExpressions
  • WhenExpressions type in the PipelineTask
  • Variable substitution for Results and Parameters used in
    WhenExpressions, either in Input or in Values -- can be extended to
    use other variables.
  • Resource dependency mapping when resources (e.g. Results) are
    used from other Tasks
  • Validation of WhenExpressions, variable substitution, among others
  • Evaluation of WhenExpressions and updates to Task execution
    based on the evaluation results
  • WhenExpressionsStatus that exposes the evaluation inputs and results
  • A practical example of using WhenExpression, tests and documentation

Changes

Submitter Checklist

These are the criteria that every PR should meet, please check them off as you
review them:

  • Includes tests (if functionality changed/added)
  • Includes docs (if user facing)
  • Commit messages follow commit message best practices
  • Release notes block has been filled in or deleted (only if no user facing changes)

See the contribution guide for more details.

Double check this list of stuff that's easy to miss:

Reviewer Notes

If API changes are included, additive changes must be approved by at least two OWNERS and backwards incompatible changes must be approved by more than 50% of the OWNERS, and they must first be added in a backwards compatible way.

Release Notes

- To run a `Task` only when certain criteria are met, it is now possible to _guard_ task execution using
the `when` field, which allows you to list a series of references to `WhenExpressions` that contain
an `Input`, an `Operator` and `Values`. The `WhenExpressions` are ANDed.
- Conditions are deprecated, use WhenExpressions instead.

@tekton-robot tekton-robot added the release-note Denotes a PR that will be considered when it comes time to generate release notes. label Aug 19, 2020
@tekton-robot tekton-robot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Aug 19, 2020
@tekton-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign dlorenc
You can assign the PR to them by writing /assign @dlorenc in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@jerop jerop marked this pull request as draft August 19, 2020 17:55
@jerop
Copy link
Member Author

jerop commented Aug 19, 2020

/kind feature
/cc @bobcatfish @jlpettersson

@tekton-robot tekton-robot added kind/feature Categorizes issue or PR as related to a new feature. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. labels Aug 19, 2020
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
internal/builder/v1beta1/pipeline.go 81.2% 81.4% 0.2
pkg/apis/pipeline/v1beta1/pipeline_types.go 62.1% 47.5% -14.6
pkg/apis/pipeline/v1beta1/pipeline_validation.go 96.1% 95.7% -0.4
pkg/expression/expression.go Do not exist 55.6%
pkg/reconciler/pipelinerun/pipelinerun.go 85.6% 84.5% -1.1
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go 90.2% 84.8% -5.4
pkg/reconciler/pipelinerun/resources/resultrefresolution.go 87.8% 89.3% 1.5

@jerop jerop changed the title Implement WhenExpressions Implement When Expressions Aug 19, 2020
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
internal/builder/v1beta1/pipeline.go 81.2% 81.4% 0.2
pkg/apis/pipeline/v1beta1/pipeline_types.go 62.1% 47.5% -14.6
pkg/apis/pipeline/v1beta1/pipeline_validation.go 96.1% 95.7% -0.4
pkg/expression/expression.go Do not exist 55.6%
pkg/reconciler/pipelinerun/pipelinerun.go 85.6% 84.5% -1.1
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go 90.2% 84.8% -5.4
pkg/reconciler/pipelinerun/resources/resultrefresolution.go 87.8% 87.8% -0.0

docs/pipelines.md Outdated Show resolved Hide resolved
@pritidesai
Copy link
Member

This is Very exciting 🎉 I have reviewed doc and an example so far.

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
internal/builder/v1beta1/pipeline.go 81.2% 81.4% 0.2
pkg/apis/pipeline/v1beta1/pipeline_types.go 62.1% 47.5% -14.6
pkg/apis/pipeline/v1beta1/pipeline_validation.go 96.1% 95.7% -0.4
pkg/expression/expression.go Do not exist 77.8%
pkg/reconciler/pipelinerun/pipelinerun.go 85.6% 84.5% -1.1
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go 90.2% 84.8% -5.4
pkg/reconciler/pipelinerun/resources/resultrefresolution.go 87.8% 87.8% -0.0

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
internal/builder/v1beta1/pipeline.go 81.2% 81.4% 0.2
pkg/apis/pipeline/v1beta1/pipeline_types.go 62.1% 47.5% -14.6
pkg/apis/pipeline/v1beta1/pipeline_validation.go 96.1% 95.7% -0.4
pkg/expression/expression.go Do not exist 77.8%
pkg/reconciler/pipelinerun/pipelinerun.go 85.6% 84.5% -1.1
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go 90.2% 84.8% -5.4
pkg/reconciler/pipelinerun/resources/resultrefresolution.go 87.8% 88.5% 0.7

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
internal/builder/v1beta1/pipeline.go 81.2% 81.4% 0.2
pkg/apis/pipeline/v1beta1/pipeline_types.go 62.1% 47.5% -14.6
pkg/apis/pipeline/v1beta1/pipeline_validation.go 96.1% 95.7% -0.4
pkg/expression/expression.go Do not exist 77.8%
pkg/reconciler/pipelinerun/pipelinerun.go 85.6% 84.5% -1.1
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go 90.2% 90.8% 0.6
pkg/reconciler/pipelinerun/resources/resultrefresolution.go 87.8% 87.8% -0.0

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
internal/builder/v1beta1/pipeline.go 81.2% 81.4% 0.2
pkg/apis/pipeline/v1beta1/pipeline_types.go 62.1% 47.5% -14.6
pkg/apis/pipeline/v1beta1/pipeline_validation.go 96.1% 95.7% -0.4
pkg/expression/expression.go Do not exist 77.8%
pkg/reconciler/pipelinerun/pipelinerun.go 85.6% 85.8% 0.2
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go 90.2% 90.8% 0.6
pkg/reconciler/pipelinerun/resources/resultrefresolution.go 87.8% 89.3% 1.5

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
internal/builder/v1beta1/pipeline.go 81.2% 81.4% 0.2
pkg/apis/pipeline/v1beta1/pipeline_types.go 62.1% 57.5% -4.6
pkg/apis/pipeline/v1beta1/pipeline_validation.go 96.1% 96.1% 0.0
pkg/expression/expression.go Do not exist 77.8%
pkg/reconciler/pipelinerun/pipelinerun.go 85.6% 85.8% 0.2
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go 90.2% 90.8% 0.6
pkg/reconciler/pipelinerun/resources/resultrefresolution.go 87.8% 87.8% -0.0

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
internal/builder/v1beta1/pipeline.go 81.8% 82.0% 0.2
pkg/apis/pipeline/v1beta1/pipeline_types.go 60.0% 63.9% 3.9
pkg/apis/pipeline/v1beta1/pipeline_validation.go 96.1% 94.3% -1.8
pkg/apis/pipeline/v1beta1/when_types.go Do not exist 61.3%
pkg/apis/pipeline/v1beta1/when_validation.go Do not exist 92.9%
pkg/reconciler/pipelinerun/pipelinerun.go 86.3% 85.4% -0.9
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go 90.2% 90.6% 0.4
pkg/reconciler/pipelinerun/resources/resultrefresolution.go 88.7% 93.1% 4.4
pkg/when/when.go Do not exist 100.0%

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
internal/builder/v1beta1/pipeline.go 81.8% 82.0% 0.2
pkg/apis/pipeline/v1beta1/pipeline_types.go 60.0% 63.9% 3.9
pkg/apis/pipeline/v1beta1/pipeline_validation.go 96.1% 94.3% -1.8
pkg/apis/pipeline/v1beta1/when_types.go Do not exist 61.3%
pkg/apis/pipeline/v1beta1/when_validation.go Do not exist 92.9%
pkg/reconciler/pipelinerun/pipelinerun.go 86.3% 85.4% -0.9
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go 90.2% 90.6% 0.4
pkg/reconciler/pipelinerun/resources/resultrefresolution.go 88.7% 91.6% 2.9
pkg/when/when.go Do not exist 100.0%

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
internal/builder/v1beta1/pipeline.go 81.8% 82.0% 0.2
pkg/apis/pipeline/v1beta1/pipeline_types.go 60.0% 63.9% 3.9
pkg/apis/pipeline/v1beta1/pipeline_validation.go 96.1% 94.3% -1.8
pkg/apis/pipeline/v1beta1/when_types.go Do not exist 61.3%
pkg/apis/pipeline/v1beta1/when_validation.go Do not exist 92.9%
pkg/reconciler/pipelinerun/pipelinerun.go 86.3% 86.1% -0.2
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go 90.2% 90.6% 0.4
pkg/reconciler/pipelinerun/resources/resultrefresolution.go 88.7% 91.6% 2.9
pkg/when/when.go Do not exist 100.0%

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
internal/builder/v1beta1/pipeline.go 81.8% 82.0% 0.2
pkg/apis/pipeline/v1beta1/pipeline_types.go 60.0% 63.9% 3.9
pkg/apis/pipeline/v1beta1/pipeline_validation.go 96.1% 94.3% -1.8
pkg/apis/pipeline/v1beta1/when_types.go Do not exist 61.3%
pkg/apis/pipeline/v1beta1/when_validation.go Do not exist 92.9%
pkg/reconciler/pipelinerun/pipelinerun.go 86.3% 86.6% 0.3
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go 90.2% 90.6% 0.4
pkg/reconciler/pipelinerun/resources/resultrefresolution.go 88.7% 91.6% 2.9
pkg/when/when.go Do not exist 100.0%

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
internal/builder/v1beta1/pipeline.go 81.8% 82.0% 0.2
pkg/apis/pipeline/v1beta1/pipeline_types.go 60.0% 63.9% 3.9
pkg/apis/pipeline/v1beta1/pipeline_validation.go 96.1% 96.1% 0.0
pkg/apis/pipeline/v1beta1/when_types.go Do not exist 45.5%
pkg/apis/pipeline/v1beta1/when_validation.go Do not exist 96.7%
pkg/reconciler/pipelinerun/pipelinerun.go 86.3% 86.6% 0.3
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go 90.2% 90.6% 0.4
pkg/reconciler/pipelinerun/resources/resultrefresolution.go 88.7% 91.6% 2.9
pkg/when/when.go Do not exist 100.0%

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
internal/builder/v1beta1/pipeline.go 81.8% 82.0% 0.2
pkg/apis/pipeline/v1beta1/pipeline_types.go 60.0% 63.9% 3.9
pkg/apis/pipeline/v1beta1/pipeline_validation.go 96.1% 96.1% 0.0
pkg/apis/pipeline/v1beta1/when_types.go Do not exist 86.4%
pkg/apis/pipeline/v1beta1/when_validation.go Do not exist 100.0%
pkg/reconciler/pipelinerun/pipelinerun.go 86.3% 86.6% 0.3
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go 90.2% 90.6% 0.4
pkg/reconciler/pipelinerun/resources/resultrefresolution.go 88.7% 93.1% 4.4
pkg/when/when.go Do not exist 100.0%

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
internal/builder/v1beta1/pipeline.go 81.8% 82.0% 0.2
pkg/apis/pipeline/v1beta1/pipeline_types.go 60.0% 63.9% 3.9
pkg/apis/pipeline/v1beta1/pipeline_validation.go 96.1% 96.1% 0.0
pkg/apis/pipeline/v1beta1/when_types.go Do not exist 86.4%
pkg/apis/pipeline/v1beta1/when_validation.go Do not exist 100.0%
pkg/reconciler/pipelinerun/pipelinerun.go 86.3% 86.6% 0.3
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go 90.2% 90.6% 0.4
pkg/reconciler/pipelinerun/resources/resultrefresolution.go 88.7% 91.6% 2.9
pkg/when/when.go Do not exist 100.0%

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
internal/builder/v1beta1/pipeline.go 81.8% 82.0% 0.2
pkg/apis/pipeline/v1beta1/pipeline_types.go 60.0% 63.9% 3.9
pkg/apis/pipeline/v1beta1/pipeline_validation.go 96.1% 96.1% 0.0
pkg/apis/pipeline/v1beta1/when_types.go Do not exist 100.0%
pkg/apis/pipeline/v1beta1/when_validation.go Do not exist 100.0%
pkg/reconciler/pipelinerun/pipelinerun.go 86.3% 86.6% 0.3
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go 90.2% 90.6% 0.4
pkg/reconciler/pipelinerun/resources/resultrefresolution.go 88.7% 91.6% 2.9
pkg/when/when.go Do not exist 100.0%

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
internal/builder/v1beta1/pipeline.go 81.8% 82.0% 0.2
pkg/apis/pipeline/v1beta1/pipeline_types.go 60.0% 63.9% 3.9
pkg/apis/pipeline/v1beta1/pipeline_validation.go 96.1% 96.1% 0.0
pkg/apis/pipeline/v1beta1/when_types.go Do not exist 100.0%
pkg/apis/pipeline/v1beta1/when_validation.go Do not exist 100.0%
pkg/reconciler/pipelinerun/pipelinerun.go 86.3% 86.6% 0.3
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go 90.2% 90.6% 0.4
pkg/reconciler/pipelinerun/resources/resultrefresolution.go 88.7% 91.6% 2.9
pkg/when/when.go Do not exist 100.0%

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
internal/builder/v1beta1/pipeline.go 81.8% 82.0% 0.2
pkg/apis/pipeline/v1beta1/pipeline_types.go 60.0% 63.9% 3.9
pkg/apis/pipeline/v1beta1/pipeline_validation.go 96.1% 96.1% 0.0
pkg/apis/pipeline/v1beta1/when_types.go Do not exist 100.0%
pkg/apis/pipeline/v1beta1/when_validation.go Do not exist 100.0%
pkg/reconciler/pipelinerun/pipelinerun.go 86.3% 86.6% 0.3
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go 90.2% 90.6% 0.4
pkg/reconciler/pipelinerun/resources/resultrefresolution.go 88.7% 93.1% 4.4
pkg/when/when.go Do not exist 100.0%

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
internal/builder/v1beta1/pipeline.go 81.8% 82.0% 0.2
pkg/apis/pipeline/v1beta1/pipeline_types.go 60.0% 63.9% 3.9
pkg/apis/pipeline/v1beta1/pipeline_validation.go 96.1% 96.1% 0.0
pkg/apis/pipeline/v1beta1/when_types.go Do not exist 69.2%
pkg/apis/pipeline/v1beta1/when_validation.go Do not exist 100.0%
pkg/reconciler/pipelinerun/pipelinerun.go 86.3% 85.9% -0.5
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go 90.2% 88.3% -1.8
pkg/reconciler/pipelinerun/resources/resultrefresolution.go 87.8% 91.6% 3.8

Discussed with other pipelines owners today (@vdemeester, @sbwsg,
@dibyom @imjasonh and we think it makes sense to include this in the
roadmap tho we'd want to flesh out the details and might not want it
exactly as described in the original issue.

Also added area/roadmap labels to the items listed here!
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
internal/builder/v1beta1/pipeline.go 81.8% 82.0% 0.2
pkg/apis/pipeline/v1beta1/pipeline_types.go 60.0% 63.9% 3.9
pkg/apis/pipeline/v1beta1/pipeline_validation.go 96.1% 96.1% 0.0
pkg/apis/pipeline/v1beta1/when_types.go Do not exist 69.2%
pkg/apis/pipeline/v1beta1/when_validation.go Do not exist 100.0%
pkg/reconciler/pipelinerun/pipelinerun.go 86.3% 85.9% -0.5
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go 90.2% 88.3% -1.8
pkg/reconciler/pipelinerun/resources/resultrefresolution.go 87.8% 91.6% 3.8

@tekton-robot
Copy link
Collaborator

@jerop: The following tests failed, say /retest to rerun them all:

Test name Commit Details Rerun command
pull-tekton-pipeline-build-tests cfc1ff6 link /test pull-tekton-pipeline-build-tests
pull-tekton-pipeline-integration-tests cfc1ff6 link /test pull-tekton-pipeline-integration-tests

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

1 similar comment
@tekton-robot
Copy link
Collaborator

@jerop: The following tests failed, say /retest to rerun them all:

Test name Commit Details Rerun command
pull-tekton-pipeline-build-tests cfc1ff6 link /test pull-tekton-pipeline-build-tests
pull-tekton-pipeline-integration-tests cfc1ff6 link /test pull-tekton-pipeline-integration-tests

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@jerop jerop closed this Aug 25, 2020
@pritidesai
Copy link
Member

Hey @jerop what happened? creating a new/separate PR?

@jerop
Copy link
Member Author

jerop commented Aug 25, 2020

@pritidesai yes, here it is: #3135

@jerop jerop deleted the when branch September 30, 2020 16:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature Categorizes issue or PR as related to a new feature. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants