-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
TEP-0059: Skipping Strategies #4085
Conversation
The following is the coverage report on the affected files.
|
84da9bd
to
ffa24e4
Compare
The following is the coverage report on the affected files.
|
ffa24e4
to
e1bc867
Compare
The following is the coverage report on the affected files.
|
/assign @sbwsg |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great! Suggest also updating docs/install.md
's feature flags section with info about this new flag.
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: sbwsg The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
so exciting to see this change happening!!! hope you don't mind a barrage of feedback from me - mostly about docs + some code/test clarity
also i didnt see much in the docs or tests handling the behavioral differences around resource vs. ordering dependencies - my understanding is that even if the when expression is scoped to just the task, that if other tasks depend on results (and eventually workspaces) from that task, they will still be skipped (because we have no way to provide a valid result to them). is that right? (and either way i think the docs + tests maybe should represent these cases also)
p.s. great commit message!! :D
after migration, we'll change the global default for the feature flag to true to guard a Task only by default
i'm wondering how to minimize the problems this could cause 🤔 hopefully no one is relying on it and if they are the see our warnings 😟
maybe we should send some announcements to tekton-dev and tekton-users about this
we could even have something like the CLI notice when ppl are using when expressions to guard branches and warn them - probably too far but we could even have the CLI 'phone home' and tell us when folks are doing that; phoning home should probably be discussed on its own tho, maybe its enough to loudly warn ppl
at the very least i think we should make the release note that accompanies this (and subsequent ones) stand out a bit more - we need to keep announcing this as loudly as we can
e1bc867
to
379ba8d
Compare
The following is the coverage report on the affected files.
|
379ba8d
to
6cbeb6f
Compare
/test pull-tekton-pipeline-alpha-integration-tests |
/lgtm |
today, we existence of variables to determine whether to add `when` expressions to the skipped `tasks` status for finally tasks (we only add them when they are resolved, have no variables) we recently added skipping reason in tektoncd#4085 now that we have skipping reason, we can use it to check that the reason for skipping is that `when` expressions evaluated to false before including the resolved `when` expressions to the skipped `tasks` status we plan to explore adding skipping reason to the skipped `tasks` status in the future, but for now this change reuses this new functionality
today, we existence of variables to determine whether to add `when` expressions to the skipped `tasks` status for finally tasks (we only add them when they are resolved, have no variables) we recently added skipping reason in tektoncd#4085 now that we have skipping reason, we can use it to check that the reason for skipping is that `when` expressions evaluated to false before including the resolved `when` expressions to the skipped `tasks` status we plan to explore adding skipping reason to the skipped `tasks` status in the future, but for now this change reuses this new functionality
today, we existence of variables to determine whether to add `when` expressions to the skipped `tasks` status for finally tasks (we only add them when they are resolved, have no variables) we recently added skipping reason in tektoncd#4085 now that we have skipping reason, we can use it to check that the reason for skipping is that `when` expressions evaluated to false before including the resolved `when` expressions to the skipped `tasks` status we plan to explore adding skipping reason to the skipped `tasks` status in the future, but for now this change reuses this new functionality
today, we existence of variables to determine whether to add `when` expressions to the skipped `tasks` status for finally tasks (we only add them when they are resolved, have no variables) we recently added skipping reason in #4085 now that we have skipping reason, we can use it to check that the reason for skipping is that `when` expressions evaluated to false before including the resolved `when` expressions to the skipped `tasks` status we plan to explore adding skipping reason to the skipped `tasks` status in the future, but for now this change reuses this new functionality
In [TEP-0007: Conditions Beta][tep-0007], we introduced `when` expressions to guard execution of `Tasks` in `Pipelines. To align with `Conditions`, the scope of `when` expressions was set to the `Task` and its dependent `Tasks`. In [TEP-0059: Skipping Strategies][tep-0059], we proposed changing the scope of `when` expressions to the guarded `Task` only. This was implemented in tektoncd#4085 where we provided a feature flag, `scope-when-expressions-to-task`, to support migration. It defaulted to `false` for 9 months per our [Beta API compatibility policy][policy], meaning that we continued to guard the `Task` and its dependent `Tasks`. In this change, we flip the flag to `true` to guard the `Task` only by default. [tep-0007]: https://github.com/tektoncd/community/blob/main/teps/0007-conditions-beta.md [tep-0059]: https://github.com/tektoncd/community/blob/main/teps/0059-skipping-strategies.md [policy]: https://github.com/tektoncd/pipeline/blob/main/api_compatibility_policy.md
In [TEP-0007: Conditions Beta][tep-0007], we introduced `when` expressions to guard execution of `Tasks` in `Pipelines`. To align with `Conditions`, we set scope of `when` expressions to the guarded `Task` and its dependent `Tasks`. In [TEP-0059: Skipping Strategies][tep-0059], we proposed changing the scope of `when` expressions to the guarded `Task` only. This was implemented in tektoncd#4085. We provided a feature flag, `scope-when-expressions-to-task`, to support migration. It defaulted to `false` for 9 months per our [Beta API compatibility policy][policy], meaning that we continued to guard the `Task` and its dependent `Tasks`. In this change, we flip the flag to `true` to guard the `Task` only by default. [tep-0007]: https://github.com/tektoncd/community/blob/main/teps/0007-conditions-beta.md [tep-0059]: https://github.com/tektoncd/community/blob/main/teps/0059-skipping-strategies.md [policy]: https://github.com/tektoncd/pipeline/blob/main/api_compatibility_policy.md
In [TEP-0007: Conditions Beta][tep-0007], we introduced `when` expressions to guard execution of `Tasks` in `Pipelines`. To align with `Conditions`, we set scope of `when` expressions to the guarded `Task` and its dependent `Tasks`. In [TEP-0059: Skipping Strategies][tep-0059], we proposed changing the scope of `when` expressions to the guarded `Task` only. This was implemented in tektoncd#4085. We provided a feature flag, `scope-when-expressions-to-task`, to support migration. It defaulted to `false` for 9 months per our [Beta API compatibility policy][policy], meaning that we continued to guard the `Task` and its dependent `Tasks`. In this change, we flip the flag to `true` to guard the `Task` only by default. [tep-0007]: https://github.com/tektoncd/community/blob/main/teps/0007-conditions-beta.md [tep-0059]: https://github.com/tektoncd/community/blob/main/teps/0059-skipping-strategies.md [policy]: https://github.com/tektoncd/pipeline/blob/main/api_compatibility_policy.md
In [TEP-0007: Conditions Beta][tep-0007], we introduced `when` expressions to guard execution of `Tasks` in `Pipelines`. To align with `Conditions`, we set scope of `when` expressions to the guarded `Task` and its dependent `Tasks`. In [TEP-0059: Skipping Strategies][tep-0059], we proposed changing the scope of `when` expressions to the guarded `Task` only. This was implemented in tektoncd#4085. We provided a feature flag, `scope-when-expressions-to-task`, to support migration. It defaulted to `false` for 9 months per our [Beta API compatibility policy][policy], meaning that we continued to guard the `Task` and its dependent `Tasks`. In this change, we flip the flag to `true` to guard the `Task` only by default. [tep-0007]: https://github.com/tektoncd/community/blob/main/teps/0007-conditions-beta.md [tep-0059]: https://github.com/tektoncd/community/blob/main/teps/0059-skipping-strategies.md [policy]: https://github.com/tektoncd/pipeline/blob/main/api_compatibility_policy.md
In [TEP-0007: Conditions Beta][tep-0007], we introduced `when` expressions to guard execution of `Tasks` in `Pipelines`. To align with `Conditions`, we set scope of `when` expressions to the guarded `Task` and its dependent `Tasks`. In [TEP-0059: Skipping Strategies][tep-0059], we proposed changing the scope of `when` expressions to the guarded `Task` only. This was implemented in tektoncd#4085. We provided a feature flag, `scope-when-expressions-to-task`, to support migration. It defaulted to `false` for 9 months per our [Beta API compatibility policy][policy], meaning that we continued to guard the `Task` and its dependent `Tasks`. In this change, we flip the flag to `true` to guard the `Task` only by default. [tep-0007]: https://github.com/tektoncd/community/blob/main/teps/0007-conditions-beta.md [tep-0059]: https://github.com/tektoncd/community/blob/main/teps/0059-skipping-strategies.md [policy]: https://github.com/tektoncd/pipeline/blob/main/api_compatibility_policy.md
In [TEP-0007: Conditions Beta][tep-0007], we introduced `when` expressions to guard execution of `Tasks` in `Pipelines`. To align with `Conditions`, we set scope of `when` expressions to the guarded `Task` and its dependent `Tasks`. In [TEP-0059: Skipping Strategies][tep-0059], we proposed changing the scope of `when` expressions to the guarded `Task` only. This was implemented in #4085. We provided a feature flag, `scope-when-expressions-to-task`, to support migration. It defaulted to `false` for 9 months per our [Beta API compatibility policy][policy], meaning that we continued to guard the `Task` and its dependent `Tasks`. In this change, we flip the flag to `true` to guard the `Task` only by default. [tep-0007]: https://github.com/tektoncd/community/blob/main/teps/0007-conditions-beta.md [tep-0059]: https://github.com/tektoncd/community/blob/main/teps/0059-skipping-strategies.md [policy]: https://github.com/tektoncd/pipeline/blob/main/api_compatibility_policy.md
* cleanup - ApplyContext parameters Instead of passing around the entire resolvedTaskResources, which is not necessary at this point, just pass the task name. No functional changes expected. * use podtemplate imagepullsecrets to resolve entrypoint * Update write_test.go Fixed a typo * Fix links to Why Aren't PipelineResources in Beta? Links to the "Why Aren't PipelineResources in Beta?" section in the docs should have `aren-t` in the fragment instead of `arent`. This can be confirmed by clicking the link icon beside the heading and checking the browser address bar. * Fix tekton_pipelines_controller_taskrun_count recount bug Added before and after condition check to avoid taskrun metrics recount bug. * debug is an alpha feature Documenting that the debug feature is still alpha. The feature was introduced in pipelines release 0.26 behind enable-api-fields flag. * Consider osversion when determining platform uniqueness Prior to this change, an image (such as `golang:1.17`) that provided two images that shared the same OS+architecture+variant would be considered invalid, even if they described two different images whose platforms differed on, for example, osversion (used by Windows images). This change relaxes our platform uniqueness logic to take this into account, unblocking Linux users from running such images. There's still an issue for Windows users however, since when they attempt to run these images they'll fail to find the correct command taking into account their osversion. Workarounds in this case include specifying a single-platform image, or avoiding multi-platform images that provide two Windows images differing only by osversion. This also updates our selection logic to take into account slightly malformed multi-platform images that specify two images with the same OS+architecture[+variant], so long as the duplicate entries describe the same image by digest (e.g., anchore/syft:v0.37.10) * [TEP-0059] Scope `when` expressions to `Task` only In [TEP-0007: Conditions Beta][tep-0007], we introduced `when` expressions to guard execution of `Tasks` in `Pipelines`. To align with `Conditions`, we set scope of `when` expressions to the guarded `Task` and its dependent `Tasks`. In [TEP-0059: Skipping Strategies][tep-0059], we proposed changing the scope of `when` expressions to the guarded `Task` only. This was implemented in tektoncd#4085. We provided a feature flag, `scope-when-expressions-to-task`, to support migration. It defaulted to `false` for 9 months per our [Beta API compatibility policy][policy], meaning that we continued to guard the `Task` and its dependent `Tasks`. In this change, we flip the flag to `true` to guard the `Task` only by default. [tep-0007]: https://github.com/tektoncd/community/blob/main/teps/0007-conditions-beta.md [tep-0059]: https://github.com/tektoncd/community/blob/main/teps/0059-skipping-strategies.md [policy]: https://github.com/tektoncd/pipeline/blob/main/api_compatibility_policy.md * Update the `scope-when-expressions-to-task` feature flag docs In tektoncd#4580, we changed the flag default from "false" to "true". However, the documentation above the flag was still describing what setting it to "true" would do. In this change, we update the documentation to focus on the non-default option that users can choose to set - "false". We also add a reference to TEP-0059 and relevant docs for more details. * Patch temp GOPATH hack script to handle nounset option Prior to this commit the setup-temporary-gopath.sh used the GOPATH variable without first checking that it was set. When `set -o nounset` is working this causes the script to exit with an error. This commit adds a variable wrapping $GOPATH and setting a default if it's missing, which should work around the `nounset`. * use helper functions - MarkResource* Replace updating the conditions directly with the helper functions - MarkResourceRunning and MarkRunning. No functional change expected. * Update the deprecations table The tekton.dev/task label for ClusterTasks have been removed in tektoncd#2533, but the table has not been updated yet, so doing it in here. Signed-off-by: Andrea Frittoli <andrea.frittoli@uk.ibm.com> * Remove deprecated flags home-env and working-dir This change removes two flags: - disable-home-env-overwrite - disable-working-dir-overwrite That two flags that were originally introduced with default to false and the feature associated to them was deprecated. Nine months later (as per policy), in Dec 2020, the default value was switched to default true and the flags were deprecated. Nine months later we are finally removing the flags. Signed-off-by: Andrea Frittoli <andrea.frittoli@uk.ibm.com> * Fix for some arm64 machines. As said in GoogleContainerTools/distroless#657, in the past, distroless/base:debug used an arm32 busybox binary in its arm64 image. Which doesn't work on some arm64 machines, e.g., Ubuntu 21 arm64 on Parallel Desktop on Apple Silicon M1. It caused this error: " $ docker run -it gcr.io/distroless/base@sha256:cfdc553400d41b47fd231b028403469811fcdbc0e69d66ea8030c5a0b5fbac2b standard_init_linux.go:228: exec user process caused: exec format error " This PR GoogleContainerTools/distroless#960 fixes this bug. Hence, update the distroless/base:debug used by Tekton Pipeline in this commit. * Add Step and Sidecar Overrides to TaskRun API This commit adds TaskRunStepOverrides and TaskRunSidecarOverrides to TaskRun.Spec and PipelineRun.Spec.PipelineTaskRunSpec, gated behind the "alpha" API flag. This is part 1 of implementing TEP-0094: Configuring Resource Requirements at Runtime. https://github.com/tektoncd/community/blob/main/teps/0094-configuring-resources-at-runtime.md * WIP spire. Signed-off-by: Dan Lorenc <dlorenc@google.com> changed to use spiffe-csi Add pod SPIFFE id annotation for workload registrar Signed-off-by: Brandon Lum <lumjjb@gmail.com> removed spire jwt updated obtaining trust bundle Added SPIFFE entry registration and SVID entrypointer backoff (#2) * Added SPIFFE entry registration and SVID entrypointer backoff Signed-off-by: Brandon Lum <lumjjb@gmail.com> * Allow SPIRE configuration through opts Signed-off-by: Brandon Lum <lumjjb@gmail.com> * Add validation of SpireConfig Signed-off-by: Brandon Lum <lumjjb@gmail.com> * merged upstream Signed-off-by: pxp928 <parth.psu@gmail.com> * added manifest check * [WIP] Add SPIRE docs (#4) * merged upstream * Add several features/optimizations for SPIRE (#3) * Record pod latency before SPIRE entry creation Signed-off-by: Brandon Lum <lumjjb@gmail.com> * SPIRE client connection caching Signed-off-by: Brandon Lum <lumjjb@gmail.com> * Optimize spire entry creation Signed-off-by: Brandon Lum <lumjjb@gmail.com> * Add TTL for workload entry based on taskrun timeout Signed-off-by: Brandon Lum <lumjjb@gmail.com> * Add SPIRE non-falsification doc Signed-off-by: Brandon Lum <lumjjb@gmail.com> Co-authored-by: pxp928 <parth.psu@gmail.com> * merged upstream Signed-off-by: pxp928 <parth.psu@gmail.com> Co-authored-by: pritidesai <pdesai@us.ibm.com> Co-authored-by: Yongxuan Zhang <yongxuanzhang@google.com> Co-authored-by: Anupama Baskar <anu.baskar@ibm.com> Co-authored-by: Alan Greene <github.com@alangreene.net> Co-authored-by: Khurram Baig <kbaig@redhat.com> Co-authored-by: Jason Hall <jasonhall@redhat.com> Co-authored-by: Jerop <jerop@google.com> Co-authored-by: Scott <sbws@google.com> Co-authored-by: Andrea Frittoli <andrea.frittoli@uk.ibm.com> Co-authored-by: Meng-Yuan Huang <myh@live.com> Co-authored-by: Lee Bernick <lee.a.bernick@gmail.com> Co-authored-by: Dan Lorenc <dlorenc@google.com> Co-authored-by: Brandon Lum <lumjjb@gmail.com>
In [TEP-0007: Conditions Beta][tep-0007], we introduced `when` expressions to guard execution of `Tasks` in `Pipelines`. To align with `Conditions`, we set scope of `when` expressions to the guarded `Task` and its dependent `Tasks`. In [TEP-0059: Skipping Strategies][tep-0059], we proposed changing the scope of `when` expressions to the guarded `Task` only. This was implemented in tektoncd#4085. We provided a feature flag, `scope-when-expressions-to-task`, to support migration. It defaulted to `false` for 9 months per our [Beta API compatibility policy][policy], meaning that we continued to guard the `Task` and its dependent `Tasks`. Then in tektoncd#4580, we flipped the flag to `true` to guard the `Task` only by default. In this change, we remove the `scope-when-expressions-to-task` flag and complete the migration. [tep-0007]: https://github.com/tektoncd/community/blob/main/teps/0007-conditions-beta.md [tep-0059]: https://github.com/tektoncd/community/blob/main/teps/0059-skipping-strategies.md [policy]: https://github.com/tektoncd/pipeline/blob/main/api_compatibility_policy.md
In [TEP-0007: Conditions Beta][tep-0007], we introduced `when` expressions to guard execution of `Tasks` in `Pipelines`. To align with `Conditions`, we set scope of `when` expressions to the guarded `Task` and its dependent `Tasks`. In [TEP-0059: Skipping Strategies][tep-0059], we proposed changing the scope of `when` expressions to the guarded `Task` only. This was implemented in tektoncd#4085. We provided a feature flag, `scope-when-expressions-to-task`, to support migration. It defaulted to `false` for 9 months per our [Beta API compatibility policy][policy], meaning that we continued to guard the `Task` and its dependent `Tasks`. Then in tektoncd#4580, we flipped the flag to `true` to guard the `Task` only by default. In this change, we remove the `scope-when-expressions-to-task` flag and complete the migration. Closes tektoncd#4461. [tep-0007]: https://github.com/tektoncd/community/blob/main/teps/0007-conditions-beta.md [tep-0059]: https://github.com/tektoncd/community/blob/main/teps/0059-skipping-strategies.md [policy]: https://github.com/tektoncd/pipeline/blob/main/api_compatibility_policy.md
In [TEP-0007: Conditions Beta][tep-0007], we introduced `when` expressions to guard execution of `Tasks` in `Pipelines`. To align with `Conditions`, we set scope of `when` expressions to the guarded `Task` and its dependent `Tasks`. In [TEP-0059: Skipping Strategies][tep-0059], we proposed changing the scope of `when` expressions to the guarded `Task` only. This was implemented in tektoncd#4085. We provided a feature flag, `scope-when-expressions-to-task`, to support migration. It defaulted to `false` for 9 months per our [Beta API compatibility policy][policy], meaning that we continued to guard the `Task` and its dependent `Tasks`. Then in tektoncd#4580, we flipped the flag to `true` to guard the `Task` only by default. In this change, we remove the `scope-when-expressions-to-task` flag and complete the migration. Closes tektoncd#4461. [tep-0007]: https://github.com/tektoncd/community/blob/main/teps/0007-conditions-beta.md [tep-0059]: https://github.com/tektoncd/community/blob/main/teps/0059-skipping-strategies.md [policy]: https://github.com/tektoncd/pipeline/blob/main/api_compatibility_policy.md
In [TEP-0007: Conditions Beta][tep-0007], we introduced `when` expressions to guard execution of `Tasks` in `Pipelines`. To align with `Conditions`, we set scope of `when` expressions to the guarded `Task` and its dependent `Tasks`. In [TEP-0059: Skipping Strategies][tep-0059], we proposed changing the scope of `when` expressions to the guarded `Task` only. This was implemented in #4085. We provided a feature flag, `scope-when-expressions-to-task`, to support migration. It defaulted to `false` for 9 months per our [Beta API compatibility policy][policy], meaning that we continued to guard the `Task` and its dependent `Tasks`. Then in #4580, we flipped the flag to `true` to guard the `Task` only by default. In this change, we remove the `scope-when-expressions-to-task` flag and complete the migration. Closes #4461. [tep-0007]: https://github.com/tektoncd/community/blob/main/teps/0007-conditions-beta.md [tep-0059]: https://github.com/tektoncd/community/blob/main/teps/0059-skipping-strategies.md [policy]: https://github.com/tektoncd/pipeline/blob/main/api_compatibility_policy.md
- Enable TEP-0059 tektoncd/pipeline#4085 - Build definitons repository is ready for this change
In [TEP-0007: Conditions Beta][tep-0007], we introduced `when` expressions to guard execution of `Tasks` in `Pipelines`. To align with `Conditions`, we set scope of `when` expressions to the guarded `Task` and its dependent `Tasks`. In [TEP-0059: Skipping Strategies][tep-0059], we proposed changing the scope of `when` expressions to the guarded `Task` only. This was implemented in tektoncd#4085. We provided a feature flag, `scope-when-expressions-to-task`, to support migration. It defaulted to `false` for 9 months per our [Beta API compatibility policy][policy], meaning that we continued to guard the `Task` and its dependent `Tasks`. In this change, we flip the flag to `true` to guard the `Task` only by default. [tep-0007]: https://github.com/tektoncd/community/blob/main/teps/0007-conditions-beta.md [tep-0059]: https://github.com/tektoncd/community/blob/main/teps/0059-skipping-strategies.md [policy]: https://github.com/tektoncd/pipeline/blob/main/api_compatibility_policy.md
Changes
This change implements skipping strategies to give users the flexibility
to skip a single guarded Task only and unblock execution of its
dependent Tasks.
Today, WhenExpressions are specified within Tasks but they guard the
Task and its dependent Tasks. To provide flexible skipping strategies,
we want to change the scope of WhenExpressions from guarding a Task and
its dependent Tasks to guarding the Task only. If a user wants to guard
a Task and its dependent Tasks, they can:
guarded and executed together using Pipelines in Pipelines (but this is
still an experimental feature)
Changing the scope of WhenExpressions to guard the Task only is
backwards-incompatible, so to make the transition smooth:
to true to guard a Task only by default
forward
Implements TEP-0059: Skipping Strategies
Closes #2127
Submitter Checklist
As the author of this PR, please check off the items in this checklist:
functionality, content, code)
Release Notes