Release v1 versions of Task, TaskRun, Pipeline, and PipelineRun CRDs #5541

lbernick · 2022-09-22T14:16:32Z

This issue tracks the release of v1 api versions for Task, TaskRun, Pipeline, and PipelineRun.

Before doing this, we would like to wait for remote resolution beta (#4710), since we want to provide an alternative to taskref.bundle and pipelineref.bundle.

It would be nice to wait for custom tasks beta (#4313) so we can release v1 versions of these CRDs along with custom task beta, but there's still more work to do there and I'm not sure this should be a blocker.

Steps:

Create examples tests for v1 versions of our CRDs (probably just copy pasting existing examples tests, swapping api versions, and removing any that aren't relevant for v1 such as those that test pipelineresources). This will probably require step 2 but it would be nice if there's a way to do it first. It will probably be easiest to do all the CRDs at once, but it's not necessary to do so.
Update CRDs to serve v1 versions. At this point, we're "committing" to the structure of the v1 api-- no more changes.
Update CRDs to use v1 as the stored version, and in the same PR, update the reconcilers to reconcile the v1 versions of the CRDs. Again, we don't have to do all the CRDs at once here, but we might find it easier to do so. One catch is that we must deserialize PipelineResources from annotations in v1 CRDs if they are present. (They may have been created by conversion from v1beta1 CRDs referencing PipelineResources.) Failing to support PipelineResources in this way would be violating our compatibility policy by not supporting a v1beta1 feature for 9 months following the v1 release. After completing step 3, we'll be able to add new features in v1 and should stop making any changes to v1beta1.

lbernick · 2022-09-22T18:56:33Z

@tektoncd/core-maintainers since remote resolution beta is in our 0.41.0 milestone, I think we should start work on this rather than blocking on the remaining work that needs to be done for custom tasks beta. It would be nice to release them all at once, but I don't see any reason why these CRDs in v1 wouldn't work with v1alpha1 Runs for the time being. Thoughts?

lbernick · 2022-09-26T18:57:22Z

Discussed in today's api wg: folks seem to be OK with going forward with this plan without blocking on custom tasks beta but would like to block on remote resolution beta. We should review our plan for supporting deprecated fields (#4546) before swapping our stored versions over to v1.

JeromeJu · 2022-11-21T15:04:37Z

Would like to confirm the understanding on the part iii.

Basically what is needed here is changing most v1beta1 to v1 in reconcilers and regarding directories. While for the deprecated fields, I am referring to the previous v1alpha1 to v1beta1 changes and in the example of deprecated resources for taskRun for v1, I would need to check tr.ObjectMeta.Annotations contains resources and if so reusing the v1beta1 reconciling codes while marking it as deprecated in comments?

lbernick · 2022-11-21T15:06:47Z

Yes, that sounds right to me!

JeromeJu · 2023-01-10T22:03:43Z

Deprecated features that needs to be handled as part of the storage version migration:

Removing PipelineResources
Related with Handle PipelineResources when swapping v1 to the storage version #5967
Removed PipelineRunStatus (can be removed in release v0.44 Jan 25)

The followings are not blockers of the v1 storage swap:

PipelineRun.Timeout (Feb 25)
CloudEvents (July 13)
ClusterTask (July 13)
related with Handle v1beta1.taskObject interface and v1beta1.pipelineObject interface for V1 as the storage version #5979

JeromeJu · 2023-03-27T16:52:06Z

What needs to be done before swapping v1 storage:

Refine conversion test: as described before for preventing issues ie. Pipelines v0.43.1 installation nondeterministically fails to update PipelineRun status with TaskRun statuses #5964 - added Add RoundTrip to Conversion Integration Test #6439
Handle v1beta1.taskObject interface and v1beta1.pipelineObject interface for V1 as the storage version #5979
Handle PipelineResources when swapping v1 to the storage version #5967

lbernick · 2023-03-27T17:01:08Z

I don't think addressing issues like #5964 is a blocker for swapping the storage version to v1. It seems like #5979 and #5967 are the only blockers?

JeromeJu · 2023-03-27T17:10:33Z

I don't think addressing issues like #5964 is a blocker for swapping the storage version to v1.

If that makes sense, my understanding is instead of addressing issues like #5964 , the round-trip conversion test is to prevent such issue from happening.

It seems like #5979 and #5967 are the only blockers?

Yes, I agree. But I think #5979 is going to be handled within the same PR of the storage swap since I could not figure out a way to separately isolate the TaskObject interface.

lbernick · 2023-03-27T18:57:19Z

Would you mind updating #5979 with the details of what you tried?

JeromeJu · 2023-03-28T12:30:35Z

Thanks to the guidance and talked offline with @vdemeester , the following would also be blockers/ bugs for v1 storage version change:

[v0.44.x] Conversion webhook is misbehaving on kubectl patch #6443 - this might relate with the test cases we currently have
Align on Handling v1 storage conversion #6382

JeromeJu · 2023-04-04T16:57:02Z

Aligned in WG @4 April that we are going to delay to v0.48, which is after the earliest LTS v0.47 to minimize impacts on LTS users while we are going to merge v1 storage swap soon as possible in early v0.48 to get enough window for fixing for v0.48.

This commit makes the conversions required for reconciling the deprecated v1beta1 CustomTask public in the apis. part of tektoncd#5541

This commit makes the conversions required for reconciling the deprecated v1beta1 CustomTask exported in the apis. part of tektoncd#5541

JeromeJu · 2023-04-05T15:52:40Z

Current blockers/ questions:

Shall we Export required Conversions for Reconciling the Deprecated CustomRun #6496
And also for alpha feature in TEP-0094: Configuring Resource Requirements at Runtime shall we Sync v1 TaskRun StepSpecs and SidecarSpecs Merge with v1beta1 Overrides Merge #6497

This commit exports the conversions required for reconciling the deprecated v1beta1 CustomRun in the apis. part of tektoncd#5541

JeromeJu · 2023-04-27T14:59:17Z

This is to update the current blockers:

Handle Loss of Deprecated Container Fields in v1 for Step LivenessProbes and etc.
We need to update the default type assignment for empty map instantiated in conversions: (for context, taskResults.properties are created as an empty map in conversions) Fix conversion of non-object results declared in Tasks #6606
Decoupling API versioning and Feature versioning for features turned on by default #6592
- Resolver to Stable #6579

One of the other issue uncovered from the previous issues is that the resultRef resolution at

pipeline/pkg/reconciler/pipelinerun/pipelinerun.go

Lines 720 to 723 in 3d6ec63

    
           resolvedResultRefs, _, err := resources.ResolveResultRefs(pipelineRunFacts.State, nextRpts) 
        
           if err != nil { 
        
           	logger.Infof("Failed to resolve task result reference for %q with error %v", pr.Name, err) 
        
           	pr.Status.MarkFailed(ReasonInvalidTaskResultReference, err.Error())

when I tried to run the e2e test TestLargerResultsSidecarLogs or examples test ie. examples/v1/pipelineruns/pipeline-with-displayname.yaml. at branch https://github.com/JeromeJu/pipeline/tree/v1-storage

~~[ ] [v0.44.x] conversion can fail if the PipelineRun is too big with embedded-status set to both (or full) #6561~~ This affects v0.44 not newer versions

dibyom · 2023-05-02T16:41:27Z

Change the Storage Version to V1 Types #6444 is the WIP PR for storage swap + moving reconcilers to use v1
WIP: Switch storage version to v1 and use v1 in the reconciler #6608 is my attempts at fixing up some of the remaining work that @JeromeJu mentioned in that PR

dibyom · 2023-05-03T14:04:42Z

@lbernick and I did some investigating around the test failures summarized in #6608 (tl;dr unit tests pass, some integration tests fail due to the issue in #6592)

TODOs

Fix Unit tests for WIP: Switch storage version to v1 and use v1 in the reconciler #6608
Fix Integration test for WIP: Switch storage version to v1 and use v1 in the reconciler #6608 (likely blocked on Decoupling API versioning and Feature versioning for features turned on by default #6592 and Handle Loss of Deprecated Container Fields in v1 for Step LivenessProbes and etc. #6575)
Find a way forward for Decoupling API versioning and Feature versioning for features turned on by default #6592
Fix Handle Loss of Deprecated Container Fields in v1 for Step LivenessProbes and etc. #6575
Fix Versioned validation of referenced Pipelines/Tasks #6616
Fix Trusted resources works only for storage version of the API #6618

pritidesai · 2023-05-16T16:36:41Z

Pipelines WG - this is at risk, blocked by an issue #6616

chuangw6 · 2023-05-19T15:27:03Z

Pipelines WG - this is at risk, blocked by an issue #6616

Seems like it's less possible to include this in v0.48. Should we move it to next release?

chuangw6 · 2023-05-19T15:27:13Z

cc @jerop

tekton-robot assigned JeromeJu Sep 26, 2022

JeromeJu mentioned this issue Sep 28, 2022

V1 CRD release #5579

Merged

6 tasks

This was referenced Jan 3, 2023

WIP: Swap Task to storage version #5938

Closed

Handle v1beta1.taskObject interface and v1beta1.pipelineObject interface for V1 as the storage version #5979

Closed

Handle PipelineResources when swapping v1 to the storage version #5967

Closed

JeromeJu mentioned this issue Mar 28, 2023

Add RoundTrip to Conversion Integration Test #6439

Merged

3 tasks

pritidesai added this to the Pipelines v0.48 milestone Apr 4, 2023

JeromeJu added a commit to JeromeJu/pipeline that referenced this issue Apr 5, 2023

Make Deprecated CustomTask required Conversions Public

3292699

This commit makes the conversions required for reconciling the deprecated v1beta1 CustomTask public in the apis. part of tektoncd#5541

JeromeJu mentioned this issue Apr 5, 2023

Export required Conversions for Reconciling the Deprecated CustomRun #6496

Closed

4 tasks

JeromeJu added a commit to JeromeJu/pipeline that referenced this issue Apr 5, 2023

Export Deprecated CustomTask required Conversions

b519267

This commit makes the conversions required for reconciling the deprecated v1beta1 CustomTask exported in the apis. part of tektoncd#5541

JeromeJu added a commit to JeromeJu/pipeline that referenced this issue Apr 5, 2023

Export Deprecated CustomRun required Conversions

b330396

This commit exports the conversions required for reconciling the deprecated v1beta1 CustomRun in the apis. part of tektoncd#5541

lbernick mentioned this issue May 8, 2023

Use internal versions of Tekton CRDs in reconcilers #5504

Open

jerop modified the milestones: Pipelines v0.48, Pipelines v0.49 May 23, 2023

lbernick mentioned this issue Jun 16, 2023

Change the Storage Version to V1 Types #6444

Merged

7 tasks

tekton-robot closed this as completed in #6444 Jun 16, 2023

lbernick mentioned this issue Jun 27, 2023

Mark v1beta1 apis as deprecated #6876

Merged

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Release v1 versions of Task, TaskRun, Pipeline, and PipelineRun CRDs #5541

Release v1 versions of Task, TaskRun, Pipeline, and PipelineRun CRDs #5541

lbernick commented Sep 22, 2022

lbernick commented Sep 22, 2022

lbernick commented Sep 26, 2022

JeromeJu commented Nov 21, 2022 •

edited

lbernick commented Nov 21, 2022

JeromeJu commented Jan 10, 2023 •

edited

JeromeJu commented Mar 27, 2023 •

edited

lbernick commented Mar 27, 2023

JeromeJu commented Mar 27, 2023 •

edited

lbernick commented Mar 27, 2023

JeromeJu commented Mar 28, 2023 •

edited

JeromeJu commented Apr 4, 2023

JeromeJu commented Apr 5, 2023 •

edited

JeromeJu commented Apr 27, 2023 •

edited

dibyom commented May 2, 2023

dibyom commented May 3, 2023 •

edited

pritidesai commented May 16, 2023

chuangw6 commented May 19, 2023

chuangw6 commented May 19, 2023

Release v1 versions of Task, TaskRun, Pipeline, and PipelineRun CRDs #5541

Release v1 versions of Task, TaskRun, Pipeline, and PipelineRun CRDs #5541

Comments

lbernick commented Sep 22, 2022

lbernick commented Sep 22, 2022

lbernick commented Sep 26, 2022

JeromeJu commented Nov 21, 2022 • edited

lbernick commented Nov 21, 2022

JeromeJu commented Jan 10, 2023 • edited

JeromeJu commented Mar 27, 2023 • edited

lbernick commented Mar 27, 2023

JeromeJu commented Mar 27, 2023 • edited

lbernick commented Mar 27, 2023

JeromeJu commented Mar 28, 2023 • edited

JeromeJu commented Apr 4, 2023

JeromeJu commented Apr 5, 2023 • edited

JeromeJu commented Apr 27, 2023 • edited

dibyom commented May 2, 2023

dibyom commented May 3, 2023 • edited

TODOs

pritidesai commented May 16, 2023

chuangw6 commented May 19, 2023

chuangw6 commented May 19, 2023

JeromeJu commented Nov 21, 2022 •

edited

JeromeJu commented Jan 10, 2023 •

edited

JeromeJu commented Mar 27, 2023 •

edited

JeromeJu commented Mar 27, 2023 •

edited

JeromeJu commented Mar 28, 2023 •

edited

JeromeJu commented Apr 5, 2023 •

edited

JeromeJu commented Apr 27, 2023 •

edited

dibyom commented May 3, 2023 •

edited