Add default pod template support #1541

eddycharly · 2019-11-08T12:19:08Z

This PR adds support for a default pod template through the default ConfigMap.

This is useful when one wants to apply a pod template to all task and pipeline runs, for example specifying a node selector, a security policy, etc...

When a pod template is specified for a PipelineRun or TaskRun it takes precedence over the default pod template.
Therefore, the default pod template only applies to PipelineRun or TaskRun that don't have a pod template.

Submitter Checklist

Includes tests (if functionality changed/added)
Includes docs (if user facing)
Commit messages follow commit message best practices

tekton-robot · 2019-11-08T12:19:23Z

Hi @eddycharly. Thanks for your PR.

I'm waiting for a tektoncd member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

vdemeester · 2019-11-08T13:27:22Z

Yet to be defined : what behavior should we implement when a pod template and a default pod template are defined ? Merge both ? Pod template overrides completely the default one ? How can we bypass the default pod template if wanted (explicitly specifying null clears out the default pod template) ?

We should stick to what we do for the rest, aka, a Task step containerspec takes over the same field the stepTemplate field. In that case, a user using podTemplate in his TaskRun would take over the field from the "default" config podTemplate. And as for stepTemplate we would merge the fields.

eddycharly · 2019-11-08T13:53:23Z

@vdemeester thanks for the feedback. i agree that sticking to what is already done is good for consistency.

Trying to build right now, my changes introduce a cyclic dependency as config needs v1alpha1 (for PodTemplate struct), and v1alpha1 needs config (in pipelinerun_defaults.go for example).

How would you recommend solving this dependency issue ?

Should i create an interface ? A package to hold structs like PodTemplate ?

vdemeester · 2019-11-08T14:01:09Z

Trying to build right now, my changes introduce a cyclic dependency as config needs v1alpha1 (for PodTemplate struct), and v1alpha1 needs config (in pipelinerun_defaults.go for example).

How would you recommend solving this dependency issue ?

Should i create an interface ? A package to hold structs like PodTemplate ?

I think we have a little bit time to think about that. I would want to wait for other feedback on this feature before tackling this issue.

That said, on top of my head, we can extract the sturct into a package and use type aliasing to not have the cyclic dependency. I might take this approach for #1526 (more or less).

eddycharly · 2019-11-08T14:52:50Z

Just trying to figure out what would have to be done to implement it... in case it is accepted ;)

Still waiting for feedback/discussion of course.

ghost · 2019-12-18T17:45:03Z

what behavior should we implement when a pod template and a default pod template are defined ? Merge both ?

I would suggest not trying to perform a merge. My preference would be that the pod gets either the one specified on the TaskRun or the default if not specified, not an amalgamation of both.

eddycharly · 2020-01-08T20:09:29Z

@sbwsg @vdemeester should i go ahead with this or do you think it's useless ?

vdemeester · 2020-01-09T08:25:56Z

@eddycharly I think it make sense 👼 and I think, as a start I do like @sbwsg suggestion. If there is no PodTemplate on a TaskRun, let's use the one from the default (if any), and if there is a PodTemplate, it overrides them all 👼

vdemeester · 2020-01-09T08:26:21Z

/ok-to-test

tekton-robot · 2020-01-09T13:49:47Z

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/config/default.go	81.8%	62.5%	-19.3

eddycharly · 2020-01-09T13:56:08Z

@vdemeester to resolve the cyclic reference issue i had with pod.go, i moved it in pkg/apis/pipeline instead of pkg/apis/pipeline/v1alpha1.
Do you have a recommandation for the package where it should be or pipeline is fine ?

tekton-robot · 2020-01-09T14:27:50Z

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/config/default.go	81.8%	62.5%	-19.3

vdemeester · 2020-01-09T15:43:38Z

@vdemeester to resolve the cyclic reference issue i had with pod.go, i moved it in pkg/apis/pipeline instead of pkg/apis/pipeline/v1alpha1.
Do you have a recommandation for the package where it should be or pipeline is fine ?

hum 🤔 why is there a cyclic reference issue ?

eddycharly · 2020-01-09T17:16:32Z

It introduces a dependency from config to v1alpha1 (to deserialize the pod template struct from the config map), but v1alpha1 already depends on config...

vdemeester

It introduces a dependency from config to v1alpha1 (to deserialize the pod template struct from the config map), but v1alpha1 already depends on config...

ah dang… then we need to at least use type aliasing to make sure we don't break downstream go user of tektoncd/pipeline (like dashboard, cli, …).

I would prefer a package like pod for moving the pod.go code, so something like pkg/apis/pipeline/pod/ with a template.go file in there ?

And in v1alpha1/pod.go having,

// […]

package v1alpha1

import (
	// […]
	"github.com/tektoncd/pipeline/pkg/apis/pipeline/pod"
	// […]
)

// […]

type PodTemplate = pod.PodTemplate

wdyt ?

tekton-robot · 2020-01-09T18:28:50Z

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/config/default.go	81.8%	62.5%	-19.3

pkg/apis/pipeline/pod/template.go

tekton-robot · 2020-01-15T12:10:32Z

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/config/default.go	81.8%	100.0%	18.2
test/builder/pipeline.go	87.9%	87.0%	-0.9
test/builder/task.go	82.3%	81.1%	-1.1

fixed review comments use ghodss/yaml go mod tidy

tekton-robot · 2020-01-16T17:31:35Z

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/config/default.go	81.8%	90.0%	8.2
test/builder/pipeline.go	87.9%	87.0%	-0.9
test/builder/task.go	82.3%	81.1%	-1.1

eddycharly · 2020-01-17T17:08:26Z

Outch.. conficts :(
Can i resolve conflicts and someone merge the PR please ? @vdemeester @bobcatfish

bobcatfish · 2020-01-17T19:17:38Z

Thanks for keeping at this @eddycharly !

To resolve the conflicts you'll need to rebase your branch onto the latest version of master and then force push, then we can lgtm and merge. Let us know if you want us to point you in the right direction to do that!

tekton-robot · 2020-01-17T21:12:10Z

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/config/default.go	84.6%	90.9%	6.3
test/builder/pipeline.go	88.2%	87.3%	-0.9
test/builder/task.go	82.3%	81.1%	-1.1

tekton-robot · 2020-01-17T21:24:29Z

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/config/default.go	84.6%	90.9%	6.3
test/builder/pipeline.go	88.2%	87.3%	-0.9
test/builder/task.go	82.3%	81.1%	-1.1

eddycharly · 2020-01-17T21:30:20Z

/test pull-tekton-pipeline-integration-tests

eddycharly · 2020-01-17T21:35:41Z

@bobcatfish i resolved the conflicts.

bobcatfish · 2020-01-17T21:39:26Z

Something went wrong: failed to prepare test environment: --provider=gke boskos failed to acquire project: resources not found

This is happening over in #1888 as well, re-opened tektoncd/plumbing#29

eddycharly · 2020-01-17T21:39:43Z

boskos seems to be out of resources :(

eddycharly · 2020-01-17T21:41:51Z

/test pull-tekton-pipeline-integration-tests

bobcatfish · 2020-01-17T21:49:23Z

Sorry about this @eddycharly trying to figure out what's going on 😭

eddycharly · 2020-01-17T21:53:50Z

Thanks for your support @bobcatfish !
Good luck, i will retry later, better let boskos alone for now ;)

eddycharly · 2020-01-17T22:52:43Z

/test pull-tekton-pipeline-integration-tests

bobcatfish · 2020-01-18T02:10:28Z

/test pull-tekton-pipeline-integration-tests

eddycharly · 2020-01-18T07:35:21Z

/test pull-tekton-pipeline-integration-tests

eddycharly · 2020-01-18T18:46:26Z

/test pull-tekton-pipeline-integration-tests

eddycharly · 2020-01-20T08:06:10Z

@bobcatfish all tests are passing now. Can you give a look please ?

vdemeester · 2020-01-20T10:27:03Z

@eddycharly can you squash your commits ?

fix taskrun tests

tekton-robot · 2020-01-20T10:54:45Z

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/config/default.go	84.6%	90.9%	6.3

eddycharly · 2020-01-20T11:02:40Z

/test tekton-pipeline-unit-tests

tekton-robot · 2020-01-20T11:08:37Z

@eddycharly: The following tests failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
pull-tekton-pipeline-unit-tests	`38a0894`	link	`/test tekton-pipeline-unit-tests`
pull-tekton-pipeline-build-tests	`38a0894`	link	`/test pull-tekton-pipeline-build-tests`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

eddycharly · 2020-01-20T11:09:55Z

@vdemeester any idea why unit tests are failing ?
I0120 11:06:38.619] 2020-01-20T11:06:37.193Z ERROR TestReconcile/success-with-cluster-task taskrun/controller.go:59 Failed to create taskrun metrics recorder running_taskruns_count: cannot register view "running_taskruns_count"; a different view with the same name is already registered

vdemeester · 2020-01-20T12:20:55Z

@eddycharly interesting… I thought we fixed that test… There might be a slight race in there still… 🤔

tekton-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 8, 2019

tekton-robot requested review from dibyom and vdemeester November 8, 2019 12:19

googlebot added the cla: yes Trying to make the CLA bot happy with ppl from different companies work on one commit label Nov 8, 2019

tekton-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Nov 8, 2019

tekton-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jan 9, 2020

vdemeester requested changes Jan 9, 2020

View reviewed changes

tekton-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jan 9, 2020

eddycharly commented Jan 15, 2020

View reviewed changes

pkg/apis/pipeline/pod/template.go Outdated Show resolved Hide resolved

add default-pod-template support

c55909e

fixed review comments use ghodss/yaml go mod tidy

Merge branch 'master' into default-pod-template

0f2ffe3

bobcatfish mentioned this pull request Jan 18, 2020

Boskos seems to be wedged tektoncd/plumbing#186

Closed

rebase on master

38a0894

fix taskrun tests

eddycharly closed this Jan 20, 2020

Add default pod template support #1541

Add default pod template support #1541

Conversation

eddycharly commented Nov 8, 2019 • edited Loading

Submitter Checklist

tekton-robot commented Nov 8, 2019

vdemeester commented Nov 8, 2019

eddycharly commented Nov 8, 2019 • edited Loading

vdemeester commented Nov 8, 2019

eddycharly commented Nov 8, 2019

ghost commented Dec 18, 2019 • edited by ghost Loading

eddycharly commented Jan 8, 2020

vdemeester commented Jan 9, 2020

vdemeester commented Jan 9, 2020

tekton-robot commented Jan 9, 2020

eddycharly commented Jan 9, 2020

tekton-robot commented Jan 9, 2020

vdemeester commented Jan 9, 2020

eddycharly commented Jan 9, 2020

vdemeester left a comment

Choose a reason for hiding this comment

tekton-robot commented Jan 9, 2020

tekton-robot commented Jan 15, 2020

tekton-robot commented Jan 16, 2020

eddycharly commented Jan 17, 2020

bobcatfish commented Jan 17, 2020

tekton-robot commented Jan 17, 2020

tekton-robot commented Jan 17, 2020

eddycharly commented Jan 17, 2020

eddycharly commented Jan 17, 2020

bobcatfish commented Jan 17, 2020

eddycharly commented Jan 17, 2020

eddycharly commented Jan 17, 2020

bobcatfish commented Jan 17, 2020

eddycharly commented Jan 17, 2020

eddycharly commented Jan 17, 2020

bobcatfish commented Jan 18, 2020

eddycharly commented Jan 18, 2020

eddycharly commented Jan 18, 2020

eddycharly commented Jan 20, 2020

vdemeester commented Jan 20, 2020

tekton-robot commented Jan 20, 2020

eddycharly commented Jan 20, 2020

tekton-robot commented Jan 20, 2020

eddycharly commented Jan 20, 2020

vdemeester commented Jan 20, 2020

eddycharly commented Nov 8, 2019 •

edited

Loading

eddycharly commented Nov 8, 2019 •

edited

Loading

ghost commented Dec 18, 2019 •

edited by ghost

Loading