Add container template configuration to Task #767

abayer · 2019-04-17T16:46:58Z

Changes

The container template defined on the Task will be propagated to all
steps within the Task, and can be overridden on individual steps.

fixes #254

Submitter Checklist

These are the criteria that every PR should meet, please check them off as you
review them:

Includes tests (if functionality changed/added)
Includes docs (if user facing)
Commit messages follow commit message best practices

See the contribution guide
for more details.

Release Notes

Adds `containerTemplate` to `TaskSpec`, allowing for configuration of a `Container` to be used as the basis for all steps within the `Task`.

abayer · 2019-04-17T16:47:18Z

/assign @dwnusbaum
/assign @vdemeester
/assign @bobcatfish

vdemeester · 2019-04-17T16:53:46Z

@abayer linking @bobcatfish comment here too #714 (comment)

Consider having some other type that holds "templating" information about how a pod should be created?

This applies too 👼

abayer · 2019-04-17T16:57:29Z

@vdemeester Yeah, this was just me resurrecting my old PR that did Pipeline and TaskRun too, removing those bits, and opening a new PR - quite happy to iterate. =)

abayer · 2019-04-17T17:02:18Z

@bobcatfish @vdemeester re that - I'm thinking of maybe adding a BaseContainer field to TaskSpec, with the field just being a *corev1.Container, which is then used as an initial container that steps are merged onto. I've got similar logic in Jenkins X that's working great for us.

bobcatfish

PR is looking great!! 🎉 Just a few minor comments

@bobcatfish @vdemeester re that - I'm thinking of maybe adding a BaseContainer field to TaskSpec, with the field just being a *corev1.Container, which is then used as an initial container that steps are merged onto. I've got similar logic in Jenkins X that's working great for us.

I'm leaning toward this solution as well but I want to get some more opinions before we go ahead with it maybe @dlorenc @imjasonh @vdemeester I'm interested in what you think about this.

@abayer do you have other corev1.Container fields specifically that you'd find it handy to be able to declare at the Task level?

And I'm happy to either:

Put this PR on hold while we discuss adding a container spec OR
Merge this PR and iterate (would mean adding backwards incompatible changes if we decide to add the container spec tho since we'd remove env in that case)

bobcatfish · 2019-04-17T18:31:37Z

docs/tasks.md

+      - name: "FOO"
+        value: "baz"
+```
+


wooooot great docs 🙇‍♀️ :D

Albeit ones I now have to rewrite...

and the new ones are great too!!

bobcatfish · 2019-04-17T18:32:34Z

pkg/reconciler/v1alpha1/taskrun/taskrun.go

@@ -524,6 +533,38 @@ func createRedirectedTaskSpec(kubeclient kubernetes.Interface, ts *v1alpha1.Task
 	return ts, nil
 }

+// combineEnvVars takes a base slice of environment variables and a slice of environment
+// variables to overlay on top of the base slice, returning the result.
+func combineEnvVars(base []corev1.EnvVar, override []corev1.EnvVar) []corev1.EnvVar {


looks like a great interface! i think this is a good candidate for moving into its own package with its own unit tests 😜 😄

Welp, I switched to a new function. Which I will move etc. =)

bobcatfish · 2019-04-17T18:32:50Z

pkg/reconciler/v1alpha1/taskrun/taskrun.go

+		envVars = append(envVars, k)
+	}
+
+	// Avoid nondeterministic results by sorting the keys and appending vars in that order.


bobcatfish · 2019-04-17T18:34:55Z

pkg/reconciler/v1alpha1/taskrun/taskrun.go

+			step.Env = combineEnvVars(ts.Env, step.Env)
+		}
+	}
+


I think this code is solid, but I'm not sure createRedirectedTaskSpec is the place to put it because I think that createRedirectedTaskSpec is specifically about adding our magic entrypoint override business, and manipulating the environment variables doesn't seem like part of that to me - maybe we could move this up into createPod? (open to other ideas too!)

Which is exactly what I did in the latest version. =)

bobcatfish · 2019-04-17T18:35:56Z

pkg/reconciler/v1alpha1/taskrun/taskrun_test.go

@@ -82,6 +82,13 @@ var (

 	saTask = tb.Task("test-with-sa", "foo", tb.TaskSpec(tb.Step("sa-step", "foo", tb.Command("/mycmd"))))

+	taskEnvTask = tb.Task("test-task-env", "foo", tb.TaskSpec(
+		tb.TaskEnvVar("FRUIT", "APPLE"),


maybe include one we don't override as well?

Latest iteration shows this working right - it shows up in the init containers and the nop container.

abayer · 2019-04-17T19:35:51Z

/hold

@bobcatfish @vdemeester Modified to add baseContainer and (for #714's use case) podSpec to TaskSpec. I need to add the logic and tests for podSpec, which I'll do tomorrow - I'm heading out shortly. =)

bobcatfish

Looking good (except the docs of course which I can totally understand your hesitance to update XD)! And I think we should tackle the pod spec separately with maybe a bit more deliberation.

I'd like to hear from a couple of @vdemeester @imjasonh @dlorenc on their thoughts on this before we go ahead with the container spec but I'd say 👍 from me.

bobcatfish · 2019-04-17T19:53:02Z

pkg/apis/pipeline/v1alpha1/task_types.go

+
+	// PodSpec can be used to specify advanced configuration for the pod used
+	// by the Task.
+	PodSpec *corev1.PodSpec `json:"podSpec,omitempty"`


id prefer we start with just the BaseContainer for now and do the pod spec separately - specifically b/c if we have the pod spec we'll probably want to remove some attributes (e.g. affinity etc.) at the same time (and it seems like we want the pod spec to be a runtime thing... AND i wonder if we actually want it in the run, or referred to by the run... AND @chmouel has some interesting thoughts on that #714 (comment) long story short i think we should do a bit of design re. the pod spec before going for it

Sounds good!

bobcatfish · 2019-04-17T19:58:47Z

pkg/apis/pipeline/v1alpha1/task_types.go

+
+	// BaseContainer can be used as the basis for all step containers within the
+	// Task, so that the steps inherit settings on the base container.
+	BaseContainer *corev1.Container `json:"baseContainer,omitempty"`


@vdemeester pointed out the similarity between what we're doing here and deployment.template and job.template so I think it could be good to try to stick to a similar naming scheme, maybe something like containerTemplate or stepTemplate?

Sure thing.

bobcatfish · 2019-04-17T20:00:12Z

pkg/reconciler/v1alpha1/taskrun/resources/pod.go

@@ -229,7 +244,29 @@ func MakePod(taskRun *v1alpha1.TaskRun, taskSpec v1alpha1.TaskSpec, kubeclient k

 	nopContainer := &corev1.Container{Name: "nop", Image: *nopImage, Command: []string{"/ko-app/nop"}}
 	entrypoint.RedirectStep(cache, len(podContainers), nopContainer, kubeclient, taskRun, logger)
-	podContainers = append(podContainers, *nopContainer)
+	mergedStep, err := mergeStepWithBaseContainer(baseContainer, nopContainer)


hm, instead of calling mergeStepWithBaseContainer several times, is there a way we can call it with all the steps together once we have them? (maybe on podContainers below?)

Yeah, can do

bobcatfish · 2019-04-17T20:03:32Z

pkg/reconciler/v1alpha1/taskrun/resources/pod.go

+		return nil, err
+	}
+
+	mergedData, err := strategicpatch.StrategicMergePatchUsingLookupPatchMeta(baseData, patch, schema)


would you mind adding some comments before each of the strategicpatch calls elaborating on what's going on (from my ignorant-of-strategic-merge point of view it seems weird that there are 3 different calls required here and im not sure about details like "why do we need 'using lookup patch meta'"

...yeah, I’ll be honest, it’s a bit black magic to me - I just kept hitting things until I got the result I needed. =) But I got there right before wrapping up for the day, so I’ll add explanatory comments tomorrow.

bobcatfish · 2019-04-17T20:04:19Z

pkg/reconciler/v1alpha1/taskrun/resources/pod.go

 	}, nil
 }

+
+func mergeStepWithBaseContainer(base, override *corev1.Container) (*corev1.Container, error) {


I think we could have this in it's own package with unit tests :D

(No worries if you want to wait to till we decide if we want to go forward with this impl vs the previous one)

While I'm thinking of it, a question - what's the motivation for putting functions like this in their own packages, rather than just unit testing them in place? Just trying to understand. =)

abayer · 2019-04-18T13:50:08Z

/hold cancel

Ok, I think we're good now, so I'm removing the hold.

vdemeester

Few comments — the merging piece is interesting… is it how deployment & co do it ?

vdemeester · 2019-04-18T14:06:15Z

pkg/reconciler/v1alpha1/taskrun/resources/pod.go

@@ -23,6 +23,7 @@ import (
 	"encoding/hex"
 	"flag"
 	"fmt"
+	"github.com/tektoncd/pipeline/pkg/merge"


nit: misplaced import 😅

stupid IDE!

vdemeester · 2019-04-18T14:07:58Z

pkg/reconciler/v1alpha1/taskrun/resources/pod.go

@@ -170,17 +171,19 @@ func MakePod(taskRun *v1alpha1.TaskRun, taskSpec v1alpha1.TaskSpec, kubeclient k
 	}
 	annotations["sidecar.istio.io/inject"] = "false"

+	baseContainer := taskSpec.ContainerTemplate


nit: s/baseContainer/containerTemplate/g ?

abayer · 2019-04-18T14:10:33Z

@vdemeester This is basically how kubectl does it - that's what I derived this from.

Container configuration defined on the `Task` will be propagated to all steps within the `Task`, and can be overridden on individual steps.

Derived from what I've done over at tektoncd/pipeline#767 - this is just cleaner, simpler, better.

bobcatfish

Looks awesome @abayer ! For once in my life I have nothing to nit, it all looks great :D

p.s. I also spoke briefly with @dlorenc and @imjasonh offline and they're on board with this approach.

Thanks again for this (so quickly!) and also thanks for your patience re. my original reluctance to add this feature 🙇‍♀️

/approve
/lgtm
/meow space

bobcatfish · 2019-04-19T16:46:26Z

docs/tasks.md

@@ -72,7 +73,9 @@ following fields:
  - [`outputs`](#outputs) - Specifies [`PipelineResources`](resources.md) needed
    by your `Task`
  - [`volumes`](#volumes) - Specifies one or more volumes that you want to make
-    available to your build.
+    available to your `Task`'s steps.


whoops, thanks for fixing! ❤️

bobcatfish · 2019-04-19T16:46:44Z

docs/tasks.md

+      - name: "FOO"
+        value: "baz"
+```
+


and the new ones are great too!!

bobcatfish · 2019-04-19T16:47:59Z

pkg/merge/merge.go

+			return nil, err
+		}
+
+		// Actually apply the merge patch to the template JSON.


thanks so much for these detailed comments, I think it really clears up what's going on! 🙏 🎉

tekton-robot · 2019-04-19T16:51:30Z

@bobcatfish:

In response to this:

Looks awesome @abayer ! For once in my life I have nothing to nit, it all looks great :D

p.s. I also spoke briefly with @dlorenc and @imjasonh offline and they're on board with this approach.

Thanks again for this (so quickly!) and also thanks for your patience re. my original reluctance to add this feature 🙇‍♀️

/approve
/lgtm
/meow space

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

tekton-robot · 2019-04-19T16:51:34Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: abayer, bobcatfish

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [bobcatfish]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

abayer · 2019-04-19T17:48:11Z

/test pull-tekton-pipeline-integration-tests

@abayer

@abayer has contributed features such as step templating (tektoncd#767) and Pipeline level params (tektoncd#471), as well as many general tweaks and fixes, and 40+ reviews! Not to mention that since he works on Jenkins X (built on Tekton) he has invaluable feedback from the perspective of someone actively building on top of Tekton! Let it be noted forever that @bobcatfish should have gotten around to doing this ages ago and she very much appreciates @abayer's patience 🙏

@abayer

@abayer has contributed features such as step templating (#767) and Pipeline level params (#471), as well as many general tweaks and fixes, and 40+ reviews! Not to mention that since he works on Jenkins X (built on Tekton) he has invaluable feedback from the perspective of someone actively building on top of Tekton! Let it be noted forever that @bobcatfish should have gotten around to doing this ages ago and she very much appreciates @abayer's patience 🙏

tekton-robot requested review from imjasonh and vdemeester April 17, 2019 16:47

googlebot added the cla: yes Trying to make the CLA bot happy with ppl from different companies work on one commit label Apr 17, 2019

tekton-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Apr 17, 2019

tekton-robot assigned bobcatfish, dwnusbaum and vdemeester Apr 17, 2019

abayer force-pushed the task-env-vars branch from 1cc05a0 to bc9c0e1 Compare April 17, 2019 17:02

bobcatfish reviewed Apr 17, 2019

View reviewed changes

abayer force-pushed the task-env-vars branch from bc9c0e1 to 3a928f2 Compare April 17, 2019 19:34

tekton-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 17, 2019

abayer force-pushed the task-env-vars branch 4 times, most recently from 3e1f910 to 2389eb0 Compare April 17, 2019 19:46

abayer changed the title ~~Add Task-level environment variables~~ Add container and podspec base configuration to Task Apr 17, 2019

bobcatfish reviewed Apr 17, 2019

View reviewed changes

abayer force-pushed the task-env-vars branch from 2389eb0 to 6d80525 Compare April 18, 2019 12:38

abayer changed the title ~~Add container and podspec base configuration to Task~~ Add container template configuration to Task Apr 18, 2019

abayer force-pushed the task-env-vars branch from 6d80525 to 279153a Compare April 18, 2019 13:49

tekton-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 18, 2019

abayer force-pushed the task-env-vars branch 2 times, most recently from 04b8078 to d433124 Compare April 18, 2019 13:57

tekton-robot removed the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Apr 18, 2019

tekton-robot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Apr 18, 2019

vdemeester reviewed Apr 18, 2019

View reviewed changes

Add container template configuration for Task

e963c1d

Container configuration defined on the `Task` will be propagated to all steps within the `Task`, and can be overridden on individual steps.

abayer force-pushed the task-env-vars branch from d433124 to e963c1d Compare April 18, 2019 14:11

abayer added a commit to abayer/jx that referenced this pull request Apr 18, 2019

chore: Simplify merging of container options and environment

cbe9705

Derived from what I've done over at tektoncd/pipeline#767 - this is just cleaner, simpler, better.

abayer mentioned this pull request Apr 18, 2019

fix: Simplify merging and fix override of top level env vars jenkins-x/jx#3679

Merged

2 tasks

abayer added a commit to abayer/jx that referenced this pull request Apr 18, 2019

fix: Simplify merging and fix override of top level env vars

a05367e

Derived from what I've done over at tektoncd/pipeline#767 - this is just cleaner, simpler, better.

abayer added a commit to abayer/jx that referenced this pull request Apr 18, 2019

fix: Simplify merging and fix override of top level env vars

3406b37

Derived from what I've done over at tektoncd/pipeline#767 - this is just cleaner, simpler, better.

jenkins-x-bot pushed a commit to jenkins-x/jx that referenced this pull request Apr 18, 2019

fix: Simplify merging and fix override of top level env vars (#3679)

5e9ee91

Derived from what I've done over at tektoncd/pipeline#767 - this is just cleaner, simpler, better.

bobcatfish reviewed Apr 19, 2019

View reviewed changes

tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Apr 19, 2019

tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 19, 2019

tekton-robot merged commit 4a84deb into tektoncd:master Apr 19, 2019

vdemeester mentioned this pull request Apr 23, 2019

Add SecurityContext to TaskSpec #714

Closed

3 tasks

rokii mentioned this pull request Jul 10, 2019

Support customization of injected containers for taskrun #1051

Closed

bobcatfish mentioned this pull request Jul 22, 2019

Add @abayer as an OWNER 🐈 #1108

Merged

1 task

Add container template configuration to Task #767

Add container template configuration to Task #767

Conversation

abayer commented Apr 17, 2019 • edited Loading

Changes

Submitter Checklist

Release Notes

abayer commented Apr 17, 2019

vdemeester commented Apr 17, 2019

abayer commented Apr 17, 2019

abayer commented Apr 17, 2019

bobcatfish left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abayer commented Apr 17, 2019

bobcatfish left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abayer commented Apr 18, 2019

vdemeester left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abayer commented Apr 18, 2019

bobcatfish left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tekton-robot commented Apr 19, 2019

tekton-robot commented Apr 19, 2019

abayer commented Apr 19, 2019

abayer commented Apr 17, 2019 •

edited

Loading