ConfigMap Build Sources #19655

adambkaplan · 2018-05-08T20:40:55Z

Allow ConfigMaps to be a build source.

Trello Card

RFE 1540978

adambkaplan · 2018-05-09T18:16:14Z

(5) Allow using a ConfigMap as an input to a build [builds]

adambkaplan · 2018-05-09T19:42:46Z

/retest

adambkaplan · 2018-05-10T13:32:56Z

/retest

adambkaplan · 2018-05-11T14:38:02Z

/assign @bparees

bparees · 2018-05-11T15:13:21Z

pkg/build/builder/docker.go

-				}
-				return nil
-			}
+func (d *DockerBuilder) copyLocalObject(s localObjectBuildSource, buildDir, mountPath string) error {


s/buildDir/targetDir/
s/mountPath/sourceDir/

(and i'd probably reorder those args to be (src,dst) instead of (dst,src))

bparees · 2018-05-11T17:35:20Z

pkg/build/controller/strategy/util.go

@@ -273,6 +321,14 @@ func setupAdditionalSecrets(pod *v1.Pod, container *v1.Container, secrets []buil
 	}
 }

+// setupAdditionalConfigMaps creates volume mounts in the given pod for the given list of configMaps
+func setupAdditionalConfigMaps(pod *v1.Pod, container *v1.Container, configMaps []buildapi.ConfigMapSpec) {


what uses this?

this is a mirror of the secrets in the Custom Build Strategy. TBH I don't fully understand the use case of this feature, since secrets are supported in the Source part of the BuildConfig

I don't see anything calling it, so let's remove both of them. (the configmap one and the secret one)

oh sorry, i see where it's called. looking.

I think the custom build strategy secrets field predated our generic "input secrets" support.

I think we should not introduce a configmap equivalent for custom builds, just let people use the "input configmaps" feature to inject configmap content into their custom builds.

bparees · 2018-05-11T17:38:29Z

test/extended/testdata/builds/build-secrets/s2i-binary-dir/.s2i/bin/assemble

+mkdir -p "${HOME}/testconfig"
+if [[ -f /tmp/configmap/foo ]]; then
+    # Copy three secrets defined in configmap1 fixture to directory


s/secrets/configmap keys/ ?

/s/keys/entries/ may be better (map key is the file name, map value is the file content)

sure. mostly just trying to not call them secrets since they aren't.

bparees · 2018-05-11T17:42:02Z

test/extended/testdata/builds/build-secrets/s2i-binary-dir/.s2i/bin/run

@@ -8,8 +8,15 @@ for s in /tmp/secret? secret?; do
    fi
 done

+for c in /tmp/configmap/* configmap2/*; do
+    if [[ ! -s "${c}" ]]; then


this makes me realize the check above for secrets is insufficient. it needs to check that the file exists and has a size zero. Currently the check above will pass if the file does not exist at all.

bparees · 2018-05-11T17:44:54Z

test/extended/testdata/builds/build-secrets/s2i-binary-dir/.s2i/bin/run

@@ -8,8 +8,15 @@ for s in /tmp/secret? secret?; do
    fi
 done

+for c in /tmp/configmap/* configmap2/*; do
+    if [[ ! -s "${c}" ]]; then
+        echo "Found configmap file which should not have been truncated: ${s}"


"configmap file is missing or has size zero: "
(and even better, indicate which it is)

bparees · 2018-05-11T19:51:29Z

pkg/build/controller/strategy/custom.go

@@ -125,5 +125,6 @@ func (bs *CustomBuildStrategy) CreateBuildPod(build *buildapi.Build) (*v1.Pod, e
 	setupSourceSecrets(pod, &pod.Spec.Containers[0], build.Spec.Source.SourceSecret)
 	setupInputSecrets(pod, &pod.Spec.Containers[0], build.Spec.Source.Secrets)
 	setupAdditionalSecrets(pod, &pod.Spec.Containers[0], build.Spec.Strategy.CustomStrategy.Secrets)
+	setupAdditionalConfigMaps(pod, &pod.Spec.Containers[0], build.Spec.Strategy.CustomStrategy.ConfigMaps)


so doesn't this need to be doing a "setupInputConfigMaps" also?

yes (had it in my prior branch, missed carrying it over)

adambkaplan · 2018-05-11T19:58:42Z

@bparees QE brought up a test case injecting configMaps into a Jenkins pipeline build. It's a stretch, but you could define a Secret or ConfigMap with a jenkinsfile, then reference that as the jenkinsfile in the pipeline build.

Worth adding this?

bparees · 2018-05-11T20:03:32Z

@bparees QE brought up a test case injecting configMaps into a Jenkins pipeline build. It's a stretch, but you could define a Secret or ConfigMap with a jenkinsfile, then reference that as the jenkinsfile in the pipeline build.

no, not worth taking that on here. that would likely entail changes to the sync plugin anyway.

adambkaplan · 2018-05-14T20:57:33Z

@bparees ptal (api changes held while we continue iterating here)

bparees · 2018-05-14T21:06:07Z

lgtm

gabemontero · 2018-05-14T21:33:31Z

pkg/build/apis/build/types.go

+	// build.
+	ConfigMap kapi.LocalObjectReference
+
+	// DestinationDir is the directory where the files from the secret should be


"from the secret" -> "from the configMap"

adambkaplan · 2018-06-19T20:39:30Z

@bparees @deads2k rebased & modified commit history - ptal

bparees · 2018-06-19T20:45:47Z

@adambkaplan did you filter the glide updates or was it really this clean?

bparees · 2018-06-19T20:46:50Z

pkg/build/apis/build/types.go

+
+	// ConfigMaps represents a list of configMaps and their destinations that will
+	// be used only for the build.
+	ConfigMaps []ConfigMapBuildSource


not sure if @deads2k wanted the internal api changes with the update commit, or separate. I think he wanted it separate (ie only generated code in one of the commits, only manual changes in the other commit).

adambkaplan · 2018-06-20T17:40:14Z

/retest

* api: 0ce1df2db7debb15eddb25f3ae76df4180777221 * k8s.io/kubernetes: 6d1dd08538863a8d01b84012b8fdff5b2e6f2240

RFE/bug 1540978

* update deepcopy and converters * update swagger and protobuf spec RFE/bug 1540978

RFE/bug 1540978

adambkaplan · 2018-06-21T15:50:07Z

@deads2k updated commit history to isolate the new type.

adambkaplan · 2018-06-21T15:52:33Z

I've managed to confuse GitHub's history on this one. Tree in SourceTree is accurate:

bparees · 2018-06-21T16:41:55Z

/lgtm

openshift-bot · 2018-06-21T16:44:30Z

/retest

Please review the full test history for this PR and help us cut down flakes.

deads2k · 2018-06-21T16:49:47Z

API update seems fine.

Please remove the commit for oc, open a different pull, and wait for the rebase. oc is undergoing a massive refactor as part of the rebase to react to upstream changes.

bparees · 2018-06-21T17:03:07Z

/hold
(for @adambkaplan to remove the new-build commit per @deads2k's request)

adambkaplan · 2018-06-21T18:08:30Z

/hold cancel
oc changes included in #20064 per @deads2k

bparees · 2018-06-21T18:14:11Z

/lgtm

openshift-ci-robot · 2018-06-21T18:14:17Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: adambkaplan, bparees

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [bparees]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-bot · 2018-06-22T02:49:00Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-ci-robot · 2018-06-22T12:31:26Z

@adambkaplan: The following test failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
ci/openshift-jenkins/unit	`86324f6`	link	`/test unit`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

bparees · 2018-06-22T14:25:46Z

/retest

openshift-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 8, 2018

openshift-ci-robot requested review from gabemontero and smarterclayton May 8, 2018 20:40

openshift-ci-robot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label May 8, 2018

adambkaplan force-pushed the feature/configmap-build-src branch 3 times, most recently from 4b7d9b6 to 867cc26 Compare May 9, 2018 17:55

openshift-ci-robot assigned bparees May 11, 2018

bparees added this to the v3.11 milestone May 11, 2018

bparees requested changes May 11, 2018

View reviewed changes

bparees reviewed May 11, 2018

View reviewed changes

adambkaplan force-pushed the feature/configmap-build-src branch from 867cc26 to 338949e Compare May 14, 2018 18:59

gabemontero reviewed May 14, 2018

View reviewed changes

openshift-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 29, 2018

adambkaplan force-pushed the feature/configmap-build-src branch from 86324f6 to ee55868 Compare June 7, 2018 20:49

openshift-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 7, 2018

adambkaplan force-pushed the feature/configmap-build-src branch from ee55868 to 4f30ce5 Compare June 19, 2018 20:32

adambkaplan changed the title ~~[WIP] ConfigMap Build Sources~~ ConfigMap Build Sources Jun 19, 2018

openshift-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 19, 2018

bparees reviewed Jun 19, 2018

View reviewed changes

bump(*):

edd3813

* api: 0ce1df2db7debb15eddb25f3ae76df4180777221 * k8s.io/kubernetes: 6d1dd08538863a8d01b84012b8fdff5b2e6f2240

adambkaplan force-pushed the feature/configmap-build-src branch from 4f30ce5 to e0530fc Compare June 21, 2018 15:42

adambkaplan added 3 commits June 21, 2018 11:47

build: add origin ConfigMap build source types

1239d1f

RFE/bug 1540978

update-api

0d0ebdf

* update deepcopy and converters * update swagger and protobuf spec RFE/bug 1540978

build: add ConfigMap build sources

9be6877

RFE/bug 1540978

adambkaplan force-pushed the feature/configmap-build-src branch from e0530fc to d8d488c Compare June 21, 2018 15:47

openshift-ci-robot added lgtm Indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Jun 21, 2018

openshift-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 21, 2018

adambkaplan force-pushed the feature/configmap-build-src branch from d8d488c to 9be6877 Compare June 21, 2018 18:03

openshift-ci-robot removed the lgtm Indicates that a PR is ready to be merged. label Jun 21, 2018

adambkaplan mentioned this pull request Jun 21, 2018

oc: ConfigMap build sources #20064

Merged

openshift-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 21, 2018

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Jun 21, 2018

openshift-merge-robot merged commit 0b10fb1 into openshift:master Jun 22, 2018

deads2k mentioned this pull request Jun 22, 2018

DO NOT MERGE Revert "ConfigMap Build Sources" #20082

Closed

adambkaplan deleted the feature/configmap-build-src branch June 25, 2018 18:33

ConfigMap Build Sources #19655

ConfigMap Build Sources #19655

Conversation

adambkaplan commented May 8, 2018 • edited Loading

adambkaplan commented May 9, 2018

adambkaplan commented May 9, 2018

adambkaplan commented May 10, 2018

adambkaplan commented May 11, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adambkaplan May 11, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adambkaplan commented May 11, 2018 • edited Loading

bparees commented May 11, 2018

adambkaplan commented May 14, 2018

bparees commented May 14, 2018

Choose a reason for hiding this comment

adambkaplan commented Jun 19, 2018

bparees commented Jun 19, 2018

Choose a reason for hiding this comment

adambkaplan commented Jun 20, 2018

adambkaplan commented Jun 21, 2018

adambkaplan commented Jun 21, 2018

bparees commented Jun 21, 2018

openshift-bot commented Jun 21, 2018

deads2k commented Jun 21, 2018

bparees commented Jun 21, 2018

adambkaplan commented Jun 21, 2018

bparees commented Jun 21, 2018

openshift-ci-robot commented Jun 21, 2018

openshift-bot commented Jun 22, 2018

openshift-ci-robot commented Jun 22, 2018 • edited Loading

bparees commented Jun 22, 2018

adambkaplan commented May 8, 2018 •

edited

Loading

adambkaplan May 11, 2018 •

edited

Loading

adambkaplan commented May 11, 2018 •

edited

Loading

openshift-ci-robot commented Jun 22, 2018 •

edited

Loading