Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run generate.CompleteSpec() for initContainers as well #18385

Merged

Conversation

dcermak
Copy link
Contributor

@dcermak dcermak commented Apr 28, 2023

initContainers in kubernetes deployments had no call to CompleteSpec in the generation, which means that the default environment is not configured for these. This causes issues with missing default environment variables like $HOME or $PATH.

This fixes #18384

Does this PR introduce a user-facing change?

- ensure that initContainers in k8s deployments have a correctly setup default environment

Copy link
Member

@Luap99 Luap99 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a test for it? Also see #17107 for an example, I guess you can just add it there.

pkg/domain/infra/abi/play.go Outdated Show resolved Hide resolved
@dcermak dcermak force-pushed the setup-env-in-init-containers branch from b98b6f6 to b40921a Compare May 2, 2023 09:03
@dcermak
Copy link
Contributor Author

dcermak commented May 2, 2023

I have added a regression test. I am not 100% happy with it though, as it just tests that the (by default unset) environment variable container equals podman in the initContainers' CMD. If you have ideas how to improve it, I'd be more than happy to get your input.

@dcermak dcermak requested a review from Luap99 May 2, 2023 09:06
@Luap99
Copy link
Member

Luap99 commented May 2, 2023

I have added a regression test. I am not 100% happy with it though, as it just tests that the (by default unset) environment variable container equals podman in the initContainers' CMD. If you have ideas how to improve it, I'd be more than happy to get your input.

Couldn't you just add your test directly to the one in https://github.com/containers/podman/pull/17107/files, just add a init container to the yaml there and the check the env var for the init container.

@dcermak
Copy link
Contributor Author

dcermak commented May 2, 2023

Couldn't you just add your test directly to the one in https://github.com/containers/podman/pull/17107/files, just add a init container to the yaml there and the check the env var for the init container.

That would be an option as well. I picked the e2e tests hoping that I would be able to access podman's internal data structures, but that turned out to be a misconception on my end. I can move this to the same place as #17107 if you consider that a better place.

@dcermak dcermak force-pushed the setup-env-in-init-containers branch from b40921a to ec52d02 Compare May 2, 2023 09:39
@Luap99
Copy link
Member

Luap99 commented May 2, 2023

if you consider that a better place.

I think that makes sense. General speaking they test the same thing so having them to together is better IMO.
However because you already wrote the e2e test I am totally fine with that, I don't want to put extra work on you. The important part is to have a test not where it is.

@dcermak dcermak force-pushed the setup-env-in-init-containers branch 2 times, most recently from d8330e5 to 659ae77 Compare May 2, 2023 12:42
@dcermak
Copy link
Contributor Author

dcermak commented May 2, 2023

if you consider that a better place.

I think that makes sense. General speaking they test the same thing so having them to together is better IMO. However because you already wrote the e2e test I am totally fine with that, I don't want to put extra work on you. The important part is to have a test not where it is.

I have added it into the bats tests as a fixup commit. Please pick your favorite, I'll delete the other :-)

Copy link
Member

@Luap99 Luap99 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@edsantiago @vrothberg PTAL at the system test.
code LGTM

@TomSweeneyRedHat
Copy link
Member

LGTM
would like a head nod from @edsantiago

Copy link
Collaborator

@edsantiago edsantiago left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! One comment inline re: my neuroses about testing... and, as my atonement for ranting, here's a possible patch that is short, minimally invasive, fails on podman@main, passes with your PR.

diff --git a/test/e2e/play_kube_test.go b/test/e2e/play_kube_test.go
index ca0624b13..e43c93045 100644
--- a/test/e2e/play_kube_test.go
+++ b/test/e2e/play_kube_test.go
@@ -2210,7 +2210,7 @@ var _ = Describe("Podman play kube", func() {
 	// With annotation set to always
 	It("podman play kube test with init containers and annotation set", func() {
 		// With the init container type annotation set to always
-		pod := getPod(withAnnotation("io.podman.annotations.init.container.type", "always"), withPodInitCtr(getCtr(withImage(ALPINE), withCmd([]string{"echo", "hello"}), withInitCtr(), withName("init-test"))), withCtr(getCtr(withImage(ALPINE), withCmd([]string{"top"}))))
+		pod := getPod(withAnnotation("io.podman.annotations.init.container.type", "always"), withPodInitCtr(getCtr(withImage(ALPINE), withCmd([]string{"printenv", "container"}), withInitCtr(), withName("init-test"))), withCtr(getCtr(withImage(ALPINE), withCmd([]string{"top"}))))
 		err := generateKubeYaml("pod", pod, kubeYaml)
 		Expect(err).ToNot(HaveOccurred())
 
@@ -2233,6 +2233,12 @@ var _ = Describe("Podman play kube", func() {
 		inspect.WaitWithDefaultTimeout()
 		Expect(inspect).Should(Exit(0))
 		Expect(inspect.OutputToString()).To(ContainSubstring("running"))
+
+		// Init containers need environment too! #18385
+		logs := podmanTest.Podman([]string{"logs", "testPod-init-test"})
+		logs.WaitWithDefaultTimeout()
+		Expect(logs).Should(Exit(0))
+		Expect(logs.OutputToString()).To(Equal("podman"))
 	})
 
 	// If you have an init container in the pod yaml, podman should create and run the init container with play kube

This is a suggestion only, there are many ways to do something similar.

Now, two favors please:

  1. squash commits, for a nice clean history, and
  2. rebase, because there have been many drastic changes to our e2e tests since your branch point.

Thank you!

// as well. If it is not, then initCtr will fail
initCtr := getCtr(
withImage(ALPINE),
withCmd([]string{"sh", "-c", "\"[ $container = 'podman' ]\""}),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As the poor schmoe who has to look at test failures, I reeeeeeeeally hate this kind of test. Here's why:

Sample failure:
     test failed! Exit status was nonzero!

...and, for comparison, here is what makes me wiggle my toes with joy:

    test failed: '$container' variable in container is undefined

or

    test failed: `$container` variable in container is 'something-surprising' (expected it to be 'podman')

That latter one is (probably) impossible IRL, so, yes, the test as it is will almost certainly show a failure to set envariable... but if you make a general practice of show-the-value type of testing, every so often you will get a surprise failure and the test logs will show exactly what happened and you will have a fix ready in minutes instead of hours.

And, I write this in the spirit of appreciation (you fixed a bug! you took the time to write a test! Then you wrote MORE tests!). So, thank you!

initContainers in kubernetes deployments had no call to CompleteSpec in the
generation, which means that the default environment is not configured for
these. This causes issues with missing default environment variables like $HOME
or $PÄTH.

Also, switch to using logrus.Warn() instead of fmt.Fprintf(os.Stderr)

This fixes containers#18384

Co-authored-by: Ed Santiago <santiago@redhat.com>
Signed-off-by: Dan Čermák <dcermak@suse.com>
@dcermak dcermak force-pushed the setup-env-in-init-containers branch from 659ae77 to 75d92f4 Compare May 4, 2023 06:22
@dcermak
Copy link
Contributor Author

dcermak commented May 4, 2023

@edsantiago Thanks for the extensive review! I have applied your testing patch as it tests the important part and is far smaller than my own tests (I removed these as well, as there's imho no point in testing the same thing three ways, but I can certainly add them back if you prefer that)

@dcermak dcermak requested a review from edsantiago May 4, 2023 06:33
Copy link
Collaborator

@edsantiago edsantiago left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks again!

@openshift-ci
Copy link
Contributor

openshift-ci bot commented May 4, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dcermak, edsantiago

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 4, 2023
@Luap99
Copy link
Member

Luap99 commented May 4, 2023

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label May 4, 2023
@openshift-merge-robot openshift-merge-robot merged commit d1a696a into containers:main May 4, 2023
87 checks passed
@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Aug 25, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 25, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. release-note
Projects
None yet
Development

Successfully merging this pull request may close these issues.

podman play kube fails to launch initContainers without a PATH in the environment
5 participants