Use imageutils instead of hardcoded image paths #81093

johnSchnake · 2019-08-07T18:12:01Z

What type of PR is this?
/kind cleanup

What this PR does / why we need it:
A number of tests were using hardcoded image paths instead of
going through the imageutils package. The reason for centralizing
the logic there is to keep an eye on what images we use and where
they come from.

Which issue(s) this PR fixes:
Fixes #76617

Special notes for your reviewer:
Found one hardcoded variable here that was duplicating a registry that was already specified elsewhere. This would lead to 2 of the packages pulling from their own, hardcoded location instead of respecting the user overrides if provided.

In addition, there seemed to be one image being pulled from someone's ( @humblec ) private docker registry. This seems like an issue to me and should be remedied in its own PR; the point of this PR was just to centralize the image/registry lists.

@humblec maybe you can comment on why it is using that image instead of being integrated into the k8s registries?

Does this PR introduce a user-facing change?:

NONE

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

NONE

/sig testing
/priority important-soon
/area e2e-test-framework

davidz627 · 2019-08-07T18:24:52Z

/assign @wongma7
/cc @msau42

msau42 · 2019-08-07T18:27:47Z

test/utils/image/manifest.go

@@ -65,6 +67,7 @@ func initReg() RegistryList {
 	registry := RegistryList{
 		GcAuthenticatedRegistry: "gcr.io/authenticated-image-pulling",
 		DockerLibraryRegistry:   "docker.io/library",
+		DockerHumbleC:           "docker.io/humblec",


@humblec can you point us to a public registry where we can pull the gluster provisioner image from?

@msau42 @johnSchnake while developing the e2e I was trying to use gcr or some other common repo where we host of our images, but somehow I didnt get the push access. Is there a common repo path where I can push this image ? If yes, please share it, so thatI can move or copy this container to that common path and update it via PR.

@spiffxp Could you comment on the issue above?

I'm not sure really where the image comes from, but I would assume he needs to put any code to build the image into test/images then just ask someone in to bulid/push it?

/hold

we cannot be adding personal repos!

@humblec please work with @jsafrane and @childsb to push your image to the quay repo for kubernetes-incubator/external-storage projects

@humblec why not say https://hub.docker.com/u/gluster ? (in other words, why should it be in the k8s gcr repo?)

All the external-provisioners developed under kubernetes-incubator/external-storage are using quay.io/external-storage to store their images. I think that is the most obvious place to put the gluster image

@msau42 working with Jan and Brad on this , Hopefully this will be sorted out soon.

Looks like @childsb is on pto. Jan dont have access of the repo. @wongma7 is it something you can help ?

@msau42 @johnSchnake @dims Discussed with @childsb and as per him, it is better to host it under gluster org in docker hub. I have moved the image to gluster org and updated the code by #82115.

msau42 · 2019-08-07T18:28:26Z

/assign @humblec

johnSchnake · 2019-08-07T19:46:42Z

/retest

johnSchnake · 2019-08-08T14:19:43Z

I'm confused by the failure in the gce-csi run; it has failed multiple times in the same way so it seems like either the a 3rd party item isn't working, test-infra has an issue, or my change somehow caused the breakage.

However, it shows that tests pass and then the scripts fail to move logs around and then e2e-down times out.

W0807 20:40:25.792] Specify --start=40588 in the next get-serial-port-output invocation to get only the new output starting from here.
W0807 20:40:30.049] scp: /var/log/cluster-autoscaler.log*: No such file or directory
W0807 20:40:30.121] scp: /var/log/fluentd.log*: No such file or directory
W0807 20:40:30.121] scp: /var/log/kubelet.cov*: No such file or directory
W0807 20:40:30.121] scp: /var/log/startupscript.log*: No such file or directory
W0807 20:40:30.131] ERROR: (gcloud.compute.scp) [/usr/bin/scp] exited with return code [1].

If anyone has an idea/lead on this I'd appreciate it.

johnSchnake · 2019-08-08T15:01:19Z

I have another PR for something completely different with the same error (for gce-csi) so I think it is a test-infra issue of some sort. Going to do some searching in slack and such to see if this is a known issue.

johnSchnake · 2019-08-08T18:56:14Z

/test pull-kubernetes-e2e-gce-csi-serial

spiffxp · 2019-08-21T05:45:39Z

/retest
needs rebase

A number of tests were using hardcoded image paths instead of going through the imageutils package. The reason for centralizing the logic there is to keep an eye on what images we use and where they come from.

alejandrox1 · 2019-09-29T18:59:11Z

ping @dims @msau42 @johnSchnake
any updates on this?
/remove-area e2e-test-framework
/milestone v1.17

k8s-ci-robot · 2019-09-29T18:59:12Z

@alejandrox1: Those labels are not set on the issue: area/e2e-testframework

In response to this:

ping @dims @msau42 @johnSchnake
any updates on this?
/remove-area e2e-testframework
/milestone v1.17

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

johnSchnake · 2019-09-30T17:03:55Z

Just needs approved/lgtm now. Let me know if other changes are needed.

msau42 · 2019-10-01T00:41:24Z

/lgtm
/approve
/hold cancel

k8s-ci-robot · 2019-10-01T00:41:57Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: johnSchnake, msau42

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

fejta-bot · 2019-10-01T04:53:00Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

k8s-ci-robot requested review from davidz627 and freehan August 7, 2019 18:15

k8s-ci-robot added the area/test label Aug 7, 2019

k8s-ci-robot assigned wongma7 Aug 7, 2019

k8s-ci-robot requested a review from msau42 August 7, 2019 18:24

msau42 reviewed Aug 7, 2019

View reviewed changes

k8s-ci-robot assigned humblec Aug 7, 2019

johnSchnake force-pushed the useImageUtilsForLookup branch from 5c0444c to 430e092 Compare August 7, 2019 18:41

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 8, 2019

johnSchnake mentioned this pull request Aug 8, 2019

Automation needed to lock down E2E test images #81169

Closed

humblec mentioned this pull request Aug 29, 2019

Migrate the e2e provisioner container image to a different location. #82115

Merged

johnSchnake force-pushed the useImageUtilsForLookup branch from 430e092 to ce88b4a Compare September 27, 2019 18:59

k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Sep 27, 2019

johnSchnake force-pushed the useImageUtilsForLookup branch from ce88b4a to e16940d Compare September 27, 2019 19:03

k8s-ci-robot removed the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Sep 27, 2019

k8s-ci-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Sep 27, 2019

Use imageutils instead of hardcoded image paths

2dab911

A number of tests were using hardcoded image paths instead of going through the imageutils package. The reason for centralizing the logic there is to keep an eye on what images we use and where they come from.

johnSchnake force-pushed the useImageUtilsForLookup branch from e16940d to 2dab911 Compare September 27, 2019 20:04

k8s-ci-robot added this to the v1.17 milestone Sep 29, 2019

k8s-ci-robot removed the area/e2e-test-framework Issues or PRs related to refactoring the kubernetes e2e test framework label Sep 29, 2019

johnSchnake requested a review from dims September 30, 2019 17:03

k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 1, 2019

k8s-ci-robot assigned msau42 Oct 1, 2019

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 1, 2019

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 1, 2019

k8s-ci-robot merged commit c2c8215 into kubernetes:master Oct 1, 2019

wongma7 mentioned this pull request Oct 1, 2019

nfs provisioner tests failing in alpha clusters #83380

Closed

mkumatag mentioned this pull request May 26, 2020

Backport e2e Image Override Changes #91335

Closed

r4f4 mentioned this pull request Jul 9, 2020

Automated cherry pick of #81093: Use imageutils instead of hardcoded image paths #92934

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use imageutils instead of hardcoded image paths #81093

Use imageutils instead of hardcoded image paths #81093

johnSchnake commented Aug 7, 2019

davidz627 commented Aug 7, 2019

msau42 Aug 7, 2019

humblec Aug 8, 2019 •

edited

johnSchnake Aug 8, 2019

dims Aug 8, 2019

msau42 Aug 8, 2019

dims Aug 9, 2019

msau42 Aug 9, 2019

humblec Aug 12, 2019

humblec Aug 16, 2019

humblec Aug 29, 2019

msau42 commented Aug 7, 2019

johnSchnake commented Aug 7, 2019

johnSchnake commented Aug 8, 2019

johnSchnake commented Aug 8, 2019

johnSchnake commented Aug 8, 2019

spiffxp commented Aug 21, 2019

alejandrox1 commented Sep 29, 2019 •

edited

k8s-ci-robot commented Sep 29, 2019

johnSchnake commented Sep 30, 2019

msau42 commented Oct 1, 2019

k8s-ci-robot commented Oct 1, 2019

fejta-bot commented Oct 1, 2019

Use imageutils instead of hardcoded image paths #81093

Use imageutils instead of hardcoded image paths #81093

Conversation

johnSchnake commented Aug 7, 2019

davidz627 commented Aug 7, 2019

Choose a reason for hiding this comment

humblec Aug 8, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

msau42 commented Aug 7, 2019

johnSchnake commented Aug 7, 2019

johnSchnake commented Aug 8, 2019

johnSchnake commented Aug 8, 2019

johnSchnake commented Aug 8, 2019

spiffxp commented Aug 21, 2019

alejandrox1 commented Sep 29, 2019 • edited

k8s-ci-robot commented Sep 29, 2019

johnSchnake commented Sep 30, 2019

msau42 commented Oct 1, 2019

k8s-ci-robot commented Oct 1, 2019

fejta-bot commented Oct 1, 2019

humblec Aug 8, 2019 •

edited

alejandrox1 commented Sep 29, 2019 •

edited