Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OCPBUGS-25662: Add Image Credential Provider flags for Kubelet on AWS #4103

Merged

Conversation

JoelSpeed
Copy link
Contributor

- What I did
Adds configuration for the ECR credential provider to kubelet on AWS.

Unit tests show new flags being set correctly.

Depends on the ecr-credential-provider RPM shipping in RHCOS, this is currently a WIP.
/hold

- How to verify it
Kubelet should be able to pull an image from a private ECR registry.

- Description for the changelog

@openshift-ci-robot openshift-ci-robot added jira/severity-important Referenced Jira bug's severity is important for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Jan 8, 2024
@openshift-ci-robot
Copy link
Contributor

@JoelSpeed: This pull request references Jira Issue OCPBUGS-25662, which is invalid:

  • expected the bug to target the "4.16.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

- What I did
Adds configuration for the ECR credential provider to kubelet on AWS.

Unit tests show new flags being set correctly.

Depends on the ecr-credential-provider RPM shipping in RHCOS, this is currently a WIP.
/hold

- How to verify it
Kubelet should be able to pull an image from a private ECR registry.

- Description for the changelog

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 8, 2024
@JoelSpeed
Copy link
Contributor Author

/test unit

@damdo
Copy link
Member

damdo commented Jan 9, 2024

@JoelSpeed we had a chat about this in the QE sync meeting, and it looks like the CI failures here are not flakes. Do you have a theory on why this is breaking installation on many providers?

@JoelSpeed
Copy link
Contributor Author

Will need to bring up a cluster and have a look at what's going on to understand why this is breaking, will see if I can jump on that

@JoelSpeed JoelSpeed force-pushed the aws-ecr-credential-provider branch 2 times, most recently from b7bf22a to d25ba25 Compare January 10, 2024 13:34
credentialProviderConfigFlag := "--image-credential-provider-config=/etc/kubernetes/credential-providers/"
switch cfg.Infra.Status.PlatformStatus.Type {
case configv1.AWSPlatformType:
return fmt.Sprintf("%s %s%s", credentialProviderBinDirFlag, credentialProviderConfigFlag, "ecr-credential-provider.yaml")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question (non-blocking): Would it make more sense to make ecr-credential-provider.yaml part of credentialProviderConfigFlag variable? In other words:

credentialProviderConfigFlag := "--image-credential-provider-config=/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"

Then the return statement would simplify to:

case configv1.AWSPlatformType:
	return fmt.Sprintf("%s %s", credentialProviderBinDirFlag, credentialProviderConfigFlag)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the future, this switch will support more platforms.We will want those variables separate so they can be combined in different ways to get the correct flags for the different platforms, I'm expecting the config file name itself to be different per platform

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, thanks for the clarification!

@cheesesashimi
Copy link
Member

Overall, this change looks good; modulo the CI failures. I just have a single non-blocking question, but feel free to ignore it if there are more pressing matters.

@theobarberbany
Copy link
Contributor

/test e2e-aws-ovn

@JoelSpeed
Copy link
Contributor Author

/restest-required

@JoelSpeed
Copy link
Contributor Author

/jira refresh
/hold cancel

@openshift-ci openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 17, 2024
@openshift-ci-robot openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Jan 17, 2024
@openshift-ci-robot
Copy link
Contributor

@JoelSpeed: This pull request references Jira Issue OCPBUGS-25662, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.16.0) matches configured target version for branch (4.16.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @yliu127

In response to this:

/jira refresh
/hold cancel

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Copy link
Contributor

openshift-ci bot commented Jan 17, 2024

@openshift-ci-robot: GitHub didn't allow me to request PR reviews from the following users: yliu127.

Note that only openshift members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

@JoelSpeed: This pull request references Jira Issue OCPBUGS-25662, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.16.0) matches configured target version for branch (4.16.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @yliu127

In response to this:

/jira refresh
/hold cancel

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@JoelSpeed
Copy link
Contributor Author

/retest-required

@JoelSpeed
Copy link
Contributor Author

Looks like the OS image still doesn't have the binary in place yet

@sdodson
Copy link
Member

sdodson commented Jan 17, 2024

The package has made it into RHCOS 416.92.202401170945-0, just need to wait on a 4.16 CI build to bump to that version and be accepted.

@JoelSpeed
Copy link
Contributor Author

Just checked on the CI builds again, still not updated to a newer RHCOS build yet, I believe there is an issue that ART are working on

@sdodson
Copy link
Member

sdodson commented Jan 18, 2024

/retest-required
The latest CI build has the RHCOS bump, I'm not entirely sure if the latest build needs to be accepted or not. Possibly the easiest way to tell is just run the job, so here we go.

@sdodson
Copy link
Member

sdodson commented Jan 18, 2024

e2e-gcp-op-single-node is a known failure due to https://issues.redhat.com/browse/OCPBUGS-26489 when time comes to merge ping me or MCO team to override that test if it's still broken.

@JoelSpeed
Copy link
Contributor Author

/hold cancel

@cheesesashimi
Copy link
Member

/lgtm
/approval
/override e2e-gcp-op-single-node

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jan 18, 2024
Copy link
Contributor

openshift-ci bot commented Jan 18, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cheesesashimi, JoelSpeed

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 18, 2024
Copy link
Contributor

openshift-ci bot commented Jan 18, 2024

@cheesesashimi: /override requires failed status contexts, check run or a prowjob name to operate on.
The following unknown contexts/checkruns were given:

  • e2e-gcp-op-single-node

Only the following failed contexts/checkruns were expected:

  • ci/prow/bootstrap-unit
  • ci/prow/e2e-aws-ovn
  • ci/prow/e2e-aws-ovn-upgrade
  • ci/prow/e2e-aws-ovn-upgrade-out-of-change
  • ci/prow/e2e-azure-ovn-upgrade-out-of-change
  • ci/prow/e2e-gcp-op
  • ci/prow/e2e-gcp-op-layering
  • ci/prow/e2e-gcp-op-single-node
  • ci/prow/e2e-hypershift
  • ci/prow/images
  • ci/prow/okd-images
  • ci/prow/okd-scos-e2e-aws-ovn
  • ci/prow/okd-scos-images
  • ci/prow/unit
  • ci/prow/verify
  • pull-ci-openshift-machine-config-operator-master-bootstrap-unit
  • pull-ci-openshift-machine-config-operator-master-e2e-aws-ovn
  • pull-ci-openshift-machine-config-operator-master-e2e-aws-ovn-upgrade
  • pull-ci-openshift-machine-config-operator-master-e2e-aws-ovn-upgrade-out-of-change
  • pull-ci-openshift-machine-config-operator-master-e2e-azure-ovn-upgrade-out-of-change
  • pull-ci-openshift-machine-config-operator-master-e2e-gcp-op
  • pull-ci-openshift-machine-config-operator-master-e2e-gcp-op-layering
  • pull-ci-openshift-machine-config-operator-master-e2e-gcp-op-single-node
  • pull-ci-openshift-machine-config-operator-master-e2e-hypershift
  • pull-ci-openshift-machine-config-operator-master-images
  • pull-ci-openshift-machine-config-operator-master-okd-images
  • pull-ci-openshift-machine-config-operator-master-okd-scos-e2e-aws-ovn
  • pull-ci-openshift-machine-config-operator-master-okd-scos-images
  • pull-ci-openshift-machine-config-operator-master-unit
  • pull-ci-openshift-machine-config-operator-master-verify
  • tide

If you are trying to override a checkrun that has a space in it, you must put a double quote on the context.

In response to this:

/lgtm
/approval
/override e2e-gcp-op-single-node

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@cheesesashimi
Copy link
Member

/override ci/prow/e2e-gcp-op-single-node

Copy link
Contributor

openshift-ci bot commented Jan 18, 2024

@cheesesashimi: Overrode contexts on behalf of cheesesashimi: ci/prow/e2e-gcp-op-single-node

In response to this:

/override ci/prow/e2e-gcp-op-single-node

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Copy link
Contributor

openshift-ci bot commented Jan 18, 2024

@yuqi-zhang: Overrode contexts on behalf of yuqi-zhang: ci/prow/e2e-gcp-op-single-node

In response to this:

/override ci/prow/e2e-gcp-op-single-node

The bot is finicky and requires the full context :)

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci-robot
Copy link
Contributor

/retest-required

Remaining retests: 0 against base HEAD 9fb0e62 and 2 for PR HEAD 2da4c76 in total

Copy link
Contributor

openshift-ci bot commented Jan 19, 2024

@JoelSpeed: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/okd-scos-e2e-aws-ovn 2da4c76 link false /test okd-scos-e2e-aws-ovn
ci/prow/e2e-aws-ovn-upgrade-out-of-change 2da4c76 link false /test e2e-aws-ovn-upgrade-out-of-change
ci/prow/e2e-azure-ovn-upgrade-out-of-change 2da4c76 link false /test e2e-azure-ovn-upgrade-out-of-change

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@sdodson
Copy link
Member

sdodson commented Jan 19, 2024

/retest-required
unit tests have passed on this commit before, maybe flakey

@openshift-merge-bot openshift-merge-bot bot merged commit 6745e60 into openshift:master Jan 19, 2024
13 of 16 checks passed
@openshift-ci-robot
Copy link
Contributor

@JoelSpeed: Jira Issue OCPBUGS-25662: Some pull requests linked via external trackers have merged:

The following pull requests linked via external trackers have not merged:

These pull request must merge or be unlinked from the Jira bug in order for it to move to the next state. Once unlinked, request a bug refresh with /jira refresh.

Jira Issue OCPBUGS-25662 has not been moved to the MODIFIED state.

In response to this:

- What I did
Adds configuration for the ECR credential provider to kubelet on AWS.

Unit tests show new flags being set correctly.

Depends on the ecr-credential-provider RPM shipping in RHCOS, this is currently a WIP.
/hold

- How to verify it
Kubelet should be able to pull an image from a private ECR registry.

- Description for the changelog

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-bot
Copy link
Contributor

[ART PR BUILD NOTIFIER]

This PR has been included in build openshift-proxy-pull-test-container-v4.16.0-202401191549.p0.g6745e60.assembly.stream for distgit openshift-proxy-pull-test.
All builds following this will include this PR.

@JoelSpeed
Copy link
Contributor Author

/cherry-pick release-4.15

@openshift-cherrypick-robot

@JoelSpeed: new pull request created: #4134

In response to this:

/cherry-pick release-4.15

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@vrutkovs
Copy link
Member

vrutkovs commented Jan 26, 2024

Please revert, this breaks OKD, which doesn't ship this RPM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/severity-important Referenced Jira bug's severity is important for the branch this PR is targeting. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

9 participants