IRSA - continue adding route53 permisions to masters #10529
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
k8s-ci-robot
added
cncf-cla: yes
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: rifelpet The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
k8s-ci-robot
added
the
approved
These are needed by protokube to create the kops-controller DNS record to allow nodes to bootstrap. See these logs: https://storage.googleapis.com/kubernetes-jenkins/logs/e2e-kops-grid-scenario-public-jwks/1345956556562239488/artifacts/ip-172-20-48-1.sa-east-1.compute.internal/protokube.log ``` I0104 05:03:51.264472 6482 dnscache.go:74] querying all DNS zones (no cached results) I0104 05:03:51.264570 6482 route53.go:53] AWS request: route53 ListHostedZones W0104 05:03:51.389485 6482 dnscontroller.go:124] Unexpected error in DNS controller, will retry: error querying for zones: error querying for DNS zones: AccessDenied: User: arn:aws:sts::768319786644:assumed-role/masters.e2e-kops-scenario-public-jwks.test-cncf-aws.k8s.io/i-05b1db10d1a5b8637 is not authorized to perform: route53:ListHostedZones ``` and the nodeup logs on nodes that couldn't join the cluster: ``` Jan 04 04:55:53.500187 ip-172-20-38-84 nodeup[2070]: W0104 04:55:53.500117 2070 executor.go:131] error running task "BootstrapClient/BootstrapClient" (9m52s remaining to succeed): Post "https://kops-controller.internal.e2e-kops-scenario-public-jwks.test-cncf-aws.k8s.io:3988/bootstrap": dial tcp: lookup kops-controller.internal.e2e-kops-scenario-public-jwks.test-cncf-aws.k8s.io on 127.0.0.53:53: no such host ```
k8s-ci-robot
added
size/M
|
If we have to support CSR bootstrap mechanism for bottlerocket, this could be used here as well. /lgtm |
k8s-ci-robot
added
the
lgtm
|
@justinsb the route53 record is still failing to be created: https://prow.k8s.io/view/gcs/kubernetes-jenkins/logs/e2e-kops-grid-scenario-public-jwks/1346681344700190720 We dont collect the dns-controller logs in the job artifacts but my new theory is that kubetest only allows the prow job's public IP to access apiserver but it needs to be 0.0.0.0/0 so that IAM can reach the JWKS endpoints. |
justinsb
added a commit
to justinsb/kops
that referenced
this pull request
Mar 20, 2021
Should not be needed; dns-controller should run on the control-plane node so there should not be a bootstrapping problem with the nodes. Reverts kubernetes#10529
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
These are needed by protokube to create the kops-controller DNS record to allow nodes to bootstrap. Ideally we could figure out a way for protokube to not rely on the ec2 instance's instance profile and role but until then we can revert to our original behavior here.
See these logs: https://storage.googleapis.com/kubernetes-jenkins/logs/e2e-kops-grid-scenario-public-jwks/1345956556562239488/artifacts/ip-172-20-48-1.sa-east-1.compute.internal/protokube.log
and the nodeup logs on nodes that couldn't join the cluster: