Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[azure] remove dummy MSI_ENDPOINT #2384

Merged
merged 1 commit into from
Nov 17, 2021
Merged

[azure] remove dummy MSI_ENDPOINT #2384

merged 1 commit into from
Nov 17, 2021

Conversation

polivbr
Copy link
Contributor

@polivbr polivbr commented Oct 18, 2021

For some reason, code was added to set the MSI_ENDPOINT environment to a dummy value, which prevents the retrieval of an access token when using managed identities. This PR removes this code.

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Oct 18, 2021
@k8s-ci-robot k8s-ci-robot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Oct 18, 2021
@lareeth
Copy link

lareeth commented Oct 18, 2021

Related #2383

@hamza-tumturk
Copy link

I'm facing this issue with the release v0.10.0.
level=error msg="azure.BearerAuthorizer#WithAuthorization: Failed to refresh the Token for request to https://management.azure.com/subscriptions/(subscription)/resourceGroups/(resourceGroup)/providers/Microsoft.Network/dnsZones?api-version=2018-05-01: StatusCode=0 -- Original Error: adal: Failed to execute the refresh request. Error = 'Post \"http://dummy\": dial tcp: lookup dummy on 10.0.0.10:53: no such host'"

In which release was this bug introduced? And is there a known workaround for this to get managed identities with Azure working?

@kaganmersin
Copy link

kaganmersin commented Oct 20, 2021

In which release was this bug introduced? And is there a known workaround for this to get managed identities with Azure working?

Di̇d you find any working version or workaround solution until the next release that bug will be resolved ? Thanks

@lareeth
Copy link

lareeth commented Oct 20, 2021

The last working version is https://github.com/kubernetes-sigs/external-dns/releases/tag/v0.9.0 which doesnt have the MSI_ENDPOINT commit

@kaganmersin
Copy link

The last working version is https://github.com/kubernetes-sigs/external-dns/releases/tag/v0.9.0 which doesnt have the MSI_ENDPOINT commit

Yes I just checked that 0.9.0 is works for me

@duncan485
Copy link

Please review and merge this issue. This also causes issue with using aadpodidentity in conjunction with ExternalDNS

@phillebaba
Copy link
Contributor

This seriously needs to be merged with an immediate version release as this basically breaks MSI for a lot of people. How was this merged in the first place?

@twendt
Copy link

twendt commented Nov 4, 2021

The tests are failing with this PR as the adal code tries to do an http get on the msi endpoint which is not available outside of Azure and therefore fails.

I was able to fix it by setting the MSI_ENDPOINT in the test code in azure_test.go in line 472:

	os.Setenv("MSI_ENDPOINT", "http://dummy")
	token, err := getAccessToken(cfg, env)
	os.Unsetenv("MSI_ENDPOINT")

@phillebaba
Copy link
Contributor

My question is why the tests would pass with the MSI_ENDPOINT set to dummy? In that case the tests are not testing anything really.

@njuettner is there anything that I can do to help move this along? It does not really seem like there is a maintainer for the Azure provider so dont really know who to reach out to 😃

@twendt
Copy link

twendt commented Nov 4, 2021

The MS adal package which is being used by external-dns is trying to figure out which endpoint is supposed to be used. If the MSI_ENDPOINT environment variable is set, then that is used. If not then it tries to do an http get on the http://169.254.169.254/metadata/identity/oauth2/token which only works in Azure.

The solution to set the env var in the test is fine I would say, but it is not perfect since the returned token looks a little different. But the token is still an msi token and that is being verified by the test.
It is not the purpose of the test to verify that the function provided by the external package works correctly. Therefore I think this is ok.

I compiled the PR branch and verified that it works in our environment with a user assigned managed identity.

Copy link
Member

@njuettner njuettner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Nov 17, 2021
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: njuettner, polivbr

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 17, 2021
@k8s-ci-robot k8s-ci-robot merged commit 13951ff into kubernetes-sigs:master Nov 17, 2021
@stevehipwell
Copy link
Contributor

@Raffo @njuettner @seanmalloy any idea when this will be released, I assume it could be a patch? Currently v0.10 doesn't work in Azure with managed identity, but v0.10 is the recommended version for the available AKS K8s versions.

@Raffo
Copy link
Contributor

Raffo commented Nov 29, 2021

I'm gonna have a look at the release problem this week.

@nabeelDanish
Copy link

Beginner here so can anyone help me here? I have my AKS Service configured to work with Ingress, but the Ingress log files show this same error

level=error msg="azure.BearerAuthorizer#WithAuthorization: Failed to refresh the Token for request to https://management.azure.com/subscriptions/(subscription)/resourceGroups/(resourceGroup)/providers/Microsoft.Network/dnsZones?api-version=2018-05-01: StatusCode=0 -- Original Error: adal: Failed to execute the refresh request. Error = 'Post \"http://dummy\": dial tcp: lookup dummy on 10.0.0.10:53: no such host'"

I want to know if this issue is resolved in Azure or not, and if not what is the workaround to this? Do you have to downgrade your Kubernetes or external-dns version for this? If that's the case how would you do that in Azure?

@duncan485
Copy link

Hi @nabeelDanish , what version of external-dns are you running? This issue was resolved in 0.11.0, and as far as I know only existed in 0.10.0

@nabeelDanish
Copy link

@duncan485 I am sorry I don't know that I am using Azure Kubernetes Service to deploy my multicontainerized application. Is there a way to find on Azure Cloud Shell which version of external-dns I am running? and if there is a way to upgrade the version?

@stevehipwell
Copy link
Contributor

@nabeelDanish you need to manually install ExternalDNS on an AKS cluster. Could you explain how you got it installed and also lookup the image version from the Deployment?

@stevehipwell
Copy link
Contributor

@nabeelDanish do you happen to be using HTTP application routing? If so I think that bundles ExternalDNS as a sub component and Azure/AKS should make sure they're using a compatible version of ExternalDNS.

I'm not sure if it makes a difference but it looks like the version has been updated to v0.10.2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet