agnhost: resolve service account issuer URL before invoking oidc.NewProvider

[chewong]

Signed-off-by: Ernest Wong chuwon@microsoft.com

What type of PR is this?

/sig windows
/kind failing-test
/priority important-soon
/release-note-none
/assign @jsturtevant

Ref: #99470

DNS can be available sometime after the container starts due to the way networking is set up with docker. In containerd the setup steps are different so the network will be available when the container starts. In this case the test image does a lookup before the DNS is finished coming online and fails.

We need to ensure agnhost's ability to resolve the service account issuer before invoking oidc.NewProvider.

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

[chewong]

/release-note-none

[chewong]

Test result:

[sig-auth] ServiceAccounts
  ServiceAccountIssuerDiscovery should support OIDC discovery of service account issuer [Conformance]
  /Users/ernestwong/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:630
[BeforeEach] [sig-auth] ServiceAccounts
  /Users/ernestwong/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185
STEP: Creating a kubernetes client
Mar  5 13:37:26.909: INFO: >>> kubeConfig: /Users/ernestwong/api-model/_output/chuwon-win-docker/kubeconfig/kubeconfig.westus2.json
STEP: Building a namespace api object, basename svcaccounts
W0305 13:37:27.046499   37824 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
Mar  5 13:37:27.046: INFO: Found PodSecurityPolicies; testing pod creation to see if PodSecurityPolicy is enabled
Mar  5 13:37:27.082: INFO: PSP annotation exists on dry run pod: "e2e-test-privileged-psp"; assuming PodSecurityPolicy is enabled
W0305 13:37:27.107661   37824 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
Mar  5 13:37:27.131: INFO: Found ClusterRoles; assuming RBAC is enabled.
STEP: Binding the e2e-test-privileged-psp PodSecurityPolicy to the default service account in svcaccounts-7597
STEP: Waiting for a default service account to be provisioned in namespace
[It] ServiceAccountIssuerDiscovery should support OIDC discovery of service account issuer [Conformance]
  /Users/ernestwong/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:630
Mar  5 13:37:27.382: INFO: created pod
Mar  5 13:37:27.382: INFO: Waiting up to 5m0s for pod "oidc-discovery-validator" in namespace "svcaccounts-7597" to be "Succeeded or Failed"
Mar  5 13:37:27.415: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 32.983545ms
Mar  5 13:37:29.443: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2.061053196s
Mar  5 13:37:31.465: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4.083535799s
Mar  5 13:37:33.498: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 6.115648986s
Mar  5 13:37:35.531: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 8.149222728s
Mar  5 13:37:37.551: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 10.169562416s
Mar  5 13:37:39.577: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 12.19523457s
Mar  5 13:37:41.601: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 14.218759709s
Mar  5 13:37:43.631: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 16.249002641s
Mar  5 13:37:45.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 18.27639084s
Mar  5 13:37:47.683: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 20.300555055s
Mar  5 13:37:49.711: INFO: Pod "oidc-discovery-validator": Phase="Succeeded", Reason="", readiness=false. Elapsed: 22.328850735s
STEP: Saw pod success
Mar  5 13:37:49.711: INFO: Pod "oidc-discovery-validator" satisfied condition "Succeeded or Failed"
Mar  5 13:38:19.714: INFO: polling logs
Mar  5 13:38:19.783: INFO: Pod logs:
2021/03/05 21:37:44 OK: Got token
2021/03/05 21:37:45 OK: got issuer https://kubernetes.default.svc.cluster.local
2021/03/05 21:37:45 Full, not-validated claims:
openidmetadata.claims{Claims:jwt.Claims{Issuer:"https://kubernetes.default.svc.cluster.local", Subject:"system:serviceaccount:svcaccounts-7597:default", Audience:jwt.Audience{"oidc-discovery-test"}, Expiry:1614980847, NotBefore:1614980247, IssuedAt:1614980247, ID:""}, Kubernetes:openidmetadata.kubeClaims{Namespace:"svcaccounts-7597", ServiceAccount:openidmetadata.kubeName{Name:"default", UID:"da4f2ced-41b6-484a-b62d-9045a1dca8fc"}}}
2021/03/05 21:37:45 Ensuring Windows DNS availability
2021/03/05 21:37:45 lookup kubernetes.default.svc.cluster.local: no such host
2021/03/05 21:37:46 OK: Resolved host kubernetes.default.svc.cluster.local: [10.0.0.1]
2021/03/05 21:37:46 OK: Constructed OIDC provider for issuer https://kubernetes.default.svc.cluster.local
2021/03/05 21:37:46 OK: Validated signature on JWT
2021/03/05 21:37:46 OK: Got valid claims from token!
2021/03/05 21:37:46 Full, validated claims:
&openidmetadata.claims{Claims:jwt.Claims{Issuer:"https://kubernetes.default.svc.cluster.local", Subject:"system:serviceaccount:svcaccounts-7597:default", Audience:jwt.Audience{"oidc-discovery-test"}, Expiry:1614980847, NotBefore:1614980247, IssuedAt:1614980247, ID:""}, Kubernetes:openidmetadata.kubeClaims{Namespace:"svcaccounts-7597", ServiceAccount:openidmetadata.kubeName{Name:"default", UID:"da4f2ced-41b6-484a-b62d-9045a1dca8fc"}}}

Mar  5 13:38:19.783: INFO: completed pod
[AfterEach] [sig-auth] ServiceAccounts
  /Users/ernestwong/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186
Mar  5 13:38:19.808: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "svcaccounts-7597" for this suite.

• [SLOW TEST:52.974 seconds]
[sig-auth] ServiceAccounts
/Users/ernestwong/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/auth/framework.go:23
  ServiceAccountIssuerDiscovery should support OIDC discovery of service account issuer [Conformance]
  /Users/ernestwong/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:630

[jsturtevant]

/triage accepted
/milestone v1.21
/sig windows

[chewong]

Assigning agnhost's owner:
/assign @claudiubelu

assigning the test case owner:
/assign @mtaufen

Thanks!

[jsturtevant]

/lgtm

[mtaufen]

/lgtm

but want to confirm the timeout is enough

[dims]

/approve

@chewong there's still one open question from @mtaufen

/assign @spiffxp

[spiffxp]

/hold
For question
/milestone v1.21
Targeting test freeze

[dims]

please see error in CI logs

INFO Validating zeitgeist options...              
ERRO /home/prow/go/src/k8s.io/kubernetes/build/dependencies.yaml indicates that agnhost should be at version 2.29, but the following files didn't match: test/images/agnhost/VERSION, test/images/agnhost/agnhost.go 

[spiffxp]

/approve
/lgtm
/hold cancel

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: chewong, dims, spiffxp

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• build/OWNERS [spiffxp]
• test/images/agnhost/OWNERS [dims,spiffxp]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment