Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dualstack downward api #83123

Merged
merged 1 commit into from Nov 15, 2019
Merged

Conversation

@aramase
Copy link
Member

aramase commented Sep 25, 2019

What type of PR is this?

Uncomment only one /kind <> line, hit enter to put that in a new line, and remove leading whitespaces from that line:

/kind api-change
/kind bug
/kind cleanup
/kind design
/kind documentation
/kind failing-test

/kind feature

/kind flake

What this PR does / why we need it:

https://github.com/kubernetes/enhancements/blob/master/keps/sig-network/20180612-ipv4-ipv6-dual-stack.md#container-environment-variables

  • adds downward api support for podIPs
  • updates the host file with v4 and v6 pod IPs for dualstack
        env:
        - name: MY_POD_IPS
          valueFrom:
            fieldRef:
              fieldPath: status.podIPs
$ kubectl exec -ti nginx-8bff7fdb7-xdzj6 -- env | grep POD_IPS
MY_POD_IPS=10.244.0.6,fd00::6
$ kubectl exec -ti nginx-8bff7fdb7-xdzj6 -- cat /etc/hosts
# Kubernetes-managed hosts file.
127.0.0.1    localhost
::1    localhost ip6-localhost ip6-loopback
fe00::0    ip6-localnet
fe00::0    ip6-mcastprefix
fe00::1    ip6-allnodes
fe00::2    ip6-allrouters
10.244.0.6    nginx-8bff7fdb7-xdzj6
fd00::6    nginx-8bff7fdb7-xdzj6

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

The following information is available through environment variables:

status.podIPs - the pod's IP addresses

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

KEP - https://github.com/kubernetes/enhancements/blob/master/keps/sig-network/20180612-ipv4-ipv6-dual-stack.md
@k8s-ci-robot k8s-ci-robot requested review from deads2k and derekwaynecarr Sep 25, 2019
@aramase aramase force-pushed the aramase:dualstack-downward-api branch from 3b19ab7 to 7d09498 Sep 25, 2019
@k8s-ci-robot k8s-ci-robot added size/M and removed size/L labels Sep 25, 2019
@aramase aramase force-pushed the aramase:dualstack-downward-api branch from ceb3bf0 to 9fe9d4d Sep 25, 2019
@k8s-ci-robot k8s-ci-robot added size/L and removed size/M labels Sep 25, 2019
@aramase aramase force-pushed the aramase:dualstack-downward-api branch 5 times, most recently from fff4bb3 to b0e439f Sep 25, 2019
@aramase aramase changed the title [WIP] Dualstack downward api Dualstack downward api Sep 26, 2019
@fejta-bot

This comment has been minimized.

Copy link

fejta-bot commented Sep 26, 2019

This PR may require API review.

If so, when the changes are ready, complete the pre-review checklist and request an API review.

Status of requested reviews is tracked in the API Review project.

@aojea

This comment has been minimized.

Copy link
Contributor

aojea commented Oct 8, 2019

SGTM,
I tested it locally with kind and containerd and the entries are obtained via env variables

MY_POD_IPs=10.244.1.29,fd00:200::1d
MY_POD_IP=10.244.1.29

and are present in the /etc/hosts file

10.244.1.29     dapi-envars-fieldref
fd00:200::1d    dapi-envars-fieldref
@aramase aramase force-pushed the aramase:dualstack-downward-api branch from d238238 to 94192f1 Oct 16, 2019
@aramase

This comment has been minimized.

Copy link
Member Author

aramase commented Oct 16, 2019

are we sure that PodIPs will have values if the gate is turned off? (while we send both PodIP and PodIPs, as we dive deep into call chain we depend only on PodIP) -- Maybe we should leave a comment on the need to move everything to PodIP (if we ensure that it will have values even if the gate is off)

Verified that PodIPs have values even if the gate is turned off. So I've added a note to move everything to use only PodIPs instead of PodIP.

@khenidak I've updated the PR addressing review comments. PTAL when you get a chance.

@aramase

This comment has been minimized.

Copy link
Member Author

aramase commented Oct 16, 2019

/test pull-kubernetes-integration

@aramase

This comment has been minimized.

Copy link
Member Author

aramase commented Oct 17, 2019

/retest

@aramase aramase force-pushed the aramase:dualstack-downward-api branch from 2e4391b to 0b451ef Oct 17, 2019
@aramase

This comment has been minimized.

Copy link
Member Author

aramase commented Oct 17, 2019

/test pull-kubernetes-e2e-gce

Copy link
Contributor

khenidak left a comment

Another review, fewer set of comments. Getting close.

add host file write for podIPs

update tests

remove import alias

update type check

update type check

remove import alias

update open api spec

add tests

update test

add tests

address review comments

update imports

remove todo and import alias
@aramase aramase force-pushed the aramase:dualstack-downward-api branch from d414f48 to af4d18c Oct 25, 2019
@aramase

This comment has been minimized.

Copy link
Member Author

aramase commented Oct 25, 2019

/test pull-kubernetes-e2e-gce-100-performance

@khenidak

This comment has been minimized.

Copy link
Contributor

khenidak commented Oct 29, 2019

/lgtm

@aojea

This comment has been minimized.

Copy link
Contributor

aojea commented Oct 29, 2019

/test pull-kubernetes-conformance-kind-ipv6

@aramase

This comment has been minimized.

Copy link
Member Author

aramase commented Nov 6, 2019

/assign @thockin

// Kubernetes only mounts on /etc/hosts if:
// - container is not an infrastructure (pause) container
// - container is not already mounting on /etc/hosts
// - OS is not Windows
// Kubernetes will not mount /etc/hosts if:
// - when the Pod sandbox is being created, its IP is still unknown. Hence, PodIP will not have been set.
mountEtcHostsFile := len(podIP) > 0 && runtime.GOOS != "windows"
klog.V(3).Infof("container: %v/%v/%v podIP: %q creating hosts mount: %v", pod.Namespace, pod.Name, container.Name, podIP, mountEtcHostsFile)
mountEtcHostsFile := len(podIPs) > 0 && runtime.GOOS != "windows"

This comment has been minimized.

Copy link
@andrewsykim

andrewsykim Nov 6, 2019

Member

This seems scary, is there a reason we can't keep this as len(podIP) > 0? It ensures existing behavior and has the same result since primary pod IP is guaranteed to be set here anyways.

This comment has been minimized.

Copy link
@aramase

aramase Nov 6, 2019

Author Member

The podIP and podIPs are passed from caller SyncPod. Here podIP is set explicitly to the first value of podIPs - https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/kuberuntime/kuberuntime_manager.go#L744-L747. So podIPs is guaranteed to be set right? If podIPs is empty, then podIP is going to be empty here.

This comment has been minimized.

Copy link
@thockin

thockin Nov 15, 2019

Member

This is changing the meaning of the test. Before it was checking whether the podIP was valid (kind of - does it have any bytes). Now it is testing whether the list of podIPs has and values listed.

That is meaningfully different, though I think it will come out the same, since we should never set podIPs[0] if there is not a valid IP. I think. I checked the conversion code and it seems safe.

@lachie83

This comment has been minimized.

Copy link
Member

lachie83 commented Nov 14, 2019

/milestone v1.17

@k8s-ci-robot k8s-ci-robot added this to the v1.17 milestone Nov 14, 2019
Copy link
Member

thockin left a comment

Thanks! Fixes in followups is OK.

/lgtm
/approve

@@ -45,7 +45,7 @@ type HandlerRunner interface {
// RuntimeHelper wraps kubelet to make container runtime
// able to get necessary informations like the RunContainerOptions, DNS settings, Host IP.
type RuntimeHelper interface {
GenerateRunContainerOptions(pod *v1.Pod, container *v1.Container, podIP string) (contOpts *RunContainerOptions, cleanupAction func(), err error)
GenerateRunContainerOptions(pod *v1.Pod, container *v1.Container, podIP string, podIPs []string) (contOpts *RunContainerOptions, cleanupAction func(), err error)

This comment has been minimized.

Copy link
@thockin

thockin Nov 15, 2019

Member

nit: isn't podIP always podIPs[0] ? We should not need both. Ok for followup.

This comment has been minimized.

Copy link
@aramase

aramase Nov 15, 2019

Author Member

@thockin correct, podIP is always podsIPs[0]. Will do a followup PR. Thank you for the review.

// Kubernetes only mounts on /etc/hosts if:
// - container is not an infrastructure (pause) container
// - container is not already mounting on /etc/hosts
// - OS is not Windows
// Kubernetes will not mount /etc/hosts if:
// - when the Pod sandbox is being created, its IP is still unknown. Hence, PodIP will not have been set.
mountEtcHostsFile := len(podIP) > 0 && runtime.GOOS != "windows"
klog.V(3).Infof("container: %v/%v/%v podIP: %q creating hosts mount: %v", pod.Namespace, pod.Name, container.Name, podIP, mountEtcHostsFile)
mountEtcHostsFile := len(podIPs) > 0 && runtime.GOOS != "windows"

This comment has been minimized.

Copy link
@thockin

thockin Nov 15, 2019

Member

This is changing the meaning of the test. Before it was checking whether the podIP was valid (kind of - does it have any bytes). Now it is testing whether the list of podIPs has and values listed.

That is meaningfully different, though I think it will come out the same, since we should never set podIPs[0] if there is not a valid IP. I think. I checked the conversion code and it seems safe.

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

k8s-ci-robot commented Nov 15, 2019

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: aramase, thockin

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@aramase

This comment has been minimized.

Copy link
Member Author

aramase commented Nov 15, 2019

/test pull-kubernetes-kubemark-e2e-gce-big

@k8s-ci-robot k8s-ci-robot merged commit 4e45328 into kubernetes:master Nov 15, 2019
16 checks passed
16 checks passed
cla/linuxfoundation aramase authorized
Details
pull-kubernetes-bazel-build Job succeeded.
Details
pull-kubernetes-bazel-test Job succeeded.
Details
pull-kubernetes-conformance-kind-ipv6 Job succeeded.
Details
pull-kubernetes-dependencies Job succeeded.
Details
pull-kubernetes-e2e-gce Job succeeded.
Details
pull-kubernetes-e2e-gce-100-performance Job succeeded.
Details
pull-kubernetes-e2e-gce-device-plugin-gpu Job succeeded.
Details
pull-kubernetes-e2e-kind Job succeeded.
Details
pull-kubernetes-integration Job succeeded.
Details
pull-kubernetes-kubemark-e2e-gce-big Job succeeded.
Details
pull-kubernetes-node-e2e Job succeeded.
Details
pull-kubernetes-node-e2e-containerd Job succeeded.
Details
pull-kubernetes-typecheck Job succeeded.
Details
pull-kubernetes-verify Job succeeded.
Details
tide In merge pool.
Details
@aramase aramase deleted the aramase:dualstack-downward-api branch Nov 15, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.