Windows containers do not have entries in the hosts file

[claudiubelu]

Is this a BUG REPORT or FEATURE REQUEST?:

/kind bug

/sig windows

What happened:

When deploying Kubernetes pods, the Linux containers will contain the following entries in the /etc/hosts file:

# Kubernetes-managed hosts file.
127.0.0.1       localhost
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
fe00::0 ip6-mcastprefix
fe00::1 ip6-allnodes
fe00::2 ip6-allrouters
10.240.0.67     dns-querier-1.dns-test-service.e2e-tests-dns-x649p.svc.cluster.local    dns-querier-1

However, these are not set on Windows containers; the container's C:\Windows\System32\drivers\etc\hosts file does not contain any entry. This can be attributed to the fact that individual files cannot be mounted on Windows containers the same way they are on Linux containers.

It can be fixed with a change to the kubelet once Windows Server 2019 has shipped, and once CRI-ContainerD is enabled on Windows. We cannot fix it for Windows Server 1803 or earlier with dockershim.

What you expected to happen:

The :\Windows\System32\drivers\etc\hosts file should be managed and populated by Kubelet.

How to reproduce it (as minimally and precisely as possible):

Spawn a Windows pod, and check its container's :\Windows\System32\drivers\etc\hosts file.

kubectl create namespace e2e-tests-dns-x649p
kubectl create -f windows_pod.yaml  # https://paste.ubuntu.com/p/ZgfQS9Wgcf/
kubectl exec -n e2e-tests-dns-x649p dns-test-2ed58282-6d85-11e8-89cc-000c29738e2c -c querier -- cmd /S /C type C:\\Windows\\System32\\drivers\\etc\\hosts

Compare the result with a Linux pod's container's /etc/hosts file.

kubectl create namespace e2e-tests-dns-x649p
kubectl create -f linux_pod.yaml  # https://paste.ubuntu.com/p/kcFcfxYjm3/

kubectl exec -n e2e-tests-dns-x649p dns-test-2ed58282-6d85-11e8-89cc-000c29738e2c -c querier -- cat /etc/hosts

Anything else we need to know?:

Environment:

• Kubernetes version (use kubectl version): 1.12
• Cloud provider or hardware configuration: N/A
• OS (e.g. from /etc/os-release): Windows Server 1803 (10.0.17134.48)
• Install tools: acs-engine v0.24.0
• Others:

[PatrickLang]

This requires single-file mapping, which won't work in Windows Server version 1803. It should be possible with Windows Server 2019 along with ContainerD

[feiskyer]

Will it work with Docker on Windows Server 2019?

[PatrickLang]

Not in 18.09 or earlier. I'm not sure what release it will be in because it will require migrating to containerd.

[AishSundar]

@PatrickLang should this issue be tracked for 1.13 in addition to #65016? If so please apply the appropriate priority and milestone labels please.

@nikopen

[PatrickLang]

This one needs to be in v1.14. We need to get cri-containerd working first to fix this.

[AishSundar]

Thanks
/milestone v1.14

[daschott]

By default, hosts file are not being used in Windows, it is all registry based. What features is this blocking?

CC @madhanrm

[claudiubelu]

The hosts file entries are still taken into account on Windows when it has to resolve a DNS name. This is blocking DNS conformance tests, like [sig-network] DNS should provide DNS for the cluster [Conformance].

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[neolit123]

/lifecycle frozen

[soggiest]

Hello!
1.14 code freeze is coming in about 3 days, is this intended to be implemented in the following week?

[claudiubelu]

No. It is meant to be fixed in the next release, when the Containerd support will be introduced.

[nikopen]

/milestone v1.15

[k8s-ci-robot]

@marosset: The provided milestone is not valid for this repository. Milestones in this repository: [next-candidate, v1.13, v1.14, v1.15, v1.16, v1.17, v1.18, v1.19, v1.20, v1.21, v1.22, v2.0]

Use /milestone clear to clear the milestone.

In response to this:

/milestone 1.20

The OS fix is now expect to be available as part of August Windows patches and then changes to the kubelet will need to follow.

/cc @immuzz

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-ci-robot]

@marosset: The provided milestone is not valid for this repository. Milestones in this repository: [next-candidate, v1.13, v1.14, v1.15, v1.16, v1.17, v1.18, v1.19, v1.20, v1.21, v1.22, v2.0]

Use /milestone clear to clear the milestone.

In response to this:

/milestone 1.20

The OS fix is now expect to be available as part of August Windows patches and then changes to the kubelet will need to follow.

/cc @immuzz

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[kbruner]

/milestone v1.20

[immuzz]

@marosset yes it is expected be part of August Windows release

[bai]

Hey 👋 Bug Triage here. Wanted to follow up on the status of this issue as we're approaching code freeze on 12.11.2020. This issue is tagged for 1.20, is it still planned for this release?

[marosset]

@bai - Yes, we'd like to make it into 1.20.
We were waiting on some bugfixes in the Windows OS and those bugfixes are now available (as of Oct 20 2020).

[marosset]

/remove-lifecycle frozen

[marosset]

/priority important-soon

[marosset]

I believe the work involved here is to

• update
  

  kubernetes/pkg/kubelet/kubelet_pods.go

  Lines 137 to 145 in ec453ff

  	func makeMounts(pod *v1.Pod, podDir string, container *v1.Container, hostName, hostDomain string, podIPs []string, podVolumes kubecontainer.VolumeMap, hu hostutil.HostUtils, subpather subpath.Interface, expandEnvs []kubecontainer.EnvVar) ([]kubecontainer.Mount, func(), error) {
  	// Kubernetes only mounts on /etc/hosts if:
  	// - container is not an infrastructure (pause) container
  	// - container is not already mounting on /etc/hosts
  	// - OS is not Windows
  	// Kubernetes will not mount /etc/hosts if:
  	// - when the Pod sandbox is being created, its IP is still unknown. Hence, PodIP will not have been set.
  	mountEtcHostsFile := len(podIPs) > 0 && runtime.GOOS != "windows"
  	klog.V(3).Infof("container: %v/%v/%v podIPs: %q creating hosts mount: %v", pod.Namespace, pod.Name, container.Name, podIPs, mountEtcHostsFile)

  to enable this fucntionality for WIndows if container-runtime == remote (still disable this for DockerShim)
• Update/Add test coverage
• Update documentation - possible this section https://kubernetes.io/docs/setup/production-environment/windows/intro-windows-in-kubernetes/#feature-restrictions ?

[marosset]

/milestone v1.21

Moving to 1.21 since the required OS fixes were not available in time to implement this functionality in v1.20

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale

[kcmartin]

Hello! I am from the Bug Triage team! This issue has not been updated for a long time, so I'd like to check on the status. The code freeze is starting March 9th, 2021 (about 1 week from now). We want to ensure that each PR has a chance to be merged before code freeze.

As the Issue is tagged for 1.21, is it still planned for this release?

[brandongarcia-ms]

Can this be reopened? From a related problem I have experienced using HostAlias for Windows pod deployments, this issue does not seem to be resolved. Instead, it looks as if it was closed by accident because the conversation went cold.

[claudiubelu]

Can this be reopened? From a related problem I have experienced using HostAlias for Windows pod deployments, this issue does not seem to be resolved. Instead, it looks as if it was closed by accident because the conversation went cold.

Hello. What Kubernetes version are you running? Also, on the Windows nodes, do you use docker, or containerd? This is only supported on Windows nodes with containerd, since it supports single file mapping, which is required in order to mount single files into Windows containers.

[brandongarcia-ms]

Hello. What Kubernetes version are you running? Also, on the Windows nodes, do you use docker, or containerd? This is only supported on Windows nodes with containerd, since it supports single file mapping, which is required in order to mount single files into Windows containers.

We have experimented with both 1.22.11 and a 1.23 release and yea, the container runtime is containerd. Any other ideas to check up on?