Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net: Go DNS resolver does not read /etc/hosts #22846

Closed
bontibon opened this issue Nov 22, 2017 · 10 comments
Closed

net: Go DNS resolver does not read /etc/hosts #22846

bontibon opened this issue Nov 22, 2017 · 10 comments

Comments

@bontibon
Copy link
Contributor

@bontibon bontibon commented Nov 22, 2017

What version of Go are you using (go version)?

go version go1.9.2 linux/amd64

What did you do?

The following script creates a Docker container with a cgo-disabled Go binary that resolves a host defined in the /etc/hosts file.

#!/bin/bash
set -e
set -x
cat > Dockerfile <<EOF
FROM alpine:3.6

COPY internal.sh /tmp/internal.sh
COPY go_dns /tmp/go_dns

CMD /tmp/internal.sh
EOF

cat > go_dns.go <<EOF
package main

import (
	"net"
	"fmt"
)

func main() {
	fmt.Println(net.LookupHost("example.com"))
}
EOF

cat > internal.sh <<EOF
set -e
set -x

export GODEBUG=netdns=10

cat /etc/resolv.conf

cat /etc/hosts

/tmp/go_dns

nslookup example.com
EOF

chmod +x internal.sh

go version
GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -o go_dns go_dns.go
docker build --no-cache -t go_dns .
docker run --rm \
           --add-host example.com:192.168.2.50 \
           -i \
           -t \
           go_dns

What did you expect to see?

Go's DNS resolver resolve the address that is contained in /etc/hosts.

Output:
+ cat
+ cat
+ cat
+ chmod +x internal.sh
+ go version
go version go1.9.2 linux/amd64
+ GOOS=linux
+ GOARCH=amd64
+ CGO_ENABLED=0
+ go build -o go_dns go_dns.go
+ docker build --no-cache -t go_dns .
Sending build context to Docker daemon 2.691 MB
Step 1 : FROM alpine:3.6
 ---> 053cde6e8953
Step 2 : COPY internal.sh /tmp/internal.sh
 ---> 58c4e4eabbd7
Removing intermediate container e915818d62ae
Step 3 : COPY go_dns /tmp/go_dns
 ---> bcf50d520610
Removing intermediate container 85918214a98c
Step 4 : CMD /tmp/internal.sh
 ---> Running in cf4216686f20
 ---> c75e5b4ccec8
Removing intermediate container cf4216686f20
Successfully built c75e5b4ccec8
+ docker run --rm --add-host example.com:192.168.2.50 -i -t go_dns
+ export GODEBUG=netdns=10
+ cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 8.8.8.8
nameserver 8.8.4.4
#nameserver 127.0.1.1
+ cat /etc/hosts
127.0.0.1	localhost
::1	localhost ip6-localhost ip6-loopback
fe00::0	ip6-localnet
ff00::0	ip6-mcastprefix
ff02::1	ip6-allnodes
ff02::2	ip6-allrouters
192.168.2.50	example.com
172.17.0.2	b44f6648150b
+ /tmp/go_dns
go package net: built with netgo build tag; using Go's DNS resolver
go package net: hostLookupOrder(example.com) = dns,files
[93.184.216.34 2606:2800:220:1:248:1893:25c8:1946] <nil>
+ nslookup example.com
nslookup: can't resolve '(null)': Name does not resolve

Name:      example.com
Address 1: 192.168.2.50 example.com

What did you see instead?

An external DNS lookup was performed.

(This may be the same issue that was happening here #14170)

@bontibon

This comment has been minimized.

Copy link
Contributor Author

@bontibon bontibon commented Nov 22, 2017

Reading the source of net/conf.go, I see the following comment in regards to /etc/nsswitch.conf:

// glibc says the default is "dns [!UNAVAIL=return] files"
// http://www.gnu.org/software/libc/manual/html_node/Notes-on-NSS-Configuration-File.html.

It turns out that file does not exist in the container. If I create it with the following contents:

hosts: files dns

I get the expected result.

@bontibon bontibon closed this Nov 22, 2017
@Kyslik

This comment has been minimized.

Copy link

@Kyslik Kyslik commented Dec 1, 2017

@deepakchoudhary

This comment has been minimized.

Copy link

@deepakchoudhary deepakchoudhary commented Apr 12, 2018

Docker container with a cgo-disabled Go binary that needs to resolves a host defined in the /etc/hosts file but wasn't working
@bontibon 's solution to worked for me
inside container run
echo "hosts: files dns" > /etc/nsswitch.conf

@Barbery

This comment has been minimized.

Copy link

@Barbery Barbery commented Aug 19, 2018

Hi, how to fix the problems in mac osx? I can not create the /etc/nsswitch.conf file

@kfox1111

This comment has been minimized.

Copy link

@kfox1111 kfox1111 commented Aug 24, 2018

As this is a common deployment method with golang and k8s with minimal images, this has very unexpected results. Can we fix this to function as expected in a minimal environment?

@thockin

@kfox1111

This comment has been minimized.

Copy link

@kfox1111 kfox1111 commented Aug 28, 2018

Ping

@panamafrancis

This comment has been minimized.

Copy link

@panamafrancis panamafrancis commented Aug 31, 2018

@kfox1111 You should just change your base image, golang should always defer to /etc/nsswitch.conf
@Barbery Here is not the place for sysadmin questions (hint: use sudo)

jumanjiman added a commit to jumanjiman/docker-ssllabs-scan that referenced this issue Sep 13, 2018
The golang resolver uses `/etc/nsswitch.conf` if it exists.
See golang/go#22846

Resolves jumanjihouse#64
jumanjiman added a commit to jumanjiman/docker-ssllabs-scan that referenced this issue Sep 13, 2018
The golang resolver uses `/etc/nsswitch.conf` if it exists.
See golang/go#22846

Resolves jumanjihouse#64
jumanjiman added a commit to jumanjiman/docker-ssllabs-scan that referenced this issue Sep 13, 2018
The golang resolver uses `/etc/nsswitch.conf` if it exists.
See golang/go#22846

Resolves jumanjihouse#64
@qinghai5060

This comment has been minimized.

Copy link

@qinghai5060 qinghai5060 commented Oct 29, 2018

another reason could be the permission ot '/etc/hosts', it should be 644 when user is not root

@kfox1111

This comment has been minimized.

Copy link

@kfox1111 kfox1111 commented Oct 29, 2018

@panamafrancis it is common to use alpine as a base image as its size is small. it uses a non glibc implementation and does not contain nsswitch.conf. The distro assumes nonglibc behavior as its not glibc based, and behaves well in the absence of nsswitch.conf. golang chose to implement glibc's failback behaviour in the absence of nsswitch.conf, which IMO is a bad default. etc/hosts should be honored by default in the absence of nsswitch.conf. Alpine is not going away any time soon, and convincing folks not to use alpine is going to be a hard sell. I'd suggest golang change its default to a safer one, ie honor etc/hosts first, in light of this. This would be the least surprise option as well as one that hardens security.

@kfox1111

This comment has been minimized.

Copy link

@kfox1111 kfox1111 commented Nov 14, 2018

Ok... just hit this again when switching to k8s 1.12.

2opremio added a commit to 2opremio/flux that referenced this issue Jan 3, 2019
As a result, /etc/hosts will be prioritized over DNS queries

This resolves the conflict between:
 * fluxd using netgo for static compilation. netgo reads nsswitch.conf to mimic glibc,
   defaulting to prioritize DNS queries over /etc/hosts if nsswitch.conf is missing:
   golang/go#22846
 * Alpine not including a nsswitch.conf file. Since Alpine doesn't use glibc
   (it uses musl), maintainers argue that the need of nsswitch.conf is a Go bug:
   gliderlabs/docker-alpine#367 (comment)
rfgamaral added a commit to rfgamaral/docker-gphotos-uploader that referenced this issue Sep 14, 2019
rfgamaral added a commit to rfgamaral/docker-gphotos-uploader that referenced this issue Sep 14, 2019
@golang golang locked and limited conversation to collaborators Nov 14, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
8 participants
You can’t perform that action at this time.