Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net: pure Go DNS resolver hangs with ipv6 link-local address on macOS #52839

Open
mattrobenolt opened this issue May 10, 2022 · 1 comment
Open

net: pure Go DNS resolver hangs with ipv6 link-local address on macOS #52839

mattrobenolt opened this issue May 10, 2022 · 1 comment
Labels
NeedsInvestigation
Milestone

Comments

@mattrobenolt
Copy link
Contributor

@mattrobenolt mattrobenolt commented May 10, 2022

What version of Go are you using (go version)?

$ go version
go version go1.18 darwin/amd64

This is also reproduced against go1.17.8, so doesn't appear to be new to go1.18.

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

macOS 12.3, Intel (amd64)

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/xxx/Library/Caches/go-build"
GOENV="/Users/xxx/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/xxx/go/pkg/mod"
GOOS="darwin"
GOPATH="/Users/xxx/go"
GOPROXY="https://proxy.golang.org/,direct"
GOROOT="/usr/local/Cellar/go/1.18/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.18/libexec/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.18"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/pb/fk1_tmbn1nq32bgpd6njflcw0000gn/T/go-build3259567292=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Making a DNS lookup against an ipv6 link-local address hangs and fails on macOS when using thenative (non-CGO) resolver.

This appears to manifest within the non-CGO based DNS resolver, but I believe it's more generally applicable to the network stack itself, it's just easier to reproduce through DNS.

The simplest reproduction we could come up with was this compiled with CGO_ENABLED=0:

package main

import (
	"context"
	"fmt"
	"net"
	"os"
	"time"
)

func main() {
	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
	defer cancel()
	ips, err := net.DefaultResolver.LookupIP(ctx, "ip", "google.com")
	if err != nil {
		fmt.Fprintf(os.Stderr, "Could not get IPs: %v\n", err)
		os.Exit(1)
	}
	for _, ip := range ips {
		fmt.Printf("google.com. IN A %s\n", ip.String())
	}
}

Paired with an /etc/resolv.conf of:

nameserver fe80::a04e:cfff:fe2f:ad64%en0
nameserver 10.0.0.1

I've deduced this down to explicitly any link-local address (fe80::*) and not generically to ipv6. Other ipv6 addresses, both public and private network appear fine.

This behavior is especially concerning because it seems to cascade into larger macOS failures. All DNS seems to start failing on the whole system requiring ultimately a reboot to restore stability.

Given this, I believe this is also a bug within macOS that Go happens to be exploiting somehow.

I personally don't have a macOS machine to iterate on this, but we've deduced this both with customers running our CLI (https://github.com/planetscale/cli) as well as employees internally.

After a bunch of deducing, we've narrowed it down to the link-local address.

Again, when compiled with CGO_ENABLED=1, everything works correctly and is fine. The issue only occurs when compiled with CGO_ENABLED=0, triggering the pure Go DNS resolution path.

We originally thought this was related to arm64 (M1 Macs), but was able to finally reproduce it on Intel.

What did you expect to see?

Fast DNS resolution

What did you see instead?

Program hangs, eventually times out, in extreme cases, causing entire OS to become unstable.

For additional context: planetscale/discussion#181

@heschi
Copy link
Contributor

@heschi heschi commented May 11, 2022

@heschi heschi added the NeedsInvestigation label May 11, 2022
@heschi heschi added this to the Backlog milestone May 11, 2022
@mattrobenolt mattrobenolt changed the title net: pure Go DNS resolver hangs with ipv6 link-local address net: pure Go DNS resolver hangs with ipv6 link-local address on macOS May 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation
Projects
None yet
Development

No branches or pull requests

2 participants