Skip to content

runtime: signal/illegal instruction running amd64 Golang compiler on Apple M3 Max #71434

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
mleonhard-notion opened this issue Jan 25, 2025 · 8 comments
Closed

runtime: signal/illegal instruction running amd64 Golang compiler on Apple M3 Max #71434

mleonhard-notion opened this issue Jan 25, 2025 · 8 comments
Labels
compiler/runtime Issues related to the Go compiler and/or runtime.

Comments

@mleonhard-notion
Copy link

Go version

go version go1.20.5 linux/amd64

Output of go env in your module/workspace:

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.20.5"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-O2 -g"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-O2 -g"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build3669854689=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I tried to build a Golang program inside Docker on an M4 MacBook Pro, using an amd64 version of the Docker golang image.

# Dockerfile
FROM --platform=linux/amd64 golang:1.20.5-buster
RUN apt-get --allow-releaseinfo-change -y update
RUN apt-get install -y git make
WORKDIR /go/src/app
COPY google-webrisk .
RUN make

Code to reproduce: google-webrisk-amd64-docker.tar.gz

 % system_profiler SPHardwareDataType
Hardware:

    Hardware Overview:

      Model Name: MacBook Pro
      Model Identifier: Mac15,10
      Model Number: MRX53LL/A
      Chip: Apple M3 Max
      Total Number of Cores: 14 (10 performance and 4 efficiency)
      Memory: 36 GB
      System Firmware Version: 11881.61.3
      OS Loader Version: 11881.61.3

% docker version
Client:
 Version:           27.4.0
 API version:       1.47
 Go version:        go1.22.10
 Git commit:        bde2b89
 Built:             Sat Dec  7 10:35:43 2024
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.37.2 (179585)
 Engine:
  Version:          27.4.0
  API version:      1.47 (minimum version 1.24)
  Go version:       go1.22.10
  Git commit:       92a8393
  Built:            Sat Dec  7 10:38:33 2024
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.7.21
  GitCommit:        472731909fa34bd7bc9c087e4c27943f9835f111
 runc:
  Version:          1.1.13
  GitCommit:        v1.1.13-0-g58aa920
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

I'm using:

  • Apple Macbook Pro M3 Max
  • macOS 15.2 (24C101)
  • Docker 4.37.2 (179585)
    • Virtual Machine Manager: "Apple Virtualization framework"
    • Enabled: "Use Rosetta for x86_64/amd64 emulation on Apple Silicon"

What did you see happen?

The Golang compiler crashed with signal: illegal instruction

google-webrisk-amd64-docker % docker build .
[+] Building 17.2s (10/10) FINISHED                                                                                                                                                                                           docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                                                                                                                                                          0.0s
 => => transferring dockerfile: 220B                                                                                                                                                                                                          0.0s
 => WARN: FromPlatformFlagConstDisallowed: FROM --platform flag should not use constant value "linux/amd64" (line 1)                                                                                                                          0.0s
 => [internal] load metadata for docker.io/library/golang:1.20.5-buster                                                                                                                                                                       0.7s
 => [internal] load .dockerignore                                                                                                                                                                                                             0.0s
 => => transferring context: 2B                                                                                                                                                                                                               0.0s
 => [1/6] FROM docker.io/library/golang:1.20.5-buster@sha256:eb3f9ac805435c1b2c965d63ce460988e1000058e1f67881324746362baf9572                                                                                                                 0.0s
 => => resolve docker.io/library/golang:1.20.5-buster@sha256:eb3f9ac805435c1b2c965d63ce460988e1000058e1f67881324746362baf9572                                                                                                                 0.0s
 => => sha256:eb3f9ac805435c1b2c965d63ce460988e1000058e1f67881324746362baf9572 988B / 988B                                                                                                                                                    0.0s
 => => sha256:f86220e98c9d858143a68158dde675a0178854549511d9fee469767edfefc0d7 1.58kB / 1.58kB                                                                                                                                                0.0s
 => => sha256:c94784fe2595f1e1963b1dbdf525c4adfdab767569f19de7ef3da81c756e65dd 6.99kB / 6.99kB                                                                                                                                                0.0s
 => [internal] load build context                                                                                                                                                                                                             0.0s
 => => transferring context: 248B                                                                                                                                                                                                             0.0s
 => [2/6] RUN apt-get --allow-releaseinfo-change -y update                                                                                                                                                                                    2.9s
 => [3/6] RUN apt-get install -y git make                                                                                                                                                                                                     2.7s
 => [4/6] WORKDIR /go/src/app                                                                                                                                                                                                                 0.0s 
 => [5/6] COPY google-webrisk .                                                                                                                                                                                                               0.0s 
 => ERROR [6/6] RUN make                                                                                                                                                                                                                     10.7s 
------                                                                                                                                                                                                                                             
 > [6/6] RUN make:                                                                                                                                                                                                                                 
0.133 go mod vendor                                                                                                                                                                                                                                
0.496 go: downloading github.com/google/webrisk v0.0.0-20210125163530-030b4b9756fa                                                                                                                                                                 
0.830 go: downloading github.com/golang/protobuf v1.4.3                                                                                                                                                                                            
0.832 go: downloading github.com/rakyll/statik v0.1.7                                                                                                                                                                                              
0.837 go: downloading golang.org/x/net v0.0.0-20210224082022-3d97a244fca7
0.892 go: downloading google.golang.org/protobuf v1.23.0
1.013 go: downloading golang.org/x/text v0.3.3
1.618 go build -o wrserver ./vendor/github.com/google/webrisk/cmd/wrserver
6.107 path: /usr/local/go/pkg/tool/linux_amd64/compile: signal: illegal instruction
10.64 make: *** [Makefile:2: wrserver] Error 1
------

 1 warning found (use docker --debug to expand):
 - FromPlatformFlagConstDisallowed: FROM --platform flag should not use constant value "linux/amd64" (line 1)
Dockerfile:6
--------------------
   4 |     WORKDIR /go/src/app
   5 |     COPY google-webrisk .
   6 | >>> RUN make
   7 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c make" did not complete successfully: exit code: 2

View build details: docker-desktop://dashboard/build/desktop-linux/desktop-linux/xl56lu76jldpfd5k5pvywlm2l

What did you expect to see?

I expected compilation to succeed.
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Jan 25, 2025
@mleonhard-notion
Copy link
Author

Related: #71339

@mleonhard-notion
Copy link
Author

The workaround is to switch Docker's VMM setting to "Docker VMM" or "QEMU (Legacy)".

@seankhliao
Copy link
Member

1.20 isn't a supported version

@seankhliao seankhliao closed this as not planned Won't fix, can't repro, duplicate, stale Jan 25, 2025
@andrew0li
Copy link

andrew0li commented Apr 18, 2025 •
edited
Loading

I'm getting this consistently on go 1.23 when using golang:1.23 Docker image:

 => [builder 9/9] RUN --mount=type=cache,target=/workspace/.cache/go-build     CGO_ENABLED=0 GOOS  42.0s
 => => # google.golang.org/grpc/internal/resolver/dns: /usr/local/go/pkg/tool/linux_amd64/compile: signa
 => => # l: illegal instruction
 => => # go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp/internal/semconvutil: /usr/local/
 => => # go/pkg/tool/linux_amd64/compile: signal: illegal instruction
 => => # cel.dev/expr: /usr/local/go/pkg/tool/linux_amd64/compile: signal: illegal instruction

I've tried it with and without mounting the build cache. It also only happens statistically on different packages, if I run the build a few times I eventually get to finish. I'm on an M4 Pro, here's my env when using --platform=linux/amd64

# go env
GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/root/.cache/go-build'
GOENV='/root/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='local'
GOTOOLDIR='/usr/local/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.23.8'
GODEBUG=''
GOTELEMETRY='local'
GOTELEMETRYDIR='/root/.config/go/telemetry'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/dev/null'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build3937495830=/tmp/go-build -gno-record-gcc-switches'

@randall77
Copy link
Contributor

This sounds like a bug in the amd64->arm64 emulation on Macs. We never get illegal instruction faults from the toolchain on amd64 hardware. Not sure what we can do about that.

Aside: why do lots of people run the amd64 toolchain on arm64 hardware? There's an arm64 toolchain, works just fine...

@andrew0li
Copy link

That's a good point... we've been building binaries for remote amd64 VMs on Mac, but also through an amd64 docker build image on our Macs.

I'm trying an arm64 docker build image, but setting GOOS and GOARCH and all seems to work fine?

@randall77
Copy link
Contributor

Setting GOOS/GOARCH to any value will work regardless of the architecture of the toolchain. (Excepting cgo, which can be trickier.)
You can build a aix/ppc64 binary using a darwin/arm64 toolchain, and vice versa.

@cespare
Copy link
Contributor

cespare commented Apr 18, 2025

@randall77

Aside: why do lots of people run the amd64 toolchain on arm64 hardware? There's an arm64 toolchain, works just fine...

Historically, because the amd64 version was the one you got by default on macos so it was way too easy to end up with an amd64 toolchain on apple silicon macs: #52487 (comment)

Nowadays you are forced to choose between amd64 and arm64 but people might still sometimes pick the wrong one. See also #59010.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
compiler/runtime Issues related to the Go compiler and/or runtime.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@cespare @randall77 @gopherbot @seankhliao @mleonhard-notion @andrew0li