os,net,internal/poll: write and sendfile results are out of order in certain interactions with network intercepting software

[spencer-scw]

Go version

go version go version go1.26.2 darwin/arm64

Output of go env in your module/workspace:

AR='ar'
CC='cc'
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_ENABLED='1'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
CXX='c++'
GCCGO='gccgo'
GO111MODULE=''
GOARCH='arm64'
GOARM64='v8.0'
GOAUTH='netrc'
GOBIN=''
GOCACHE='/Users/spencer/Library/Caches/go-build'
GOCACHEPROG=''
GODEBUG=''
GOENV='/Users/spencer/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFIPS140='off'
GOFLAGS=''
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/pq/9v4r19_d1pl3fz77ltzn61rr0000gn/T/go-build1984811426=/tmp/go-build -gno-record-gcc-switches -fno-common'
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMOD='/Users/spencer/Documents/Repositories/qrcp/go.mod'
GOMODCACHE='/Users/spencer/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='/Users/spencer/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/opt/homebrew/Cellar/go/1.26.2/libexec'
GOSUMDB='sum.golang.org'
GOTELEMETRY='local'
GOTELEMETRYDIR='/Users/spencer/Library/Application Support/go/telemetry'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/opt/homebrew/Cellar/go/1.26.2/libexec/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.26.2'
GOWORK=''
PKG_CONFIG='pkg-config'

What did you do?

Run a minimal HTTP server bound to a non-loopback interface address (e.g. a LAN IP), serving any file larger than 512 bytes via http.ServeFile. GET requests via curl and the browser fail from malformed responses. nc and packet inspection reveal that the response is emitted out-of-order.

A script that fully reproduces the error on macOS Darwin can be found here. It doesn't reproduce on the sandbox environment because it requires LAN access and to be run on macOS (since the bug seems linked to sendfile and linux uses splice instead.)

What did you see happen?

The response bytes are emitted on the wire in this order:

[file[512:]] [HTTP/1.1 200 OK ... headers ...] [file[:512]]

curl rejects with Received HTTP/0.9 when not allowed because the first bytes on the wire are not an HTTP status line. Browsers render the bytes as garbled binary.

Additional information

I'll describe some additional investigation steps as reported by Claude:

Conditions

Variable	Reproduces?
Bound to 127.0.0.1 / loopback	No
Bound to LAN interface address (192.168.x.x, 10.x.x.x)	Yes
File size ≤ 512 bytes	No
File size > 512 bytes (tested 600 B, 29 KB, 1 MB)	Yes
HEAD request	No (response is correct)
GET request	Yes
Setting Content-Type explicitly before ServeFile	Still reproduces

The 512 boundary matches sniffLen in net/http. The loopback-vs-non-loopback split strongly suggests something specific to the real TCP / NIC path.

Workaround

Hide ReadFrom from net/http so the response writer falls back from (*net.TCPConn).ReadFrom (which invokes sendfile(2)) to a plain Write() loop. Wrap the accepted conn in a net.Conn-embedding struct:

type noReadFromConn struct{ net.Conn }

func (ln keepAliveListener) Accept() (net.Conn, error) {
    tc, err := ln.AcceptTCP()
    if err != nil { return nil, err }
    return noReadFromConn{Conn: tc}, nil
}

With this wrapper the response is correct.

Hypothesis on root cause

Looks like (*response).ReadFrom in net/http/server.go: the w.w.Flush(); w.cw.flush() sequence before rf.ReadFrom(src) isn't guaranteeing that buffered headers + the first 512 sniffed bytes reach the socket before internal/poll.SendFile issues its first sendfile(2). Loopback's in-kernel shortcut hides the race; a real NIC path exposes it.

Bisect candidate: 98b3be702b05 ("os, net, internal/poll: combine unix sendfile implementations", Oct 2024) which consolidated internal/poll/sendfile_unix.go and shipped in Go 1.25+.

Related issues

• net/http: short writes with FileServer on macos #70000 — sendfile short writes on macOS (fixed 1.24). Different symptom (truncation), same area.
• net/http: response.ReadFrom() Bypasses Content-Length Enforcement #78179 — response.ReadFrom bypasses Content-Length enforcement (FixPending 1.27). Same function, different bug.
• net/http: ResponseWriter.ReadFrom writes to HEAD response body #68609 — response.ReadFrom writes body on HEAD response (fixed 1.24). Same function, different bug.

None of these describe the headers-after-body reordering reported here.

What did you expect to see?

A standard HTTP/1.1 response: HTTP/1.1 200 OK\r\n + headers + \r\n\r\n + the file body.

curl succeeds; the saved file is byte-identical to /tmp/test.bin.

[seankhliao]

What version of macOS is this?

[spencer-scw]

What version of macOS is this?

MacOS Tahoe 26.4.1 (25E253)

[dr2chase]

@neild @nicholashusin
the bisect is https://go.dev/cl/621416

[seankhliao]

slightly simpler reproducer without net/http:

main.go

// main.go
package main

import (
	"errors"
	"io"
	"log/slog"
	"net"
	"os"
)

func handle(err error) {
	if err != nil {
		panic(err)
	}
}

func main() {
	f, err := os.Open("main.go")
	handle(err)

	addr, err := pickNonLoopbackIPv4()
	handle(err)

	lis, err := net.Listen("tcp", addr+":0")
	handle(err)
	slog.Info("listening", "addr", lis.Addr())

	go func() {
		conn, err := lis.Accept()
		handle(err)

		conn.Write([]byte("This is a message header\n"))
		io.Copy(conn, f)

		conn.Close()
	}()

	conn, err := net.Dial("tcp", lis.Addr().String())
	handle(err)
	io.Copy(os.Stdout, conn)
	slog.Info("done")
}

func pickNonLoopbackIPv4() (string, error) {
	ifaces, err := net.Interfaces()
	if err != nil {
		return "", err
	}
	for _, iface := range ifaces {
		if iface.Flags&net.FlagUp == 0 {
			continue
		}
		if iface.Flags&net.FlagLoopback != 0 {
			continue
		}
		addrs, err := iface.Addrs()
		if err != nil {
			continue
		}
		for _, a := range addrs {
			ipnet, ok := a.(*net.IPNet)
			if !ok {
				continue
			}
			ip4 := ipnet.IP.To4()
			if ip4 == nil || ip4.IsLinkLocalUnicast() {
				continue
			}
			return ip4.String(), nil
		}
	}
	return "", errors.New("no non-loopback IPv4 interface found — cannot reproduce (bug only occurs on a real NIC, not loopback)")
}

tentatively marking as a release blocker as it sounds serious?
though it has already shipped in 1.24+