Repository navigation
cmd/compile: NUL byte in string literal accepted at scanner buffer boundary #81632
Description
Go version
go version go1.27.1 linux/amd64 (the code in question is unchanged on master)
What did you do?
Compiled a file that contains a raw NUL byte inside a string literal, at a particular offset:
python3 -c 'import sys; sys.stdout.buffer.write(b"package main\n\nimport \"fmt\"\n\nvar s = \"" + b"a"*8190 + b"\x00\"\n\nfunc main() { fmt.Println(len(s), s[8190] == 0) }\n")' > main.go
go run main.go
gofmt -l main.goWhat did you see happen?
The compiler accepts the file, and the NUL byte ends up in the string:
$ go run main.go
8191 true
$ gofmt -l main.go
main.go:5:8200: illegal character NUL
go vet rejects the file too ("illegal character NUL"). With the NUL at most other positions, the compiler also reports invalid NUL character. In my tests, it is missed when the NUL is the first byte read after the scanner's buffer is refilled while reading a long literal: here at offsets 8190 and 16382 into the string (2^13-2 and 2^14-2).
What did you expect to see?
go run main.go should fail with invalid NUL character, as it does for a NUL anywhere else, and as gofmt and vet do.
Cause
In cmd/compile/internal/syntax/source.go, (*source).nextch checks for NUL only in its ASCII fast path. When s.buf[s.r] is the end-of-buffer sentinel, the general path refills the buffer and decodes the byte with utf8.DecodeRune, but it only checks for utf8.RuneError and for a BOM, not for NUL. Adding the fast path's NUL check to the general path fixes it.
Activity
arbace commentedon Sep 21, 2026
A more complete picture: with
package p; var _ = "<n × 'a'>\x00", the NUL is missed for exactly n = 4075, 8190, 16382, 32766, 65534, 131070 (checked for all n < 2^17), that is, when the NUL is the first byte after the initial buffer fill or after a buffer doubling. Every other position is reported.gopherbot commentedon Sep 25, 2026
Change https://go.dev/cl/838725 mentions this issue:
cmd/compile/internal/syntax: report NUL byte read after buffer refillgriesemer commentedon Sep 28, 2026
Nice find!
prattmic commentedon Sep 29, 2026
cc @golang/compiler
gopherbot commentedon Oct 7, 2026
Change https://go.dev/cl/846225 mentions this issue:
cmd/compile/internal/syntax: report NUL byte read after buffer refillcmd/compile/internal/syntax: report NUL byte read after buffer refill