Repository navigation
Example Go SQL cancel race
A small Go program that exercises a known race between cancelling a query and Rows.Scan in database/sql (CVE-2025-47907, GO-2025-3849; fixed in Go 1.24.6). It is built three ways so that a buggy toolchain, a fixed release and a "only the fix applied" toolchain can be compared. It is the example that reached its counted runs, so it is the best one for learning what a count of hits does and does not tell you.
Files: examples/go-sql-cancel. Full write-up: docs/showcase-go-sql-cancel-race.md.
Build the three toolchains, run the workload natively on each, see the assisted control fire on the buggy one and stay quiet on the other two, and understand how the Hermit runs in the write-up were set up and how to read their numbers.
In the affected releases (before go1.23.12, and go1.24.0 through go1.24.5), cancelling a query's context while Rows.Scan was copying a value could close the rows without waiting for the copy. The connection could then be handed to the next waiting query, whose driver could write into a buffer the first Scan was still reading. The workload uses a fake driver that reuses one row buffer per connection (legal for a driver, but it means an early release can overwrite a row still being read). There is no real database.
The property is go.sql.scan_returns_own_row, an always: every Scan that returns without error must return the row of its own query, not another query's and not a mixture. The other six sites in the report are coverage markers or harness invariants; only that one decides the outcome.
These come from the example's own README, and you should carry them with any number you take from it.
- The bug is known in advance. The defect, its fix and the conditions that expose it were public before the workload was written.
-
The workload is targeted. The driver, the one-connection pool, the four workers and the cancellers were chosen because they put the race in reach. It is not a generic test of
database/sql. -
The positive control is assisted.
-mode=yieldscans into a value that copies in two halves around a sleep. That widens the window on purpose. A result in this mode says the property and the detector work, and says nothing about how often the unassisted race is reached.-mode=stdis the unassisted workload.
| Arm | Toolchain | Role |
|---|---|---|
| go1.24.5 | the official release, unpatched | contains the defect |
| go1.24.6 | the official release | fixed release (it also contains six other commits, including runtime changes) |
| go1.24.5 + fix | go1.24.5 built from source with only the non-test sql.go and convert.go hunks of the upstream fix |
fixed in database/sql only; every other file identical to go1.24.5 |
The third arm exists because go1.24.6 changes more than the fix.
-
curl,shasum, and about 1.2 GB of disk. The build is C-free (CGO_ENABLED=0). - To run the helper scripts as written: macOS arm64.
scripts/fetch-go.shandscripts/build-patched-toolchain.shdownload and unpack thedarwin-arm64archives. On Linux you download thelinux-amd64archives yourself (their checksums are inPINS) and unpack them by hand; building go1.24.5 plus the fix also needs the go1.24.5 source archive andpatches/fix-only.patch. - For the Hermit part: Linux x86-64 with performance counters, the pinned Hermit (Installation) and the
hairspringCLI.
Copy the example out of your clone so that it can write its .cache directory:
cp -R examples/go-sql-cancel ~/go-sql-cancel && cd ~/go-sql-cancelscripts/fetch-go.shfetch go1.24.5.darwin-arm64.tar.gz
fetch go1.24.6.darwin-arm64.tar.gz
fetch go1.24.5.src.tar.gz
ok go1.24.5.darwin-arm64.tar.gz 92d30a67...
ok go1.24.6.darwin-arm64.tar.gz 4e29202c...
ok go1.24.5.src.tar.gz 74fdb09f...
This took about 8 seconds. Each archive is checked against the sha256 in PINS; a cached archive that does not match is deleted and fetched again.
scripts/build-patched-toolchain.sh
scripts/check-patched-diff.shThe build ran for about 75 seconds. The check prints:
ok differing files are exactly:
src/database/sql/convert.go
src/database/sql/sql.go
That output is what lets you call the third arm a controlled comparison: the only differences from stock go1.24.5 are the two files of the fix. Toolchains are unpacked under .cache/toolchains/; you use one without touching your PATH by setting GOROOT and GOTOOLCHAIN=local.
export GOROOT="$PWD/.cache/toolchains/go1.24.5" # or go1.24.6, or go1.24.5-fixonly
export GOTOOLCHAIN=local GOPROXY=off
"$GOROOT/bin/go" test ./...internal/hs, internal/retaindrv and internal/sqlcancel pass on all three toolchains.
"$GOROOT/bin/go" build -o sqlcancel ./cmd/sqlcancel
./sqlcancel -mode=yield -q 100 # assisted positive control
./sqlcancel -mode=std -q 1000 # unassisted workloadThe program prints one fixed-width summary line on standard output and the protocol report on standard error. It exits 0 whatever the property result. The summary line counts queries, successful and failed scans, and successful scans that returned a foreign, torn or unparsable row.
What we saw, one run per cell:
| Arm | Mode | Successful scans | Torn rows |
scan_returns_own_row false / evaluated |
|---|---|---|---|---|
| go1.24.5 | yield | 399 | 368 | 368 / 399 |
| go1.24.6 | yield | 398 | 0 | 0 / 398 |
| go1.24.5 + fix | yield | 399 | 0 | 0 / 399 |
| go1.24.5 | std | 3,600 | 0 | 0 / 3,600 |
| go1.24.6 | std | 3,559 | 0 | 0 / 3,559 |
| go1.24.5 + fix | std | 3,662 | 0 | 0 / 3,662 |
How to read it. The assisted mode fires on the buggy toolchain (368 of 399 successful scans returned a mixture of two rows) and not on either fixed arm. That is the positive control doing its job: the property and the detector work. The unassisted mode on a Mac, natively, did not hit the bug in any arm in this one run each. That is not a surprise and not a result: the race is narrow, and native scheduling does not aim at it. Other counters in the report (cancel_overlapped_scan, conn_handed_to_waiter) are coverage markers that show whether the racy situation was reached at all; they are not the verdict.
The Rust conformance tests check the workload's output against the reference protocol parser:
HS_REQUIRE_GO=1 HS_GO="$GOROOT/bin/go" SQLCANCEL_EXPECT=buggy \
cargo test -p hairspring-props --test go_shim --test go_sqlcancelUse SQLCANCEL_EXPECT=clean for go1.24.6 and go1.24.5 plus the fix. They passed on all three for us. (Without HS_REQUIRE_GO=1 the tests print a skip notice and succeed if Go is missing or older than 1.24.)
Build static Linux binaries with the flags recorded in BUILD-RECORD, one per arm:
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 GOAMD64=v1 \
"$GOROOT/bin/go" build -trimpath -buildvcs=false -ldflags=-buildid= -o sqlcancel ./cmd/sqlcancelDeclare each binary as a process target in hairspring.toml (see Running-Your-Own-Software) and run hairspring test <target> --seed <n>. A failing seed is bundled and replays with hairspring replay <bundle> in the environment that recorded it. The scheduler settings of the experiment were --chaos --max-timeslice=100000 --skid-margin=25 --panic-on-rbc-overshoot --epoch=2026-01-01T00:00:00Z --base-env=minimal --env=GOMAXPROCS=2 --env=GOTRACEBACK=all. The engine's equivalent target keys are chaos = true, max_timeslice_ns = 100000, skid_margin = 25 and hermit_args. A sketch (not run):
[target.sqlcancel-1-24-5]
kind = "process"
command = ["/path/to/sqlcancel-1.24.5", "-mode=std", "-q", "250"]
chaos = true
max_timeslice_ns = 100000
skid_margin = 25
env = { GOMAXPROCS = "2", GOTRACEBACK = "all" }
trial_timeout_secs = 60
[target.sqlcancel-1-24-5.budget]
max_trials = 100An important caveat from the example's README: the counted runs in the write-up were direct Hermit runs, not engine runs (the scripts in experiment/ drive Hermit themselves). The engine's own runs of this program were checks, described next.
On one machine (Ubuntu 24.04.5, kernel 6.17.0-42-generic, Intel Xeon E5-1410 v2) under one pinned Hermit build, the property site was false at least once in 934 of 1,996 rated runs on go1.24.5 (seeds fixed in advance; Wilson 95% interval [44.61%, 48.99%]; four runs were Hermit failures and are outside the rated runs) and in 0 of 2,000 on each of go1.24.6 and go1.24.5 plus only the fix, for the same 2,000 seeds. The first 100 hit seeds in seed order were each confirmed by three direct reruns, by the engine at the same seed (in a different Hermit environment) and by bundle replay.
Read that with the labels above: bug known in advance, targeted workload, environment pinned; go1.24.5 unassisted and built with the protocol shim; go1.24.6 compared at arm level only; go1.24.5 plus the fix a patched toolchain. These are counts of runs in one setup. They are not a rate of defects, and not a claim about any other driver, Go version or machine. The write-up also explains one place where its own outcome table and its reading of it disagree, because Hermit itself stopped in four of the unpatched runs.
How the hits were counted is what you would do with your own program: each seed is one trial; a trial is a hit if the property was false at least once; trials where Hermit itself failed are not rated; and the interval is a Wilson 95% interval over the rated trials. Sweeps-and-Confirmation explains each of those.
-
internal/retaindrv: the fake driver that reuses a buffer.internal/sqlcancelandcmd/sqlcancel: four workers on a pool limited to one connection, each query cancelled by its own canceller while the worker scans.internal/hs: a Go shim for the guest protocol, kept unchanged because the recorded binaries were built from it. The maintained SDK, derived from it, is insdks/go(Guest-SDKs-by-Language). -
patches/fix-only.patch,PINS,BUILD-RECORD: how the arms are pinned and recorded. The hashes inBUILD-RECORDwere produced on darwin-arm64. -
scripts/also has scripts that run and tabulate native and race-detector cells on macOS; the race-detector cells need cgo.
Delete the copy (rm -rf ~/go-sql-cancel), which holds about 1.2 GB of downloads and unpacked toolchains (about 190 MB of archives and about 1 GB of toolchains on our Mac). In your clone nothing was written.
docs/showcase-go-sql-cancel-race.md (bug, workload, machine and flags, counting rules, confirmation), and the example's README. Next: Examples-Overview.
Understand
- What-is-Hairspring
- Concepts-and-Vocabulary
- How-Deterministic-Simulation-Works
- Sweeps-and-Confirmation
- Limits-and-Honesty
Use
- Installation
- Your-First-Run
- Running-Your-Own-Software
- Guest-SDKs-by-Language
- Examples-Overview
- Hairspring-in-CI
- Operate-a-Server
- Upgrading-and-Versions
- Troubleshooting
Reference