Skip to content

test(dash): compare prices with a tolerance so the suite passes on arm64 - #61

Merged
OsherElhadad merged 1 commit into
rossoctl:mainfrom
itay-nakash:test/dash-arm64-price-tolerance
Aug 12, 2026
Merged

test(dash): compare prices with a tolerance so the suite passes on arm64#61
OsherElhadad merged 1 commit into
rossoctl:mainfrom
itay-nakash:test/dash-arm64-price-tolerance

Conversation

@itay-nakash

Copy link
Copy Markdown
Collaborator

TestPriceNeverReportsUnknownCostAsFree fails on darwin/arm64 at current main,
before any other change:

capture_test.go:281: cost = 0.00327; want 0.0032700000000000003
capture_test.go:284: baseline = 0.00377; want cost + 200 unique removed
                     tokens at the cache-write rate

Both sides call the same modelinfo.Price.Cost with the same arguments, and
nothing in Price or Event.Price rounds. The difference is FMA. Price.Cost is a sum
of four products:

float64(fresh)*p.Input + float64(cacheRead)*p.CacheRead +
    float64(cacheWrite)*p.CacheWrite + float64(output)*p.Output

and the Go spec PERMITS an implementation to contract x*y + z into a fused
multiply-add, which rounds once instead of twice. The contraction is applied
inconsistently between the two evaluations in the test -- the literal arguments in
the assertion fold differently from the struct-field loads inside Price -- so the
two mathematically identical sums land one bit apart, and == fails.

arm64 has an FMA instruction the compiler will use for this shape; amd64 does not
contract it. So the failure is invisible in CI (ubuntu-latest, amd64) and
reproduces for every contributor on Apple Silicon. Confirmed both ways:

GOARCH=amd64 go test -run TestPriceNeverReportsUnknownCostAsFree ./dash/   ok
native arm64, same test, -count=3                                         FAIL x3

The fix compares within 1e-12 instead of exact equality, with the reason recorded
in a comment so nobody "simplifies" it back. The property under test is the
accounting -- a cost we cannot compute must read as unknown, never as zero -- and a
1-ULP difference does not affect it. The surrounding exact assertions are left
alone: TokenAccounting is still compared with ==, and the ordering assertion
(baseline must exceed actual when tokens were removed) is untouched, so the test
still fails if the accounting logic regresses.

Test-only change, 12 insertions, 2 deletions, no production code. make lint,
go test -race ./... and go test -race -tags cg_skeleton ./... all pass on arm64
after it; before it, ./dash/ fails.

Worth noting for whoever owns pricing: the same pattern (comparing a float64
against a re-computation of the same expression) will keep producing
platform-dependent failures wherever it appears. This PR only fixes the one
occurrence that is failing today.

TestPriceNeverReportsUnknownCostAsFree fails on darwin/arm64 at origin/main
(e4ad3b5), before any local change: cost = 0.00327; want 0.0032700000000000003.

Both sides call the same Price.Cost with the same arguments, and nothing rounds.
The difference is FMA: Price.Cost is a sum of four products, and Go permits the
compiler to contract x*y + z into a fused multiply-add, which rounds once rather
than twice. The contraction is applied inconsistently between the two evaluations
here — the literal arguments in the test fold differently from the struct-field
loads inside Price — so the two mathematically identical sums land one bit apart
and == fails. amd64 does not contract this shape, so CI is green and the failure is
only visible on Apple Silicon.

Compare within 1e-12 instead. The property under test is the accounting (a cost we
cannot compute must never read as free), which a 1-ULP difference does not affect;
the surrounding exact assertions on TokenAccounting and the baseline > actual
ordering are untouched.

Separable from the rest of this branch — it fixes an upstream test on a platform
upstream CI does not run, and touches no production code.

Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Itay-Nakash <itay.nakash@ibm.com>
@OsherElhadad
OsherElhadad merged commit a981e65 into rossoctl:main Aug 12, 2026
4 checks passed
@github-project-automation github-project-automation Bot moved this from New/ToDo to Done in Rossoctl Issue Prioritization Aug 12, 2026
@itay-nakash
itay-nakash deleted the test/dash-arm64-price-tolerance branch August 12, 2026 09:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants