Skip to content

fix(noise): expression_in_range is f32 arithmetic - residual 9.5e-7 to exactly 0 (#162) - #165

Merged
wormeyman merged 1 commit into
mainfrom
fix/expression-in-range-f32
Aug 5, 2026
Merged

fix(noise): expression_in_range is f32 arithmetic - residual 9.5e-7 to exactly 0 (#162)#165
wormeyman merged 1 commit into
mainfrom
fix/expression-in-range-f32

Conversation

@wormeyman

Copy link
Copy Markdown
Owner

First conversion from the #162 audit, and the pattern it establishes is more useful than the one function.

The change

expression_in_range evaluated in f64 and rounded once. The noise machine works in f32 registers. Rounding every step takes the residual from ~9.5e-7 to exactly 0 on all 404 committed oracle samples (three sweeps: 121 + 121 + 162). The f64 form matched only 285 of them.

The formula was never wrong - the precision of the intermediate steps was the entire error. Same class of fix as fastApprox's per-operation rounding in #164.

Why it went unnoticed

The spec asserted worst < 8e-3 against an actual worst of 9.5e-7. That ceiling is ~8400x looser than the real error, so it endorsed essentially anything. This is exactly the pattern #162 was filed about, and it is worth noting that the number was not arbitrary - 8e-3 is the elevation f32-coordinate floor, borrowed wholesale into a spec for a pure interpolation builtin that has no coordinate pipeline at all.

The three sweeps now assert f32 equality with toBe, and there is a guard asserting the f64 arithmetic fails the fixture, so the exact assertions cannot quietly go vacuous later.

Two assertions moved from precision 10 to 6 - read this before calling it a loosening

They compare against exact decimals (-2, -1.0, 1.25), but intermediates like 0.5 - 0.6 and 1 - 1.2 are not representable at f32, so the correct answer is now -2.000000476837158. The exact decimal is the wrong target for an f32 computation.

Asserting the exact f32 result instead would mean recomputing f32(5 * f32(f32(1) - f32(1.2))) inside the test - re-implementing the function and then checking it against itself. So those two stay formula-shape guards (which axis drives the min, that pmax=inf does not clamp), and bit-exactness lives in the oracle sweeps above them, which are now exactly 0.

Scope

This changes shipped tile-autoplace output (sand-1's coastal term). pnpm run verify exits 0 - 200 files / 1728 tests.

Audit context (#162 stays open)

Two findings from classifying the 40 tolerance specs so far, both worth having on record:

  • Most of them are better disciplined than the issue assumed. aux, temperature, elevationNauvis, vulcanusRocks and others already state their residual mechanism and calibrate the ceiling just above a measured worst. They are not lazy tolerances.
  • But the mechanism they all cite does not survive a direct test. They attribute the residual to "the game rounds coordinates to f32 before sampling basis while we evaluate in f64". Rounding the input coordinates changes nothing (aux 1.010e-5 -> 1.015e-5), and rounding the per-octave coordinates inside multioctaveNoise makes it 27x worse (1.17e-4 -> 3.23e-3). So the residual downstream of multioctave is not understood well enough to eliminate, and those specs are not convertible today. That is a real bound on what test: only 6 of 93 oracle-reading specs compare f32-exact - which is how the fastApprox bug survived a year #162 can deliver, and it is now measured rather than assumed.

…e-7 to EXACTLY 0 (#162)

The formula was already right; the precision of its intermediate steps was the
whole error. The noise machine evaluates in f32 registers, and reproducing that
takes the residual to exactly 0 on all 404 committed oracle samples (three
sweeps: 121 + 121 + 162). The f64 form matched only 285 of them.

Same class of fix as fastApprox's per-operation rounding in #164.

The spec's ceiling was the thing that hid this. It asserted `worst < 8e-3`
against an actual worst of ~9.5e-7 - a ceiling ~8400x looser than the real
error, which would have accepted almost any regression. The three sweeps now
assert f32 equality with `toBe`, plus a guard that the f64 arithmetic FAILS the
fixture, so the exact assertions cannot quietly become vacuous.

Two hand-derived assertions moved from precision 10 to 6, and this is not a
loosening: they compare against exact decimals like -2 and -1.0, but
intermediates such as `0.5 - 0.6` and `1 - 1.2` are not representable at f32, so
the game's own answer is -2.000000476837158. The exact decimal is the wrong
target for an f32 computation. Asserting the exact f32 result instead would mean
recomputing `f32(5 * f32(f32(1) - f32(1.2)))` in the test - re-implementing the
function and checking it against itself - so those stay formula-SHAPE guards
(which axis drives the min, that pmax=inf does not clamp) while bit-exactness
lives in the oracle sweeps.

This changes shipped tile-autoplace output (sand-1's coastal term). pnpm run
verify exits 0: 200 files / 1728 tests.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DutxVPSqj12PtwNE46RtSs
@wormeyman
wormeyman merged commit 5d8936b into main Aug 5, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant