Testing a sensor fusion app without going outside

Testing a sensor-fusion app without going outside

"You can't really test this, you have to go outside" is half true, and the half that is false is much larger than people assume. The useful exercise is drawing the line precisely, because everything on the near side can be tested in milliseconds and everything on the far side needs a person, a phone and weather.

What is testable at a desk

Pure mathematics. Coordinate conversions, pose composition and inversion, homography estimation, rotation orthonormalisation, quad validation, reprojection error. Numbers in, numbers out. These deserve property-based tests, not just examples: generate a random pose, project points through it, assert the solver recovers it within tolerance. A generator explores the input space more honestly than any list of cases you would think to write.

Policies and state machines. Outlier rejection, when a correction is applied and when it is deferred, hysteresis deadbands, detection duty cycling, permission sequencing, capability gating including the timeout-that-never- settles. All of these are decisions over inputs, and all are testable by feeding inputs.

Anything with an injected seam. A module that receives its clock, its sensor feed or its storage can be tested with a fake one, in Node, deterministically — and most of the pure logic here is written that way on purpose.

Except the AR layer, and it is worth admitting. Session state lives in a module-level singleton behind the session-start call and the scene accessors — convenient for consumers, and not an injected seam. Testing around it means driving the real thing, which is what the replay path and the desktop simulators exist for.

Whole pipelines, against recordings. This is the large win. A recorded session is a fixture, so end-to-end alignment behaviour can be exercised over real sensor data — real GPS noise, real odometry, real multipath — with no device present and no simulation of any of it.

The trap: tests that assert nothing

A caution learned the expensive way while building the blog pipeline for this project, and it generalises to every property test.

A property was written to assert that anything the system publishes has a valid URL slug. It passed. It was also vacuous: the input was a random string, a random string never contains a valid publication marker, so the published branch was never reached. The property asserted a fact about the empty set.

Building the input deliberately — a valid marker plus a random page name — made it fail immediately, on a real defect: a page named only of punctuation published to a broken URL.

The lesson is uncomfortable and worth internalising: a passing property test is not evidence until you have confirmed it reaches the branch it claims to guard. The same failure mode in a fusion test would be a property about "rejected outliers" whose generator never produces an outlier.

What genuinely needs the outdoors

• Real GNSS behaviour. Multipath next to a glass façade, urban canyons, cold starts, the ionosphere. Recordings capture instances; they do not let you request a new one.
• Device tracking quality. How ARCore behaves on grass, in low sun, on a hot phone, is a property of the device and the day.
• Thermal and battery behaviour. Twenty minutes into a sunny walk is a different machine than the first minute, and no desk test reproduces it.
• The perceptual question. Whether content looks stable while you walk around it is a judgement no assertion makes for you.
• Anything the headless browser cannot request. A failure only visible in the real app stays invisible to a suite that never opens it — this project learned that from a missing favicon that survived many green runs, because headless Chromium never asks for one.

The working arrangement

• Push everything you can into pure functions with injected seams, and test them hard.
• Keep a corpus of recordings as the integration layer, and grow it deliberately toward conditions you do not yet have.
• Go outside on a schedule, not only when something is broken — the outdoor pass samples the things tests structurally cannot see.
• When a bug escapes, ask whether any test could have caught it. Sometimes the answer is a missing test. Sometimes it is honestly "no test could have", and saying so is more useful than inventing a guard that would not have fired.