Measuring a QR codes real size from the depth map

Measuring a QR code's real size from the depth map

Solving a planar marker's pose from one camera has a well-known catch: a small code close to the lens and a large code far away produce the same image. Monocular vision gives you the shape of the ray bundle, not the scale.

The universal workaround is to tell the system the marker's physical edge length in advance. It works, and it fails in ways that are irritating to diagnose.

Why hard-coded size goes wrong

• The print scaled. "Fit to page" shrinks a 200 mm code to 187 mm. Nobody measures. Every derived distance is now 6.5 % off.
• The laminate has a border. Is the size the code itself, or the quiet zone around it, or the sticker?
• Mixed deployments. Big codes on building entrances, small ones on posts — now the size is per-marker configuration that must be kept in sync with the physical world by a human.
• The failure is silent. A wrong size does not throw. It produces a confident pose at the wrong distance, and the symptom appears far away in the alignment as a mysterious scale error.

Every one of these is a data-entry problem masquerading as a computer-vision problem.

The device already knows

An ARCore-class phone in a WebXR session with the depth feature exposes a per-frame depth buffer. If you can find the code's four corners in the image — which the detector already does — you can look up the depth at those corners and unproject them into 3D camera space.

Four 3D points on a physical square. The edge length falls out of the distances between them. No configuration, no assumptions about print scale.

estimateQrSizeFromDepth does exactly this, and createQrSizeAccumulator handles the part that makes it usable: one measurement is not enough. Depth maps are noisy, especially at edges — which is precisely where a QR code's corners are, since a printed code often sits on a surface at a depth discontinuity. Accumulating measurements across frames and taking a robust central estimate converts a jittery per-frame number into a stable one.

Why edges are the hard case

It is worth understanding why the naive version disappoints. Depth sensing on phones is low-resolution relative to the colour camera and is interpolated. At a boundary between a surface and whatever is behind it, a single depth sample can belong to either — so a corner sample may report the wall behind the sign rather than the sign.

The implementation here samples the four corners and adds a few interior samples — used not to replace the corners but to strengthen a planarity check, which is what catches a corner sample that has bled onto the background. Add agreement across frames before trusting a size, and the noisy per-frame number becomes a usable one.

What it buys

• Print at any scale. Someone can reprint the code larger for a new location and nothing needs updating anywhere.
• Size stops being something a human must get right — if they leave it blank. The precedence is worth knowing and is the opposite of what you might assume: an authored physicalSizeM on the marker descriptor wins, and the depth measurement is the fallback used when none is given. So the measurement removes the obligation to supply a number; it does not override a number someone supplied wrongly.
• A consistency check for free. If the measured size disagrees wildly with the expectation, something is wrong: wrong marker, bad depth, or a reflection. That is a rejection signal you would not otherwise have.

The limits

• No depth, no measurement. Devices or sessions without the depth feature fall back to a configured size. The feature must degrade, not crash.
• Distance. Depth accuracy falls off with range; a code across a plaza will not measure well. That is acceptable, because a code that far away is not a good pose source anyway.
• Glass and gloss. Reflective surfaces produce nonsense depth. A shiny laminated code in sunlight is a genuinely hard case.

The general shape

This is a recurring pattern in AR work worth naming: prefer measuring the world to configuring it. Every constant a human has to type about a physical object is a constant that will eventually be wrong, and its wrongness will be silent. When the device carries a sensor that can determine the value, using it removes both the configuration step and an entire family of support questions.