Detection duty cycles

Detection duty cycles

The instinct is to run detection every frame. The camera produces sixty frames a second; process all sixty and you will notice the marker as early as possible.

On a phone running an outdoor AR session, this is close to the worst engineering decision available, and the reasoning is worth spelling out because it generalises well past QR codes.

What a frame of detection costs

Each detection pass involves reading pixels back from the GPU, handing them to the detector, and processing them. On a mobile device, in a session that is already running AR tracking, rendering a 3D scene, holding GPS and orientation watches open and possibly writing a recording, this competes for the same budget with everything else.

Three consequences, in increasing order of severity:

• Frame rate. Detection work steals from the render budget. A dropped frame in AR is not cosmetic — it registers as the world stuttering.
• Battery. An outdoor AR experience is being used by someone who is walking, possibly for a while, probably on a phone with other plans for that charge.
• Thermal throttling. This is the one that surprises people. A phone in direct sunlight, camera and GPU running hard, hits thermal limits within minutes. The system responds by throttling — and everything degrades at once: tracking quality, frame rate, GPS duty cycle. You do not get a warning; you get an experience that is subtly worse the longer someone uses it, which is exactly backwards.

What it buys

Almost nothing, because of what is on the other side of the calculation.

(Note the sleight of hand in that opening, too: sixty is the render rate. How often a fresh camera texture actually becomes available is a separate, usually lower number, and the scheduler's own guidance targets something like 5–10 Hz for QR regardless. Reaching for the render rate when reasoning about sensor work is precisely the habit worth breaking.)

A person pointing a phone at a printed code holds it there for a second or more — they are aiming, deliberately. At 10 Hz you get ten chances at that code. At 60 Hz you get sixty chances at the same code, in which the phone has barely moved, and consecutive frames are nearly identical: correlated information, not new information.

Worse, the marginal frames are the low-quality ones. During fast motion the frames are blurred and the detections are poor — precisely when you are burning the most cycles.

The scheduler

createDetectionScheduler makes the duty cycle explicit and configurable rather than implicit in whoever wrote the frame loop. Detection runs at a target rate, and the rate is a property of the app rather than of the hardware it happens to be running on.

Making it a first-class object is also what would let the rate become adaptive. The shipped scheduler takes one interval and keeps it — there is no mode switch today — but the reason to have an object rather than a bare constant is that these policies become expressible:

• Search mode — no marker seen recently. Detect at a moderate rate; the user may be aiming at one.
• Tracking mode — a marker was just seen and its pose is stable. Detect less often. You already know where it is, and the stabiliser is doing useful work with what you have.
• Idle mode — no marker in a long while, user walking. Detect rarely, or not at all. In an experience where the code is only used for onboarding, detection can stop entirely once the alignment is seeded.

That last case is a large saving, and it is the one that argues for building the mode switch: a single fixed interval cannot express "stop looking once the alignment is seeded".

Pair it with a quality gate

Duty cycling decides how often to look. The complementary question is whether this particular frame is worth processing at all.

The framework's image-quality gate (sharpness and luminance metrics feeding a drop/retry verdict) answers it: a motion-blurred or near-black frame will not yield a good detection, so discarding it before the expensive work is pure profit.

Worth being precise, since the two components live in the same framework and it would be easy to assume they are wired together: today the quality gate runs in the recorder's capture path, not in front of the detector. Pairing them is the obvious next step rather than a description of what happens now.

The general rule

Sensor processing on mobile should be scheduled against how fast the world changes and how fast the user acts, not against how fast the hardware can produce data.

Frame rate is a supply number. The right rate is a demand number, and demand here is set by a human holding a phone up at a sign — which is slow.