Coverage tool

Coverage tool

coverage/. A source-to-source instrumenter and a report that measure line, branch, condition and MC/DC coverage. Mojo ships no coverage tool, and the toolchain has no llvm-cov to build one on.

This is original work with no three.js lineage.

How it works

1. build_cli.mojo rewrites every covered module into coverage/build/, with a probe before each statement and around each decision. It writes a manifest of everything the probes can report.
2. The coverage tool and the suites are copied into coverage/build/ as well, and the suites run from there. Each probe writes one record to stderr. stdout is unchanged.
3. report_cli.mojo groups the records, matches them to the manifest, and prints the table. It exits with an error when anything is uncovered.

make coverage runs all three. See How to measure coverage.

Why the run happens inside the build tree

Mojo 1.1 resolves a module beside the file being compiled before it looks at any -I path. A suite run from the repository root therefore imports the real library, whatever the search path says.

The suites used to run with -I coverage/build -I . and the first -I decided. Under 1.1 that arrangement measured nothing at all and reported a clean zero, which is the worst way for a coverage tool to fail. Copying the suites in makes the instrumented copies the ones beside them.

The copies of the tool itself are never instrumented. Measuring the tool with the tool is still not attempted.

Modules

Module	Job
scanner.mojo	Find statements, decisions and loop headers in a source file.
instrument.mojo	Emit the probes. Split and and or conditions.
runtime.mojo	The probe functions the instrumented code calls.
mcdc.mojo	Reconstruct decision vectors from the ordered record stream.
report.mojo	Build the report and decide whether it is complete.
build_cli.mojo, report_cli.mojo	The two commands the Makefile runs.

Metrics

Metric	Question
Line	Did this statement run?
Branch	Did this decision go both ways? A for loop counts, including running zero times.
Condition	Did each and or or operand take both values?
MC/DC	Did each operand change the outcome on its own?

MC/DC is the masking variant. Short-circuit evaluation makes unique-cause MC/DC unreachable for most compound decisions.

Rules

• The Makefile's COVERED list is every CPU module. COVERAGE_EXCLUDE names the GPU modules, render/gpu.mojo and render/gpu_vxgi.mojo, because a kernel has no stderr.
• # pragma: no branch opts one decision out. Use it only when the other outcome is provably unreachable.
• Probes cannot reach compile-time code, and the instrumenter recognizes def and async def only.
• The tool does not measure itself.

Limits

Two threads reporting one decision at once would interleave their records. The renderer therefore defaults to one worker, and the coverage run uses it.

The capture grows with every statement run

Each statement that runs writes one record, so the capture grows with the work a suite does. A PMREM blur reads its source about half a million times, and each read ran about 150 statements. That made test_pmrem write 14 GB and run for six and a half minutes.

Keep the innermost helpers short for this reason:

• A small value type uses @fieldwise_init. Its constructor then has no body and writes no record. FloatColor and Vector3 do this.
• A helper that runs per texel or per tap is one statement where the code stays clear. Vector3.cross and the CLAMP case of wrap_index are.
• Work that is the same for every texel is done once, outside the loop. The PMREM blur finds its weights, sines, cosines and copy position once per pass.

See Benchmarks for the numbers.

Compiler hang

A Bool loop flag read after nested loops hangs the Mojo compiler. The instrumenter emits an Int counter instead. See The Mojo compiler hang.

Capture storage

The Makefile reduces each suite's stderr as it arrives, and compresses what is left. The report reads two things: the set of distinct probe payloads, and each decision's distinct MC/DC evaluations. A probe in a loop repeats both millions of times. tools/coverage_io.py's Reducer keeps each payload once, as a COVLINE: record, and each distinct evaluation once, rebuilt from its pending conditions when its decision closes. The report then sees the same payloads and the same evaluations, in the same order of first closing.

test_rasterizer writes 9.9 million records. The reduced capture holds 3,477 lines, and the report reads it in 0.02 s instead of 8 s, with an identical result. The reduction runs on the capture runners, in parallel, and the report reads the captures on one core.

The reporter reads the captures through named pipes, one suite at a time. Other output from a suite passes through unchanged, for the summary of a failed suite.