What is wrong
scripts/coverage.sh names the binaries it profiles by hand. It names three
families:
TEST_EXE="$OUT/tests/morph_tests" # positional
LADDER_TEST_EXE="$OUT/examples/common/ladder_common_tests" # -object
$OUT/examples/${_rung}/ladder_${_rung}_tests # -object, per rung
The tree builds nine test executables:
| target |
profiled by coverage.sh? |
morph_tests |
yes |
ladder_common_tests, ladder_<rung>_tests |
yes |
morph_net_tests |
no |
morph_offline_sqlite_tests |
no |
morph_qt_tests |
no |
morph_net_qt_interop_tests |
no |
morph_soak |
no |
morph_bench |
no (correctly — a benchmark) |
Each of the four missing test binaries calls apply_coverage(...) under
AF_COVERAGE, so they are instrumented and they do write .profraw that
llvm-profdata merge folds into merged.profdata. But llvm-cov maps counters
through the binary, and a binary passed via neither the positional argument
nor -object contributes no regions. The profile data is produced and then
silently dropped.
What it costs, measured
include/morph/net contributes zero files to the uploaded report:
$ jq -r '[.files[].name|split("/")[0:3]|join("/")]|unique|.[]' <codecov totals> | grep ^include/
include/morph/core include/morph/detail include/morph/forms
include/morph/journal include/morph/offline include/morph/qt
include/morph/render include/morph/session include/morph/util
include/morph/net is absent. It is not a stub: socket_backend.hpp (28.8 kB)
and socket_server.hpp (12.0 kB) are 955 lines between them, plus net/detail/
(tcp_socket, ws_handshake, ws_frame, sha1, base64), and the subsystem
holds 42 of the library's 148 throw sites — the second-largest concentration
after core. tests/net/ has eight test files driving exactly those headers.
So the network stack is tested and scored by nothing.
Two more numbers are suspect for the same reason, and should be re-read rather
than believed:
include/morph/offline/sqlite_offline_queue.hpp — 57.04%, the worst file
in the library, while morph_offline_sqlite_tests (its dedicated suite) is
unprofiled.
include/morph/qt — 13 lines total in the report, while morph_qt_tests is
unprofiled.
Why this is the third time
This is the same defect as morph#141 (rungs 2-4 shipped without ever being added
to coverage.sh, leaving ~15k lines outside the number) and morph#179
(coverage.sh's hand-copied rung list had drifted, so ledger and lims
contributed nothing while codecov.yml's ledger component scored a set of
files no report contained). Both were fixed by removing a hand-maintained list —
the rung loop now reads examples/rungs.txt.
The test-executable list was never given the same treatment, and coverage.sh's
own comment predicts this precisely:
Nothing fails when a rung is forgotten -- the script runs, the report uploads,
and the figure is simply computed over a shrinking fraction.
The constraint
morph_bench must stay out (a benchmark is not a test), and morph_soak is a
judgement call — it is a test binary but a long-running one that no ordinary
coverage leg runs. The fix is not "profile every executable"; it is "derive the
list from something that cannot drift, and state the exclusions".
Also: adding four binaries will move every number in codecov.yml, in both
directions — net appears from nothing, and offline/qt should rise. Every
target in that file was set from a measured ceiling, so those measurements have
to be retaken, not adjusted.
Verification status: reproduced
Codecov API, branch master: zero files under include/morph/net; nine
add_executable targets found by grep -rn add_executable tests/ --include=CMakeLists.txt;
apply_coverage confirmed present in tests/net/CMakeLists.txt,
tests/offline_sqlite/CMakeLists.txt and tests/qt/CMakeLists.txt;
TEST_EXE/OBJECT_ARGS assignments read from scripts/coverage.sh on master.
Not verified: that adding the four binaries actually raises the three
suspect numbers. It should, but a coverage run is the only proof and I did not
run one. It is possible sqlite_offline_queue.hpp's 57% is real and its suite
is thin — that outcome would be a different and more serious finding, not a
disappointment.
What would close this
coverage.sh derives its -object list from the build tree or from CMake
rather than from hand-written names, on the same principle that fixed
morph#179. Any deliberate exclusion (morph_bench, and morph_soak if it
stays out) is named in the script with its reason.
include/morph/net appears in the uploaded report with a real percentage.
- Something fails when a new coverage-instrumented test executable is added
and not profiled — in the spirit of scripts/check_rung_filters.sh, which
already does exactly this for the codecov components a script cannot read.
- The three suspect numbers above are re-measured and
codecov.yml's affected
targets are re-derived from the new ceilings, with the old and new figures
both recorded.
Blocks: every other coverage ticket in this sprint. A target set against
today's denominator is a target set against the wrong tree.
What is wrong
scripts/coverage.shnames the binaries it profiles by hand. It names threefamilies:
The tree builds nine test executables:
coverage.sh?morph_testsladder_common_tests,ladder_<rung>_testsmorph_net_testsmorph_offline_sqlite_testsmorph_qt_testsmorph_net_qt_interop_testsmorph_soakmorph_benchEach of the four missing test binaries calls
apply_coverage(...)underAF_COVERAGE, so they are instrumented and they do write.profrawthatllvm-profdata mergefolds intomerged.profdata. Butllvm-covmaps countersthrough the binary, and a binary passed via neither the positional argument
nor
-objectcontributes no regions. The profile data is produced and thensilently dropped.
What it costs, measured
include/morph/netcontributes zero files to the uploaded report:include/morph/netis absent. It is not a stub:socket_backend.hpp(28.8 kB)and
socket_server.hpp(12.0 kB) are 955 lines between them, plusnet/detail/(
tcp_socket,ws_handshake,ws_frame,sha1,base64), and the subsystemholds 42 of the library's 148
throwsites — the second-largest concentrationafter
core.tests/net/has eight test files driving exactly those headers.So the network stack is tested and scored by nothing.
Two more numbers are suspect for the same reason, and should be re-read rather
than believed:
include/morph/offline/sqlite_offline_queue.hpp— 57.04%, the worst filein the library, while
morph_offline_sqlite_tests(its dedicated suite) isunprofiled.
include/morph/qt— 13 lines total in the report, whilemorph_qt_testsisunprofiled.
Why this is the third time
This is the same defect as morph#141 (rungs 2-4 shipped without ever being added
to
coverage.sh, leaving ~15k lines outside the number) and morph#179(
coverage.sh's hand-copied rung list had drifted, so ledger and limscontributed nothing while
codecov.yml'sledgercomponent scored a set offiles no report contained). Both were fixed by removing a hand-maintained list —
the rung loop now reads
examples/rungs.txt.The test-executable list was never given the same treatment, and
coverage.sh'sown comment predicts this precisely:
The constraint
morph_benchmust stay out (a benchmark is not a test), andmorph_soakis ajudgement call — it is a test binary but a long-running one that no ordinary
coverage leg runs. The fix is not "profile every executable"; it is "derive the
list from something that cannot drift, and state the exclusions".
Also: adding four binaries will move every number in
codecov.yml, in bothdirections —
netappears from nothing, andoffline/qtshould rise. Everytarget in that file was set from a measured ceiling, so those measurements have
to be retaken, not adjusted.
Verification status: reproduced
Codecov API, branch
master: zero files underinclude/morph/net; nineadd_executabletargets found bygrep -rn add_executable tests/ --include=CMakeLists.txt;apply_coverageconfirmed present intests/net/CMakeLists.txt,tests/offline_sqlite/CMakeLists.txtandtests/qt/CMakeLists.txt;TEST_EXE/OBJECT_ARGSassignments read fromscripts/coverage.shonmaster.Not verified: that adding the four binaries actually raises the three
suspect numbers. It should, but a coverage run is the only proof and I did not
run one. It is possible
sqlite_offline_queue.hpp's 57% is real and its suiteis thin — that outcome would be a different and more serious finding, not a
disappointment.
What would close this
coverage.shderives its-objectlist from the build tree or from CMakerather than from hand-written names, on the same principle that fixed
morph#179. Any deliberate exclusion (
morph_bench, andmorph_soakif itstays out) is named in the script with its reason.
include/morph/netappears in the uploaded report with a real percentage.and not profiled — in the spirit of
scripts/check_rung_filters.sh, whichalready does exactly this for the codecov components a script cannot read.
codecov.yml's affectedtargets are re-derived from the new ceilings, with the old and new figures
both recorded.
Blocks: every other coverage ticket in this sprint. A target set against
today's denominator is a target set against the wrong tree.