Repository navigation
CodeWitness 1.0.24
CodeWitness 1.0.24 fixes four defects found by pointing the tool at a tree that is not its own — a polyglot
project with a Python front end and a Rust crate in a subdirectory. A crate whose Cargo.toml sits below the
project root is now found and built instead of reported as "no Cargo.toml was found"; one language's build no
longer blocks requirements in the languages that built; and two Python rules stop reporting method definitions
and docstring prose.
What changed
-
A Cargo manifest below the project root is found, and every Cargo command names it. The adapter asked one
question —Files.isRegularFile(root.resolve("Cargo.toml"))— and Cargo itself resolves no manifest above its
working directory, sormath/Cargo.tomlwas invisible while the crate's.rsfiles were still detected: a
Rust project was reported and then refused. Detection now searches a bounded depth (6) below the root, skipping
target,.git,.codewitness,node_modules,vendorand.cargo— and alsofixtures/testdata, because
a repository's sample crates are inputs to its tests, not code the run verifies. The shallowest manifest wins,
deterministically (depth, then path). Its directory becomes the build root, and crate name, edition, toolchain,
src/andtests/are read from there. Build, tests, coverage, mutation, generated tests and the dependency
inventory all run against it via--manifest-path, so a report's commands stay runnable from the project root.
A tree with several manifests and none at the root is stated rather than glossed over: the report names the
crate that was built and the manifests that were not part of the run's evidence. -
One language's build no longer blocks requirements in the languages that built.
blockedForblocked per
language only when it could attribute a requirement to one — attribution being a requirement id in a source
file or a declaredpaths/tests. A requirement written in prose has no language, and the fallback then
consulted the merged build, so on a tree where Python built and Rust did not, every requirement came back
BLOCKED— including "The Python front-end suite passes" — while the report's own limitation text said the
opposite. Such a requirement is now blocked only when nothing in the tree built; with something built it is
judged on the evidence that exists, and its verdict names the languages it could not draw on and says how to
have it judged per language next time. -
A build that was never attempted is no longer merged into a failed build.
mergeBuildsforced
attempted(true)on the merged result and counted a language whose build was never attempted as a failure,
which is how one run reportedbuild failed with exit code -1 … language(s) whose build failed: [rust]while
Rust's own per-language note saidbuild not attempted (no Cargo.toml was found).languageSummaries
compounded it by printingbuildAttempted: truefor a language whose result saidattempted() == false. The
merged result now reports attempted as any-language-attempted and success over the languages actually
attempted, each language summary carries its own flag, and the limitation text separates "failed to build"
from "no build was attempted". -
CW-PY-SEC-002no longer fires on a definition namedeval. The rule was
(?<![\w.])eval\s*\(|(?<![\w.])exec\s*\(matched against a raw line: the lookbehind sparedself.eval(, but
indef eval(the character before the name is a space, so a method definition read exactly like a call to
the builtin. On one project that was nineHIGHsecurity findings and not one call. The name a
def/async def/classline defines is now blanked before the rule table sees it. -
CW-PY-BUG-002no longer fires on docstring prose, and no longer treatsis True/is Falseas defects.
Three of its five findings on that project were English sentences inside docstrings ("1.0is 1 and2.5is
5/2", "…a negative is 0") and two werex is Falsechecks. Two separate mistakes: the analyzer matched every
line of a file with no notion of strings, comments or docstrings — it was the only one of the sixteen that
did; the others all have acodeOnlyview — and the pattern listedTrue/Falseas identity-comparison
literals althoughis True/is False/is Noneare the form PEP 8 recommends, and the runtime
SyntaxWarningit cites applies to numbers and strings. The Python analyzer now tracks string literals and
comments line by line, including triple-quoted strings across lines, and discards any match that begins
inside one — for every rule, not just this one. The rule keeps only the literals Python actually warns about,
and its message says thatis None/is True/is Falseare not reported.
What this does not fix
-
One crate per run, when the tree holds several manifests and none is at the project root. A Cargo
workspace declared at the root is built as a whole, as Cargo builds it; a tree with sibling crates is not.
docs/status.mdstates the limit and the report names the crates that were not built. -
Requirement → language attribution is unchanged. It still comes from a requirement id appearing in a
source file, or from a declaredpaths/tests. The new behaviour makes an unattributed requirement honest —
it is no longer blocked by a language that is not its own — but it cannot make it attributable. A requirement
that names its files gets a per-language verdict; one that does not is judged on whatever evidence exists,
and its verdict says so and says how to fix the mapping. -
The run-level build summary is still one line for the whole run. It succeeds only when every build that ran
succeeded. What changed is that a language nobody attempted to build no longer counts as a failure; the
per-language record, with its ownattempted/succeededpair, is in the language summaries and the evidence
ledger. -
The identity rule is now narrower than a style linter.
is True/is Falseare not reported at all. A
project that wants them flagged should use a linter; CodeWitness reports what Python warns about. -
The Python rule table is still a line-oriented heuristic table. The string and comment tracking is per
line and does not attempt a full parse, so a pathological literal — a raw string ending in a backslash, say —
can still confuse the spans. Every rule in the table gained the same view, so a wrong span suppresses or
admits a finding rather than crashing the phase. -
Python existing-test execution still depends on the project interpreter. Whether pytest and
coverage.py
are importable by the interpreter CodeWitness picks decides whether the suite and coverage can be measured;
that is unchanged and reported per run.
Verified for this release
- Full reactor suite: 1,784 tests, 0 failures, 0 errors, 0 skipped — 21 more than 1.0.23's 1,763: 14 for the
Rust manifest work (NestedCrateTest,RustMutationProfileTest, fourRustProjectDetectorTestcases) and 7
for this release's engine and Python rules. - Every new guard was proved to fire, each against the pre-fix code and each restored afterwards:
- Against the pre-fix detector, eight of the new Rust cases fail — including the exact message the defect
produced,no Cargo.toml was found, so mutants cannot be compiled; - against the pre-fix engine, the aggregate-gate case fails with
R-001 … status=BLOCKEDand the
never-attempted case fails on bothbuildAttemptedand the merged build's success; - against the pre-fix Python rules, three cases fail with the reported findings reproduced exactly —
def eval(self, text, steps=False):andasync def exec(self, text):reported asHIGHsecurity findings,
if width is False:reported as a defect, and prose in a trailing comment reported twice.
- Against the pre-fix detector, eight of the new Rust cases fail — including the exact message the defect
- Measured on a copy of the project that produced the reports (never its working tree):
CW-PY-SEC-002
went from 9HIGHfindings to 0 andCW-PY-BUG-002from 5MEDIUMto 0, whileCW-PY-REL-003(62) and
CW-PY-TEST-001(2) were unchanged — the rules that should still fire still do, and a realeval()call,
x is "literal"and a credential literal are still reported. - End to end on a polyglot fixture (Python that builds, a nested Rust crate that does not): before, every
requirementBLOCKEDand the runBLOCKED; after, R-001VERIFIEDfrom an executed scenario while Rust
failed, R-002UNVERIFIED, runPARTIALLY_VERIFIED, exit0, and the limitation naming Rust as the language
whose build failed. - Self-check against the built jar (
./codewitness verify . --requirements selfcheck/requirements.yaml --config selfcheck/codewitness.yaml --no-mutation --max-duration 30m): all seven ground rulesVERIFIED,
1,784 tests executed and passed, 0 findings — five suppressed with their reasons — coverage 71.3% line
(25,775/36,141) and 59.6% branch, 30 resolved dependencies with 0 advisories from OSV, and exit0in
1 m 50 s. RunCW-2026-11-08-BYFSS.selfcheck/README.mdquotes this run. ./codewitness versionprintscodewitness 1.0.24.- Rust end-to-end suite (
-Pe2e): 2/2.
Process note
The aggregate-gate defect is worth recording because of how it was found: not by a failing test but by a
report contradicting itself. Run CW-2026-11-08-W1KWN said every requirement was BLOCKED and, in the same
limitation list, that "requirements written in a language that did build were still verified". Both sentences
were produced by the same run, and the wrong one was in the verdicts. A sentence in a report is a claim, and
a claim that is not tested is a claim that drifts: blockedFor implemented per-language blocking and the
limitation text described it, but nothing exercised the case where a requirement has no language at all, so the
two halves were free to disagree.
The two Python rules failed the same way from the other direction. The rule table had tests, but every one of
them fed it clean code: a call, a comparison against a string. Neither rule was ever run against a definition
named eval or a docstring that uses the word "is", which is what a real project is full of. The precision
work now starts from the two shapes a real file has that a fixture does not — a method whose name is a builtin's,
and prose that looks like code — and the analyzer gained the string/comment view the other fifteen already had.
Compatibility
- No CLI command, flag, exit code contract, report key, SARIF profile, configuration key or adapter-SPI change.
Everycodewitness.yamlkey and report field behaves as it did. - Behaviour changes, all in the direction of fewer false findings and fewer false
BLOCKEDverdicts:- a polyglot run may now be
COMPLETED/PARTIALLY_VERIFIEDwhere it wasBLOCKED, and the exit code follows; CW-PY-SEC-002andCW-PY-BUG-002produce fewer findings; finding fingerprints are unchanged in shape, so
baselines and suppressions keyed on them keep working, and a previously suppressed false positive simply
stops being reported;- reports from a nested crate record
--manifest-pathin their commands and report the crate directory as the
build root; buildAttemptedin a language summary now saysfalsefor a language nobody built.
- a polyglot run may now be