v1.5.0 — the constants held on a corpus nobody here chose

Closing every item on the "could be done, has not been" list. One of them — whether the skill fires unprompted — could not be closed, and is named as such rather than left off.

A third graded repository, and the first that is not Python

benchmarks/corpus/cobra/ is cobra v1.9.1 at commit 40b5bc1, Apache-2.0, 602 units of Go, graded by forty questions.

Four constants — search.min_coverage, search.min_concentration, COMMON_TERM, COVERAGE_FULL_STRENGTH — had been fitted on two corpora that were both Python and both carried their documentation inside the declaration, where the AST hands it over. Go writes it above, outside the unit's span, so this ruler grades the line scanner and the rule table on prose the parser has to pick up.

gate	E cobra hit@1/3/MRR	cobra silence
neither	0.100/0.200/0.146	0.000
coverage only	0.075/0.175/0.121	0.767
concentration only	0.075/0.150/0.108	0.833
both (shipped)	0.075/0.150/0.108	0.900

Not one constant moved. Both bars together give the best silence there, as on the other two corpora, and this one arrived four releases after the defaults were fixed. It is the closest thing to out-of-sample evidence this project has.

It also publishes the lowest number here: 0.075 hit@1, against 0.200 on Flask and 0.429 on this repository. The cause is not the language — Go documents in one terse sentence beginning with the identifier, which the parser captures correctly and which shares almost nothing with the words a user asks in. Cold retrieval tracks prose density, not language.

Chosen over two candidates by counting collisions with the absent ruler's subject words: cobra 8, gin 17, chi 18. It still fired the absence guard twice on the day it landed, and both questions were rewritten — never the corpus, and never an exemption.

The describe queue was an instruction to make retrieval worse

describe status reported 283 units pending. 270 were test functions, which 1.0.0 measured and rejected describing — five real answers lost and six false silences gained. Nothing in the tool said so, so an agent working the queue to the end would have made retrieval worse while believing the opposite.

New setting describe.skip: path patterns whose units are withheld from the queue. Empty by default, because the measurement behind it is n=1, and set in this repository's own rag-your-code.toml. What it withholds is counted and reported as declined, never subtracted in silence — a queue that shrinks without explanation reads as "nothing left to do". Patterns match with PurePosixPath.match, so tests/*.py does not reach tests/fixtures/**; describing the parser fixtures was measured to cost nothing.

describe import now reports the same remaining as describe export and bootstrap. Two disagreeing counts is how an agent decides the queue is unfinished and keeps asking for a batch that will always come back empty.

Describing a well-documented declaration loses ground

Every unit the measurement says to describe is now described — 314 of 601, with 287 withheld and counted. Twelve of the thirteen genuinely undescribed units were free or better. The thirteenth cost a graded question every way it was tried:

parser.py::_generic_units	C hit@1/3/MRR
left to its generated sentence	0.429/0.600/0.498
a long authored description	0.414/0.557/0.471
a short authored description	0.414/0.557/0.471

An authored description replaces the generated sentence, and that sentence is the only route by which the author's own docstring reaches the weight-3 description field — so writing one over a declaration whose docstring already says what a reader would search by demotes it to the weight-1 body.

Measured and rejected: appending the docstring after the authored text. It fixes that declaration and costs the corpus, 0.443 → 0.414 hit@1, because it lengthens all 314 description fields and BM25F normalises per field length. describe.skip accepts path::name so one declaration can be recorded instead.

The measured envelope was optimistic by more than noise

Re-measured on 10,000 synthetic units: full build 3.45 s (was 1.84), incremental 0.286 s (was 0.207), speedup 12.1x, isolated load 79.3 ms (was 45.4), inverted index 420 ms (was 117.7), RSS 72.2 MiB (was 58.7), and query 15.6 ms mean / 14.6 ms median — against a published 3.90 ms carried since before BM25F replaced the scoring it was taken under. Compact storage is unchanged at 35.6%. Three runs bound the spread.

Query latency on this repository is 0.49 ms median [0.45–0.58] at 601 units ac3ae43a33e7, p95 0.85 ms, refusal 0.016 ms, ~30x cheaper. The same measurement read 0.99 ms while a coverage run was in progress: idle is load-bearing, and nothing in the report can see which one it took.

Also

• Line coverage is published: 91% of 2,190 statements, with pytest --cov=ragyourcode as the command. Nothing is asserted on it — a coverage floor rewards tests that execute lines rather than tests that check.
• The four diagrams in docs/FLOW.md are parsed by a test. A mermaid block GitHub cannot render fails silently: a grey box, no error, nothing noticed. They shipped in 1.4.2 with no gate. tests/test_diagrams.py refuses an unquoted label carrying punctuation, an escaped newline where <br/> belongs, an arrow naming an undeclared node, and a style line pointing at nothing — and feeds each check a known-bad case, because a gate nobody has seen fail is a gate nobody has seen work.
• CI grades the third corpus and runs its Grep baseline.
• Two absent questions were rewritten so no subject word of that ruler occurs in any of the three graded repositories.

387 tests, up from 356. Corpus fingerprints: C 601 units ac3ae43a33e7, B 566616fbe1e7, A 1,572 5fd51169eacc, E 602 3eabaa705477.

Still not measured

Whether the skill fires unprompted in a fresh session. claude plugin eval grades exactly this and is gated behind an account-level early access this project does not have, so the suite cannot be run once to see whether it even loads. It is the only claim in this repository with no command behind it, and it is named here rather than left off the list.

Full entry: CHANGELOG.md · on PyPI: rag-your-code