Repository navigation
v0.4.7: Register the MCP server with one command, and every generated test names the criterion it came from
Added
-
Each generated test names the acceptance criterion it came from. Test
generation is asked to end every test's docstring withCriteria: 3, or
Criteria: nonefor a test written from the requirements, with requirement
indices on their ownRequirements:line.qikly.traceabilityreads those
back out of the suite source withast, the run summary carries
criteria_coverage(schema 6, additive, still pools with 3, 4 and 5), the
HTML report gains a Criteria coverage section, and
python -m qikly.orchestrator.tuning.trace_criteriareports the same for
suites already on disk, offline and free.It exists because six refinement experiments compared two suites in
aggregate and found nothing, and an aggregate cannot tell a bar that adds
nothing from a bar the generator ignores; the measured 54%-more-criteria,
6%-smaller-suite result says the second happens. Per-criterion counts make
it direct: delete a criterion, regenerate, and the change in its row prices
that criterion with no fault model in between.The label is self-reported. A test naming criterion 3 is evidence the
generator intended to cover it, not proof that it did, so read an uncovered
criterion as a lower bound and never a coverage guarantee. Three real runs
shaped the parser: models write the line in the function body as well as
the docstring, and they will put a requirement's index on theCriteria:
line unless requirements have a line of their own. A label that could mean
two things,Criteria: 3-5, is reported rather than guessed, and a test in
a class or inside another function, which the reader does not see, is named
rather than left out of the counts in silence. The marker is a comment,
# Criteria: 3, because the model writes it in the function body as well as
the docstring andCriteria: 1, 2there is a Python syntax error that stops
the file importing: on CALC_TAX, eleven criteria, that failed test
generation three times in a row and ended the run. Measured at
8/10 convergence onADAS_HEADWAYagainst 9/10 for a matched control arm
on the same ten seeds, which is one run apart and not a detectable cost. -
qikly --install-mcpregisters the server instead of printing a block to
paste. Registering qikly with Claude Code on this machine meant finding
thatclaude mcp addis not onPATH, because the CLI is bundled inside the
VS Code extension, then reading~/.claude.jsonby hand to see whether the
server was already there. That was the author, on the machine the tool was
built on, with the documentation open.It writes project-local files only,
.mcp.jsonand.vscode/mcp.json, never
a global config, and the project file is also where the setting that actually
goes wrong belongs: which folder the server treats as the project. It merges
rather than overwrites, keeps every other server and key in the file, and
backs the file up first. Running it twice changes nothing.It refuses three cases and prints the block instead: a file holding comments,
because VS Code'smcp.jsonis JSONC and a JSON round-trip would delete
them; an existingqiklyentry that differs, because somebody changed it on
purpose, with--forceto say otherwise; and a file that is not valid JSON.
It writes into the directory you are standing in, and says so when
QIKLY_PROJECT_ROOTis set to somewhere else rather than quietly using it:
on the first real use of the command that variable was left in a shell from
an earlier session and the files landed two directories away.--dry-runshows the plan, and it exits non-zero when it refused everything,
so a provisioning step can tell that apart from nothing to do. The write is
atomic, built beside the file and moved onto it, because opening the target
truncates it and a crash in that window would leave a config holding every
other server you have empty. It also refuses a symlinked config rather than
following it out of the project, and tolerates the byte order mark
PowerShell writes by default. Cursor and Codex CLI are not covered yet;
Codex needs a TOML writer the standard library does not have.One thing to know: the file is re-serialised with two-space indentation, so a
file checked into a repository may show a whitespace diff beyond the key that
changed.
Changed
--versionsays which qikly you are running. It printed a number, and
the number answered the wrong question. A flag reported as unrecognised, a
suite exercising the released package instead of the branch, an MCP server
that would not start: each came down to which install was onPATHand
which Python ran it, and a second install shadowing the one you are editing
looks identical until the path is printed. It now prints the version, the
package directory and the interpreter, stacked.--explain --htmlputs the verdict where the evidence is. The coding
agent's column now leads, since the withheld-from side is the one a reader
came to see, and "No criterion text reaches the coding agent" sits in that
column's header rather than as a banner above both. As a banner it was a
claim the reader had to carry down to the evidence. A FAILED verdict stays
at the top, because a leak is not a caption. The lede drops the character
count, which is still in the terminal output: it sat at the end of the one
sentence that has to land, and a reader who has just been told what was cut
does not also need to know how many bytes it was.