Skip to content

v0.4.4: Scaffolded tasks can now fix the code they test: upgrade before verifying your own module

Choose a tag to compare

@github-actions github-actions released this 14 Sep 10:47
· 324 commits to main since this release

Added

  • --validate warns about scaffold placeholders left in a task. A file
    still carrying TODO in its requirements, criteria or interface passed every
    other check, and a run would have sent the placeholders to the agents as the
    specification.
  • The refinement loop proposes fixture rows for the criteria it adds.
    Refinement reads converged code and adds criteria about what that code could
    still get wrong, which is often an input the fixtures do not contain, and a
    criterion no row reaches yields a test that passes whatever the code does.
    refine_acceptance_criteria now runs the fixture proposer over the refined
    bar when it finishes, lists the added criteria first, and logs how many have
    no data behind them. Like propose_fixtures itself, it writes a report for
    review and never a fixture.

Fixed

  • A scaffolded task could never fix the code it was testing. --scaffold
    wrote interface.module as the file's path relative to the project, such as
    billing, but a run installs the code into outputs/agent_src/code/<task>/
    and runs pytest from the project root. So the tests imported the original
    file, the coding agent patched the copy, and no fix ever reached a test. The
    module is now outputs.agent_src.code.<task>.<file>, the path the bundled
    tasks already used. Found by an audit and reproduced end to end: a patched
    copy returning PATCHED against tests that saw ORIGINAL. A regression test
    now runs that same check. Affects every task scaffolded with 0.4.3 or earlier;
    change the module: line of one by hand to fix it.
  • The MCP qikly_scaffold tool and --scaffold named the same task
    differently
    , BILLING against BILLING_VERIFY. The suffix is now added in
    one place, so they agree.
  • Scaffold's next step now runs --validate --tasks <task>. Unscoped, it
    also checked every bundled example, which buried the one result that mattered.
  • The TODO warning matches only scaffold's own placeholders, so a
    requirement that mentions TODO comments is no longer flagged.
  • Scaffolding a module on a different drive from the project failed on
    Windows.
    os.path.relpath has no answer across drives and raises, so
    --scaffold and the MCP qikly_scaffold tool stopped with "path is on
    mount 'D:', start on mount 'C:'". A module on another drive is now named
    by its file, with its seed path written absolute. Found because GitHub's
    Windows runners are laid out that way, the checkout on D: and temporary
    folders on C:, and reproduced locally on a mapped drive.

Changed

  • The VS Code install button and the Claude Code line use uvx. The
    button wrote a bare qikly-mcp, which pip on Windows often installs off
    PATH, so the first click frequently produced a server that would not
    start. It now writes uvx --from qikly[mcp] qikly-mcp, the command the MCP
    Registry listing already uses, which needs uv and no separate pip install.
    In Claude Code the one line is claude mcp add qikly -- uvx --from "qikly[mcp]" qikly-mcp.
    The pip route stays documented in docs/mcp.md.
  • Fewer places saying the same thing. The README's MCP section is a short
    pointer to docs/mcp.md rather than a second copy of it, the troubleshooting
    page links the explanation of why runs stall at the unit stage instead of
    repeating it, and design_3_mechanism.md defines the task file's three parts
    once. The README also gains a free first step near the top,
    qikly --explain CALC_TAX, which needs no API key.
  • A place to say how trying it went. The README and the quick start link a
    Discussions thread for reports, whether a run worked, stalled or never got
    past install. The MCP docs now say what has been verified: the server is
    tested in VS Code and in Claude Code, and the setup for other hosts follows
    the standard but has not been checked end to end.
  • --scaffold writes one task file, not two. By default it writes
    <NAME>_VERIFY.yaml, the task that tests the code you already have, and
    --fresh writes the task for a new implementation of the same interface
    instead. It used to write both and ask you to delete one. It now ends with
    numbered next steps that name the input file to add and put --validate
    before the run: the old Then: qikly --tasks X failed straight away, because
    scaffold never creates input data. Its comments use the decisions versus
    consequences rule, and the landing page links the quick start on your own
    data beside the install commands. The MCP qikly_scaffold tool returns the
    same default task and names the file to save it as, so step 1 of the quick
    start also works from inside VS Code.
  • Quick start is its own page, docs/QUICK_START_ON_YOUR_OWN_DATA.md.
    The README kept its three-command demo, but everything needed to try qikly
    on your own module sat after about 2,600 words of explanation. The new page
    starts with the demo and five steps from --scaffold to a first run,
    including the one scaffold leaves to you, adding input data, and then keeps
    the command table and everything the old USING_YOUR_OWN_DATA.md had. That
    page is removed rather than left behind as a pointer.
  • The split between requirements and acceptance_criteria now has a rule
    you can apply.
    The docs said spec-level statements go in one and checkable
    edge cases in the other. That sends a measurement convention or an exemption
    into the withheld half, where it produces the stuck loop the very next
    paragraph warned about. The rule is now decisions versus consequences, with
    one test for any line: could two competent developers, given only the
    requirements, legitimately disagree about it? The authoring guide, the
    mechanism write-up, the starter task and the --scaffold output all say so.
    The starter task had followed the old rule: its criterion that a quantity of
    0 is rejected was a decision its requirements never stated, and the
    requirement is now there.
  • "Doesn't a failing test give the criteria away?" is answered where the
    objection is raised
    , in the README and on the landing page, rather than
    only in the case study.
  • The stale-name guard matches whole words, so a test named after the
    package, such as test_qikly_run, no longer fails CI. It still catches a
    path that names the old project.