Skip to content

v0.8.0 — setup expect guard, db_ref, multi-repo auth detection

Choose a tag to compare

@tranquocthong tranquocthong released this 03 Aug 09:18
· 4 commits to main since this release

Five dogfooding findings from an external multi-repo, multi-database project. The first is a test-integrity bug; the rest are gaps that forced manual workarounds.

  • setup: / teardown: sql expect: was silently ignored — every pre-state guard was decoration. checklist_lib/setup.py _do_sql() captured the scalar and returned; it never looked at expect:. Meanwhile templates/CHECKLIST.yaml and references/test-rigor.md both ship the exact form (expect: CREATED # pre-state confirmed) as the recommended way to confirm a seed landed. This is worse than having no guard: a checklist written from the template looks baseline-verified, so a wrong seed lets the test run anyway and PASS for an unrelated reason. Scalar expect: is now a hard assertion that aborts the setup (mismatch → the test FAILs with the query and both values); dict expect: stays descriptive, same rule as a verify: block; in teardown it degrades to a warning like every other teardown failure. sql._check_scalarsql.check_scalar (now used by two modules). Documented in references/checklist.md. 6 new Python tests.
  • db_ref: — multi-database support, the DB-side twin of base_url_ref:. ctx["db"] was one database name for the whole run while HTTP already had base_urls + base_url_ref, so a feature spanning two services could only be SQL-verified on the near side; the far side had to be inferred from an HTTP side-channel (GET /{id}/status → 404), which is a weaker assertion than reading the row. New config.databases declares named alternates — either a plain database name (same server) or a mapping of database/host/port/user/password — and db_ref: <name> selects one on any sql step: setup, teardown, seed, a verify[] item, expect.poll, and cleanup. Fields left unset still come from db-creds.sh discovery, so a second database on the same server needs only its name. An undefined ref fails rather than falling back to the default database — a silent fallback would query the wrong server and report a green PASS. db-query.sh gained --host/--port/--user/--password (the existing -d only ever overrode the database name, which is why a service on another port was unreachable). lint-checklist.sh now rejects an undeclared db_ref or base_url_ref at lint time instead of mid-suite. 11 new Python tests.
    • Related, same class as the expect: bug: templates/CHECKLIST.yaml advertised config.db.host/port/username/password, none of which the runner reads — credentials always come from db-creds.sh. The template now declares only database: and says where the rest comes from. Precedence is deliberately unchanged: honouring those fields would let the template's ${DB_PASS:-postgres} placeholder override a correct application.yml discovery on exactly the primary supported stack.
  • detect-auth.sh ignored config.repos, so multi-repo projects got an inverted answer. The detector ran against the cwd. In a spec-flow hub that holds only the SRS/SD while the services live in sibling repos, there is no service code to fingerprint — so a Summer/APISIX project classified as a custom-Bearer one, i.e. exactly backwards, and every generated test 401s. It now reads .spec-flow/config.jsonrepos, classifies each declared repo, and reconciles: one signal wins (and that repo's own hints are forwarded); repos that genuinely disagree report CONFLICT and fall back to unknown rather than picking a scaffold that is wrong for the others; a missing repo path is reported and skipped. Single-repo projects take the unchanged path. Also fixes a latent bug this exposed — HERE was computed after cd "$ROOT", resolving a relative $0 against the wrong directory.
  • checklist-gen tagged nearly every test smoke. The rule was Edge:-prefixed test-case name → regression, everything else → smoke, so an SD whose §13.2 doesn't use that naming convention (most of them) produced an all-smoke checklist: --tag smoke ran the entire set and the smoke → regression escalation the skill documents stopped meaning anything. Now the first non-edge TC of each Flow is that flow's smoke test and every other TC is regression — one smoke test per user story. Suite tags reflect what their tests actually carry.
    • Found while testing the above: hasApiSection (/^#{2,3}\s*9(\.\d+)?\s+API/) did not match ## 9. API Design — the exact heading templates/sd-template.md emits — only the ### 9.2 API Endpoints subsection. An SD with §9 but no §9.x subsection silently classified as internal and got the live-e2e scaffold instead of an HTTP stub.
  • TODO:MANUAL-REVIEW counting had one loose copy left, and the command docs told the agent to grep by hand. 0.7.1 anchored the regex in status-report and doctor, but genSd's own stats.todoManualReview still used a line-wise /TODO:MANUAL-REVIEW/ — which matches the Pass-1 preamble banner it emits two lines earlier. Worse, commands/{ingest,resync}.md just said "count remaining markers", so the agent ran a bare grep and counted revision-history entries and sd-author's TODO:MANUAL-REVIEW remaining: 0 summary as unresolved — a clean, approved SD reported 3 outstanding TODOs and the gate blocked work that was ready. The regex now lives once, as core.countSdTodos(), used by all four call sites; the command docs give the anchored grep -cE '^> \*\*TODO:MANUAL-REVIEW\*\*' and point at the reported count instead.
  • 780 Node tests green (2 new), 65 Python tests green (17 new).