Skip to content

v0.3.0

Choose a tag to compare

@linkpranay-ai linkpranay-ai released this 27 Jul 12:24
· 61 commits to main since this release
5c34295

2 PRs since v0.2.0. Headline: two case studies close the "does CEP help retrieval only, or does it
help a downstream generative task too" evidence gap, and two research-informed protocol additions
(a corpus-scaling primitive, an ingested-content-safety rule) came out of a deliberate review of
external context-engineering prior art.

Added

  • Consumer-output-quality case studies (2): case-studies/consumer-benefit-user-stories/ runs a
    real, already-built consumer skill (spw-write-user-story, vendored read-only for reproducibility,
    outside .github/skills/ and never installable — same demonstrated-against-not-adopted treatment
    as the FastAPI/Textual/Open5GS corpora) twice against the same real feature: once with an approved
    context package, once from a bare ticket-sized ask with no package. Scored both against a rubric
    fixed before either output was read (traceability, hallucination, actor coverage, NFR specificity,
    testability, convention adherence), each finding tagged Measured or Inference. Two features, two
    unrelated domains (Textual UI/accessibility, Open5GS telecom Diameter stack in C) — the second run
    checks the first wasn't a fluke, and the win is sharper there (18 real citations vs. 0; 0
    hallucinated mechanisms vs. 1; 5 distinct actors vs. 2 generic; full convention structure 7/7 vs.
    none). Both cases also trace the compounding benefit of a package's [Context: ...] tags past the
    user-story file itself into design/review, planning, test-writing, and implementation stages.
    EVIDENCE-METHODOLOGY.md gains a 4th evidence category (consumer-output-quality, alongside
    token-efficiency, fallback-relevance, and the naive-keyword-search baseline) and a bare-ask-baseline
    definition.
  • Progressive-disclosure skeleton mode: md_index.py skeleton reformats an already-built
    index.json into a doc_id + heading/clause-ID tree only, no body text — zero re-parsing, since
    the index never stored body text. Measured 6.7x compression on the real telecom-what-l1-demo
    corpus (39,076 → 5,862 bytes). Wired into what-l1-fallback-query.md/how-l1-fallback-query.md as
    an opt-in first look at a large/unfamiliar corpus before a full query; zero behavior change for
    existing configs. Closes the scaling primitive ROADMAP.md item 13 was missing.
  • Ingested-content injection guardrail: new MUST-level PROTOCOL.md §2.2 — ingested What-L1/
    How-L1/MCP-mirrored content is always data to cite, never instructions to follow. Backed by a new
    SHOULD-level heuristic script, scripts/content_safety_scan.py (a narrow, literal pattern list,
    deliberately scoped to avoid false-positiving on ordinary "shall"/"must" spec language),
    informational only, never auto-blocking. Surfaced as a new non-blocking line in
    ult-context-generate/SKILL.md's Step 9 human-review-gate template. CONFORMANCE.md §4 records it
    as a SHOULD, not the enforcement mechanism itself.
  • references/design-scratchpad-glossary.md: a plain-English index of every
    CONTEXT-ENGINEERING-DESIGN.md D<N>/§<N> label cited across this repo, so those citations
    resolve without the private, unpublished source document. Linked from CONTRIBUTING.md's existing
    citation note and from the two most heavily-cited skills (ult-context-generate, ult-repo-layout).
    At the time this glossary shipped, ROADMAP.md item 15 logged the fuller pre-1.0 citation cleanup
    it stopped short of as deferred; see the ### Fixed entry below — that cleanup is now done.
  • approved_by trust signal: replaces the old human_approved: true|false boolean across the
    context-package mechanism. Now a list — empty until a human approves (Step 9), then exactly one
    {actor: human:<id>, at: <ISO8601>} entry is appended; v1 enforces at most one (multi-approver
    review is explicit future scope). Backed by a new hard-gate script, scripts/validate_approved_by.py
    (flags a missing field, more than one entry, or a malformed entry — exit code 1 on failure, unlike
    content_safety_scan.py's informational-only exit 0), run as part of Step 9's approval flow.
  • Three other externally-sourced ideas (session-level runtime compaction, a comment-anchored
    live-studio UI, multiple ranked search modes/a multi-bundle registry) were reviewed and rejected,
    each for a reason tied to this repo's own prior decisions — logged to ROADMAP.md "Not on this
    roadmap" rather than dropped silently.

Fixed

  • Private-document citation leak: skills and scripts across the repo cited unpublished sibling
    design docs by filename (CONTEXT-ENGINEERING-DESIGN.md and two others) or bare D<N>/D-0NN
    decision-log labels a reader has no way to resolve. Rewrote every site to state the substance
    inline instead of pointing at a document that doesn't exist in this repo, and updated
    CONTRIBUTING.md's citation note to match. Closes out ROADMAP.md item 15. Also removed internal
    EngineeringOS work-package IDs (CEP-DP-001*) that had leaked into public docs, scripts, and
    tests — that governance mechanism is private and was never meant to be user-facing.

Known limitations (disclosed, not regressions)

  • A residual set of unsignposted-but-technically-resolvable citations (real in-repo section headers,
    or labels already covered by references/design-scratchpad-glossary.md) were intentionally left
    untouched — see ROADMAP.md §14's "Glossary-pointer consistency polish" item for the remaining,
    non-blocking follow-up.
  • content_safety_scan.py's pattern list is narrow and literal by design; it is not a general
    prompt-injection detector and makes no such claim.
  • Consumer-output-quality evidence is two case studies, one skill, not a blind trial — ground truth
    for traceability checks is knowable because both real features already exist upstream.
  • approved_by is a breaking rename of human_approved, with no migration shim. Acceptable
    pre-1.0 since no automated consumer ever read either field; the three real case-study runs
    recorded before this rename keep their original human_approved: true values as accurate history
    (see EVIDENCE-METHODOLOGY.md's field-name-drift note) rather than being rewritten to match.