Skip to content

v0.15.0 — M2 Agent Skills · M3 closed-loop contract · M4 Quality Knowledge Base

Choose a tag to compare

@Siddardth7 Siddardth7 released this 19 Aug 22:05
· 46 commits to main since this release
952ad5c

v0.15.0 — M2 · Agent Skills · M3 · Closed-loop contract · M4 · Quality Knowledge Base

Three milestones ship together. M2 puts an Agent Skills layer over the MCP server shipped in
0.14.0 and proves it on four CLI hosts. M3 turns the four engines into one closed loop over a
project directory on disk — FMEA → Control Plan → SPC → back to the FMEA — with MSA gating what
the SPC numbers are worth. M4 stands up the private Quality Knowledge Base the M5 research
skill will query: a licensed corpus ledger, an OCR-aware ingestion pipeline, chunking/embedding
with a file-backed vector store, a citation eval set, and a fail-closed private read path.

Added

  • OCR fallback for image-only corpus PDFs (#335, M4-2c). pypdf reads an embedded text
    layer and nothing else, so the corpus' image-only scans extracted to zero characters.
    quality_database_app/ocr.py adds an Ocr Protocol seam — the same shape as M4-3's
    Embedder — and a page whose text layer falls below MIN_WORDS is retried through it. The
    default NullOcr recognizes nothing, so CI's behaviour is byte-identical to pre-#335 and
    the gate still needs no system binary. The real backend (ocr_tesseract.py, PyMuPDF
    rasterization + tesseract) sits behind the optional ocr extra, is excluded from the
    coverage gate, and is a hand-run — its output is not bit-stable across tesseract versions or
    DPI, which is stated rather than papered over. OCR supplies text only: page numbers still
    come from pypdf's page tree and extraction_quality still comes solely from the SME-reviewed
    ledger (RULE 1), so no code-derived confidence signal is ever invented. Also fixes a
    junk-page leak in extract_pdf that let below-threshold pages through as corpus records.

  • PDF text extraction for the corpus pipeline (#329, M4-2b). extract_pdf.py is the PDF
    counterpart to segment.py, emitting the same Segment shape so everything downstream of
    extraction is format-agnostic. Three narrowings, all SME-locked: one Segment per page,
    never per heading
    (raw PDF text carries no Markdown structure, so clause is None rather
    than guessed from a "first line is a heading" heuristic); page numbers are pypdf's own page
    index, 1-indexed
    , which is strictly more reliable than the Markdown path's footer-marker
    regex; and extraction never reclassifies a row — a freshly extracted PDF whose ledger
    extraction_quality is still not-extracted produces low_confidence records until the SME
    reviews the ledger by hand. A follow-up SME extraction review then marked 15 text-layer PDFs
    clean (PR #337).

  • Private corpus storage + access boundary (#286, M4-5). The corpus store is what M4-2/M4-3
    already write — the gitignored apps/quality_database/.corpus_out/ (corpus.json +
    index/), derived from the on-machine $CORPUS_ROOT tree — so M4-5 creates no new
    location and adds no dependency
    . storage.py is the one sanctioned read path:
    load_index() fails closed, so a missing index is an error rather than a silently empty
    store, and M5-2's query endpoint is meant to import it and nothing else. Two limits are
    stated rather than implied: this module cannot yet enforce single-reader access (M5-2 does
    not exist to be gated), and it adds no auth code because a local filesystem read has no
    caller identity — when M5-2 puts it on the network it should reuse M1-8's shared-secret
    bearer posture (#267), not invent a second scheme. tests/test_no_corpus_content.py is the
    machine check that keeps corpus text out of version control.

  • Citation eval set + RAG metrics (#285, M4-4). evalset.py / build_gold_set.py build a
    gold set from the existing CITATIONS.tsv ground truth; retrieval_metrics.py scores
    Recall@k / MRR / nDCG and generation_metrics.py scores the answer side, with eval.py
    running whichever half it was given the machinery for. What CI can conclude from this is
    bounded on purpose
    : FakeEmbedder's sha256-derived vectors carry no semantic similarity,
    so CI asserts plumbing — report shape, item count, filters honoured — and never a numeric
    quality threshold; real retrieval numbers are a local hand-run with the real embedder.
    never-ship items score 0 recall by design (the licensing gate makes their chunk
    unreachable) and are scored instead by refusal_correctness, where the right answer is a
    refusal. write_report exists so M5-4 can gate on a report produced that way.

  • Chunking, embedding and vector store (#284, M4-3). The retrieval half of the Quality
    Knowledge Base, on top of M4-2's ingested corpus. quality_database_app/chunk.py turns
    corpus records into retrievable Chunks — one record is one chunk by default, since
    M4-2 already split the sources on Markdown heading boundaries; only a record longer than
    MAX_CHARS (3000) splits, on paragraph boundaries first, with a hard OVERLAP_CHARS (200)
    window as the last resort for a single over-long paragraph. Every chunk carries its record's
    standard / clause / page / serving_flag / license_class verbatim, so every hit is
    citable and the licensing context never gets lost in the index. embed.py defines the
    Embedder protocol and a deterministic offline FakeEmbedder — the only embedder CI
    runs
    ; a real local model (fastembed/ONNX) sits behind the optional embed dependency
    group in embed_fastembed.py, excluded from the coverage gate, and real embedding is a
    hand-run. store.py is a file-backed VectorStore (vectors.npy + metadata.json,
    brute-force cosine scan) — no server, no ANN index, and numpy (already a quality-core
    dependency) as the only addition. search() filters on standard / source_id / region
    and excludes never-ship chunks by default: the flag stays in the data so the gap is
    auditable, while the query layer is safe by default. index.py wires corpus file -> chunks
    -> vectors -> saved index, with no default embedder so a real run can never silently produce
    a fake index. Re-indexing unchanged inputs is byte-identical, and the four new modules join
    the Quality Database CI gate at 100% line + branch.

  • OCR-aware ingestion + cleaning pipeline (#283, M4-2). The ledger becomes text:
    pipeline.py reads docs/CORPUS_LEDGER.tsv, skips (and logs) every row it cannot ingest,
    reads each remaining source — Markdown split on headings, PDF split on pages — and emits one
    CorpusRecord per segment with the ledger's confidence, serving flag and licence class
    carried through, so no downstream consumer can lose the licensing context. Cleaning is
    format-only (RULE 4): it normalises whitespace and layout artefacts and never rewrites
    content. Re-running over unchanged inputs produces a byte-identical output file, the same
    write discipline as quality_core.project.io.write_artifact. The output is corpus-derived
    text, so it is gitignored and never committed, per M4-1's "the corpus is private; only our
    derivations are public".

  • Corpus sourcing + licensing ledger (#282, M4-1). The foundation of M4 (Quality Knowledge
    Base): every source the RAG may draw on is enumerated once, with its licensing class and an
    explicit rule for what a generated answer may do with it. docs/CORPUS_LEDGER.md is the
    policy — the corpus stays private and out of the repo (only metadata and this project's own
    derivations are committed), serving flags are tiered by source type (public ISO/SAE/NIST
    standards quote; licensed AIAG/VDA handbooks and textbooks paraphrase-and-point, locator
    only; the project's own derivations serve), and a quoted excerpt in a generated answer is
    capped at 50 words / 2 sentences with a locator. docs/CORPUS_LEDGER.tsv is the
    manifest, one row per (source, region) so a partially-usable source splits — the AIAG &
    VDA FMEA Handbook's clean DFMEA prose is paraphrase-and-point while its OCR-mangled PFMEA
    Occurrence/Detection tables (#256) are never-ship. Known gaps are recorded rather than
    hidden: the AIAG SPC edition mismatch (4th Ed. cited, 2nd Ed. held), AIAG FMEA-4 cited but
    not located, and the Western Electric / Nelson possible-primaries still logged as
    reproductions. tests/test_corpus_ledger.py makes the completeness claim machine-enforced —
    no blank cells, closed vocabularies, unique keys, and the policy's cross-field rules
    (quote implies a public licence class, serve implies own derivation, not-held implies
    nothing to serve). Corpus-presence checks skip on CI, mirroring the MSA/FMEA citation tests.

  • Loop orchestration + SECOM worked example (#281, M3-6). run_project_loop(project_root)
    — a new MCP tool plus the skills/project-loop/ Agent Skill over it — sequences the four M3
    arrows in dependency order (fmea/fmea.json → control-plan/plan.json → spc/config.json →
    spc/msa-gate.json → feedback/spc-to-fmea.json + candidate Actions back on the FMEA) and
    returns exactly what each arrow's own tool returns, so the one-call path and the four
    individual calls can never disagree. Two boundaries are documented rather than fudged: the
    loop does not produce spc/results/*.json (no arrow does — those come from a prior SPC
    charting session and are read as a precondition, and with none on disk the feedback step
    legally no-ops), and a characteristic whose MSA gate says block still produces feedback,
    because nothing ties the two arrows together today — read the gate alongside the feedback
    rather than assuming the loop filtered on it. examples/secom-quality-loop/ is the runnable
    end-to-end example on the SECOM case study.

  • MSA → SPC gate (#280, M3-5). spc_app/msa_gate_arrow.py reads spc/config.json and
    msa/gage-rr.json and writes spc/msa-gate.json — one row per monitored characteristic
    saying how far its SPC result may be trusted. No gate policy lives in the arrow; the verdict →
    status mapping is msa_gate.gate_for's. spc/config.json is required (no monitored
    characteristics means a mis-ordered pipeline, not an empty answer, so its ProjectError
    propagates), while msa/gage-rr.json is optional — a project that has not run a Gage R&R
    yet is a normal state and every row gates as "no study on file" (warn); a file that exists
    but is malformed still raises. The gate file is always written, even with zero rows,
    because "nothing is configured to monitor" is a stable state worth recording.

  • SPC → FMEA occurrence feedback arrow — the living FMEA (#279, M3-4). The leg that closes
    the loop. spc_app/fmea_feedback_arrow.py reads every spc/results/<characteristic>.json
    plus the plan and the FMEA, and writes feedback/spc-to-fmea.json with full provenance
    (chart, rule set, violating points, CAPA prompt). It proposes, it never applies:
    Cause.occurrence is never touched — the proposal lands as a quality_core.schema.action.Action
    with o_after=suggested_occurrence and status=Open, the schema's own before/after mechanism
    for the AIAG optimization loop, and a human decides. Because the loop's input is never written
    by its output, a re-run on the same violations recomputes the same suggestion: oscillation is
    structurally impossible
    , so there is deliberately no anti-oscillation guard to maintain. The
    occurrence math stays in fmea_feedback (AIAG-4 / SAE J1739, ASSUMPTIONS_LOG RULE 10).

  • Control Plan → SPC monitoring config (#278, M3-3). spc_app/project_arrow.py reads
    control-plan/plan.json and writes spc/config.json, one SPC monitoring config per plan
    characteristic. No SPC selection logic lives in the arrow — chart-key normalisation and the
    spec/sample-plan pass-through are control_plan_config's. The join key is an exact string
    match on characteristic
    — no normalisation, no fuzzy match — and the same string is
    carried onto the SPC config row, so a plan row and its config row share one identifier.
    A recommended_chart of None (what build_control_plan emits today) carries through as
    chart_key=None: no exception, and no fabricated chart type. spc_app imports
    quality_core.project and never controlplan_app (audit A11, #202).

  • FMEA → Control Plan project-file arrow (#277, M3-2). controlplan_app/project_arrow.py
    is the on-disk face of the existing connector: read fmea/fmea.json, run
    build_control_plan on the RelationalFMEA it carries, write control-plan/plan.json. No
    Control Plan logic is duplicated — the mapping rules, defaults and the source_cause_id join
    key remain the connector's. Every row written carries that source_cause_id, which is what
    M3-3 and M3-4 join on straight out of plan.json, so traceability runs the length of the
    loop. A re-run overwrites in place: no merge, no run id, no history array — git is the
    history (#276).

  • Project-file schema: quality_core.project (#276, M3-1). The M3 data contract — what a
    Quality Platform project looks like on disk — defined once so every M3 arrow (#277+) reads and
    writes the same shape instead of inventing one under deadline. project/schema.py holds the
    pydantic models for project.yaml (ProjectMeta: project identity plus the characteristic
    registry with units, tolerances and a new tolerance_source provenance field) and for the five
    artifact files (fmea/fmea.json, control-plan/plan.json, spc/results/<characteristic>.json,
    msa/gage-rr.json, feedback/spc-to-fmea.json). project/io.py holds the file graph
    (ProjectPaths / discover_project, pure path arithmetic — no filesystem access) and one
    generic load/write pair (load_artifact / write_artifact, plus load_optional_artifact for
    the arrow-that-hasn't-run-yet case), raising ProjectError — a subclass of
    quality_core.io.IngestError — with a user-safe message that names the file and the problem.
    Two SME-resolved design calls are baked in: files hold current state, git is the history
    (stable slug project_id, generated_at/generated_by on every artifact, re-runs overwrite in
    place, one SPC result file per characteristic — no run-id, no history array), and one envelope
    for all five artifacts
    , so fmea.json wraps RelationalFMEA under
    {schema_version, generated_at, generated_by, fmea} rather than being a bare dump, keeping the
    domain model free of file-format fields. Import direction is respected: only fmea.json reuses a
    live type (RelationalFMEA, which quality-core owns); the Control Plan, SPC, Gage R&R and
    feedback models are independent mirrors of the app shapes, because quality_core must never
    import an app (CI audit A11, #202). Versioning is an explicit schema_version: int checked on
    load and rejected loud on mismatch — no JSON Schema files and no new validation dependency.
    Ships with a round-tripping fixture project (packages/quality-core/tests/fixtures/project/),
    the file-graph doc docs/PROJECT_FILE_CONTRACT.md, a new
    packages/quality-core/docs/ASSUMPTIONS_LOG.md recording the six design decisions, an API.md
    STABLE SYMBOLS block, and a CI Core project coverage gate at 100% line+branch. pyyaml
    becomes a hard quality-core dependency (project.yaml is the one hand-edited file in a project);
    it is pure-python, already in the lock, and not on the Streamlit chain the core dependency
    contract forbids. No engine, app or MCP behaviour changes.

  • Skill host-compatibility matrix: skills/COMPATIBILITY.md (#275, M2-6). Closes the M2
    Agent Skills layer by turning "works everywhere" from a claim into a checked matrix — each
    shipped skill (control-plan, fmea, spc, msa) × each CLI host (Claude Code, Codex CLI,
    Cursor, Gemini CLI) — with a per-host runbook and linked evidence rather than assertions.
    The Claude Code column is verified live: every skill's shipped scripts/*.py makes a
    real MCP tool call (the quality-platform FastMCP server launched over stdio,
    python -m mcp_app.server), and the four transcripts are committed to
    skills/compat-evidence/claude-code-column.txt (control-plan's row is byte-identical to its
    SKILL.md worked example). All four host columns are now verified live (SME, 2026-08-17):
    Codex CLI, Cursor, and Gemini CLI each cloned test in isolation and reproduced the exact
    reference values (rpn 240 · cpk 2.1150000000000024 · ppk 2.8284271247461983 ·
    verdict Reject · ndc 13 · "Bracket weld — Incomplete weld") for all four skills —
    evidence in skills/compat-evidence/{codex-cli,cursor,gemini-cli}-column.txt. PASS ✓ is
    scoped to the real MCP tool call; host-native SKILL.md activation is mixed and recorded
    honestly
    in the quirks table (Gemini 1/4, Codex 3/4, Cursor 0/4 in-session) — the next M2-6
    layer, not folded into a green cell. Documents real install quirks: npx skills add <repo>
    pulls the default branch (skills reach it only after promotion), the branch-ref syntax
    differs by host (#test works on Gemini, @test fails on Cursor), all hosts install to the
    agentskills.io .agents/skills/, and the shipped scripts must be run with uv run (bare
    python fails the FastMCP import). No engine or test changes — a verification artifact.

  • Control Plan Agent Skill: skills/control-plan/ (#274, M2-5). The fourth shipped skill
    on the #270 foundation, and the first whose input is another skill's output: fmea_model is
    the same RelationalFMEA object the fmea skill's fmea_run_relational takes, so the
    skill links to skills/fmea/references/mcp-tool-contract.md for the field set rather
    than redefining it — one contract, one place, no fork. SKILL.md routes the three Control
    Plan tools (controlplan_build for the plan, controlplan_recommend_chart for a chart type
    once a characteristic is classified, controlplan_source_index for the trace back to the
    FMEA cause) and spends its body on the two things a derived plan is easiest to misread on:
    the sample_plan_is_placeholder flag — true on every controlplan_build row, marking
    sample_size/frequency/reaction_plan as connector defaults needing engineering judgment
    rather than AIAG-derived values (F-10, #196, apps/controlplan/docs/ASSUMPTIONS_LOG.md
    RULE 2) — and the always-null recommended_chart, which is a deliberate Q3 decision (the
    FMEA carries no data type or subgroup size), not a gap for the skill to fill in.
    references/mcp-tool-contract.md carries all three signatures, the eleven-key row table with
    its two provenance fields, the chart rule table, the characteristic-key parity guarantee
    between controlplan_build and controlplan_source_index, the error contract and transport;
    references/controlplan-method-notes.md carries the AIAG SPC Reference Manual 4th Ed. (2005)
    chart-table provenance with RULE 1's open Xbar-R/Xbar-S (n = 9 vs 10) third-party-sourcing
    flag quoted verbatim, RULE 2's "Not a published standard" line for the placeholder fields,
    RULE 1's upper-bound paragraph on why n > 12 raises instead of naming an uncomputable chart
    (F-07, #196), and the SME-locked granularity/naming decisions. scripts/call_controlplan_build.py
    is the runnable build call, fastmcp + stdlib only. skill-lint's SKILL.md formula denylist
    was checked and needs no extension for this skill: the only metric tokens in the Control
    Plan surface are RPN and AP (already denylisted, and named only in prose here), and the
    chart names this domain returns are rule-table lookup results, not computed values with their
    own algebra. The skill carries no formula and no computation: engine decides, skill
    orchestrates. No engine, app or MCP behaviour changes.

  • MSA Agent Skill: skills/msa/ (#273, M2-4). The third shipped skill on the #270
    foundation, and the one with the smallest tool surface behind it: MSA is a single tool,
    msa_gage_rr, so the SKILL.md carries no routing table and spends its body on the problem
    that actually decides whether the answer means anything — study intake. Its checklist
    states, before any tool call, what a valid crossed study needs: a balanced parts × appraisers
    × trials design (apps/msa/docs/ASSUMPTIONS_LOG.md RULE 11, a platform inference rather than
    an AIAG statement), the 2/2/2 computability floor against AIAG's 10 × 3 × 3 optimum (RULE 12),
    the long/tidy row shape (which is not the SPC chart tools' wide subgroups), when a tolerance
    is worth asking for, when ANOVA is worth preferring over Average-and-Range, and what a
    degenerate zero-variation study returns (RULE 13). references/mcp-tool-contract.md carries
    the msa_gage_rr signature, the long/tidy request shape, the full return-key list, the error
    contract and transport; references/msa-method-notes.md carries Average-and-Range versus
    ANOVA (RULES 1, 17), the AIAG %GRR bands on both bases (RULES 7, 8), the ndc threshold with
    its platform-only sub-bands called out as such (RULE 9), the verdict matrix and AIAG's own
    caution that the guidelines are not threshold criteria (RULE 10), and the clamping edge cases
    (RULES 13, 14) — every quotation copied verbatim from the log, not re-quoted from the manual.
    scripts/call_msa_gage_rr.py is the runnable study call, fastmcp + stdlib only. skill-lint
    was hardened in the same change: its SKILL.md formula denylist now covers the MSA metric
    tokens (GRR, EV, AV, PV, ndc) alongside the FMEA and SPC ones, so the "no
    computation" rule is load-bearing for this skill too. The skill carries no formula and no
    computation: engine decides, skill orchestrates. No engine, app or MCP behaviour changes.

  • SPC Agent Skill: skills/spc/ (#272, M2-3). The second shipped skill on the #270
    foundation, and the one with the largest tool family behind it. Its SKILL.md routes a
    control-chart or capability request by the shape of the data — spc_xbar_r/spc_xbar_s/
    spc_imr for variables, spc_p/spc_c/spc_u for attributes, spc_ewma/spc_cusum for a
    small sustained shift, spc_freeze_* then spc_apply_* for Phase I/II monitoring,
    spc_detect_we_violations or spc_detect_nelson_violations for run rules, and
    spc_assess_stability then spc_capability (with spc_normality_test) for a capability
    study — while keeping the body lean: the chart-selection matrix and the EWMA/CUSUM parameter
    detail live in the references, not inline. references/mcp-tool-contract.md carries the
    request/response shape of every SPC tool, the error contract and transport;
    references/spc-method-notes.md carries the chart-selection matrix (RULES 1-4), Phase I/II
    freezing and its soft baseline gate (RULE 11), the EWMA/CUSUM Phase I estimate requirement
    (RULES 12-13), the capability preconditions with the tri-state stability gate (RULE 7), the
    method-selection and confidence-interval rules (RULES 9, 14) and the Cpk interpretation tiers
    (RULE 6), each cited to apps/spc/docs/ASSUMPTIONS_LOG.md — including RULE 8's own
    third-party-reproduction caveat on the Western Electric and Nelson rule text, and RULE 15's
    hard prohibition on running either detector over an EWMA or CUSUM chart.
    scripts/call_spc_capability.py is the runnable capability call, fastmcp + stdlib only.
    The skill carries no formula and no computation: engine decides, skill orchestrates. No
    engine, app or MCP behaviour changes.

  • FMEA Agent Skill: skills/fmea/ (#271, M2-2). The first shipped skill on the #270
    foundation. Its SKILL.md routes an FMEA request to the right MCP tool by the shape of the
    input — fmea_score for one Severity/Occurrence/Detection triple, fmea_run for a flat
    11-column table, fmea_run_relational for a linked failure-mode model, and
    fmea_list_scales/fmea_get_scale for rating-scale text — and states explicitly that scale
    selection is presentation-only and never re-scores anything (ASSUMPTIONS_LOG.md RULE 6).
    Level-3 detail lives in references/mcp-tool-contract.md (request/response shapes for all
    five tools, the error contract, transport) and references/fmea-method-notes.md (2019
    AIAG-VDA Action Priority versus the FMEA-4 Risk Priority Number, cited to RULE 1 and RULE 7,
    including the handbook's own recommendation against an RPN-only threshold).
    scripts/call_fmea_score.py is the runnable single-triple call, fastmcp + stdlib only.
    The skill carries no formula and no computation: engine decides, skill orchestrates. No
    engine, app or MCP behaviour changes.

  • Agent Skills layer foundation: skills/ + skill-lint (#270, M2-1). A root-level
    skills/ directory (sibling to apps/ and packages/, which is what npx skills add Siddardth7/quality-platform resolves against), governed by skills/CONVENTIONS.md: the
    frontmatter contract, the description house style, the progressive-disclosure rule, the
    install path, and the invariant this layer exists to hold — engine decides, skill
    orchestrates.
    A skill calls MCP tools; it never imports or reimplements
    quality_core/<app>_app math. scripts/skill_lint.py enforces the mechanical half
    (stdlib-only frontmatter parser, name-matches-folder, description limits, a formula
    denylist over the SKILL.md body, and an engine-import denylist over scripts/*.py) and runs
    as its own Skill lint step in the CI gate job — a plain script that exits non-zero,
    mirroring the README drift check, not a new coverage surface. skills/example-skill/ is the
    copy-from template and a real working example: it scores one FMEA S/O/D triple through the
    fmea_score MCP tool over stdio, with the wire-level detail kept in
    references/mcp-tool-contract.md. No engine or app behaviour changes.