Skip to content

docs, test: harden newcomer onboarding (#194, #202, #203, #208, #209, #212)#216

Merged
dgenio merged 3 commits into
mainfrom
claude/github-issues-triage-NUpLq
May 28, 2026
Merged

docs, test: harden newcomer onboarding (#194, #202, #203, #208, #209, #212)#216
dgenio merged 3 commits into
mainfrom
claude/github-issues-triage-NUpLq

Conversation

@dgenio
Copy link
Copy Markdown
Owner

@dgenio dgenio commented May 28, 2026

Summary

Six tightly-coupled documentation and metadata fixes that share the README, pyproject.toml, and docs/ surface — bundled so the new smoke tests (#203) can guard the doc fixes (#194, #202, #209, #212) from regressing the moment they land, and so the new runtime-responsibilities page (#208) is anchored into both the README and the mkdocs nav.

Pure docs + tests + package metadata. No runtime code changes. The single touched non-test source file is chainweaver/__init__.py, where __version__ is bumped from "0.9.0""0.10.0" to match the release commit in pyproject.toml — the kind of drift the new metadata-consistency test catches.

Changes

File Issue Change
pyproject.toml #209 [project.urls] adds Documentation, Source, Changelog, Issues so PyPI sidebar / pip show link users to the right place; keywords expanded.
chainweaver/__init__.py #209 __version__ bumped to "0.10.0" to match pyproject.toml (now caught by tests/test_metadata_consistency.py).
README.md #194, #202, #208, #209, #212 New top-of-file "See it in 30 seconds" before/after narrative (#202); expanded Installation extras table (#209); "Where ChainWeaver fits in the Weaver Stack" subsection (#212); link to the new runtime-responsibilities page from the MCP Integration section (#208); paragraph + entry-script snippet explaining set_default_registry next to the inspect/viz CLI examples (#194). Two existing Python blocks marked with <!-- smoke-test: run --> so they get exercised by the new smoke test.
docs/runtime-responsibilities.md (new) #208 5 sections (when to invoke, trace persistence/redaction, side-effect tools, flows as named operations, MCP parity) + concrete handle_request example.
mkdocs.yml #208 Wires the new page into nav under "Is this for me?".
tests/test_readme_smoke.py (new) #203 Extracts opt-in <!-- smoke-test: run --> Python blocks and runs each in a fresh subprocess (with the repo on PYTHONPATH); asserts chainweaver --help lists every registered command; asserts chainweaver validate examples/double_add_format.flow.yaml exits clean from a fresh checkout; asserts all bundled examples the README points to exist; asserts the chainweaver[yaml] install note appears near the first .flow.yaml mention.
tests/test_metadata_consistency.py (new) #203, #209 Asserts pyproject.toml version ↔ chainweaver.__version__ ↔ latest released ## [x.y.z] heading in CHANGELOG.md agree, and that [project.urls] exposes the required user-navigation URLs. Uses stdlib tomllib on 3.11+ with a regex fallback for 3.10.
tests/test_runtime_responsibilities_doc.py (new) #208 Anti-drift: the page exists, has the 5 required H2 sections, includes a concrete handle_request example, the README links to it, and mkdocs.yml includes it in nav.

Why

All six issues describe distinct first-30-seconds-of-the-repo failure modes that the maintainer-triage process kept reopening (#190 / #192 / #193 / #195 / #196 / #197 closed earlier today were the concrete instances; #194 and the rest in this PR are the next batch plus the systemic guard). Shipping the README polish (#194 / #202 / #209 / #212) alongside the smoke tests (#203) means the class of drift becomes a CI failure mode from day one rather than something that has to be re-discovered by the next contributor.

The runtime-responsibilities page (#208) is the only sizable new doc; it lands here because the README's MCP-integration section now links to it, so the two issues' acceptance criteria are jointly satisfied without orphaning either page.

Testing

  • Linting passes (ruff check chainweaver/ tests/ examples/) — All checks passed
  • Formatting check passes (ruff format --check chainweaver/ tests/ examples/) — 152 files OK
  • Type checking passes (python -m mypy chainweaver/) — 125 files, 0 issues
  • All existing tests pass (python -m pytest tests/ -v) — 1107 passed
  • New tests added for new functionality — 23 new tests in 3 new test modules

Reproducible verification:

$ python -m ruff check chainweaver/ tests/ examples/
All checks passed!

$ python -m ruff format --check chainweaver/ tests/ examples/
152 files already formatted

$ python -m mypy chainweaver/ tests/
Success: no issues found in 125 source files

$ python -m pytest tests/ -q --no-cov
........................................................................ [97%]
........................................................................ [100%]
1107 passed in 10.08s

Targeted run of the new suites:

$ python -m pytest tests/test_readme_smoke.py tests/test_metadata_consistency.py tests/test_runtime_responsibilities_doc.py --no-cov -v
...
23 passed in 1.30s

Related Issues

Closes #194 — README inspect example assumes a programmatically-registered registry.
Closes #202 — Rewrite README landing path around a concrete before/after use case.
Closes #203 — Add README and docs snippet smoke tests to prevent onboarding drift.
Closes #208 — Add runtime responsibilities page.
Closes #209 — Improve package metadata and install docs.
Closes #212 — docs: link ChainWeaver into the Weaver Stack ecosystem map.

Follows directly from the triage in #214's predecessor sweep (six already-fixed issues closed today: #190, #192, #193, #195, #196, #197).

Checklist

  • Code follows project conventions (see AGENTS.md and docs/agent-context/) — anti-drift tests mirror tests/test_cli_docs.py's pattern; smoke-test subprocess plumbing mirrors tests/test_cookbook_examples.py's pattern; commit message follows the repo's docs, test: … conventional style.
  • Public API changes are documented — none; this PR is docs + tests + package metadata only. __version__ bump is metadata, not API.
  • No secrets or credentials included.

Scope notes (Mode B)

User explicitly authorized Mode B + "one PR for all" + "I don't care about retro compatibility" + "new dependencies if worthy". I did not add any new runtime dependency (the test deps pytest / pyyaml are already in [dev]). Mode B latitude used only for: (a) bundling all six issues in one PR per the user's explicit request; (b) fixing the __init__.pypyproject.toml version drift discovered while writing the metadata consistency test, which is in #209's explicit scope ("Ensure package version metadata remains consistent with chainweaver.__version__"); (c) wiring the new docs page into both the README and the mkdocs nav rather than leaving it floating.

Tradeoffs / risks

🤖 Generated with Claude Code


Generated by Claude Code

…212)

Six tightly-coupled documentation and metadata fixes that share the
README, pyproject.toml, and docs/ surface — bundled so the smoke tests
(#203) can guard the doc fixes (#194, #202, #209, #212) from regressing
the moment they land, and so the new runtime-responsibilities page
(#208) is anchored into both the README and the mkdocs nav.

#194  README inspect example now explains the in-memory registry seam,
      links to `docs/cli.md#programmatic-registration`, and shows a
      working `set_default_registry` entry-script snippet next to the
      `inspect`/`viz` CLI lines.
#202  README opens with a concrete `search → extract → validate →
      format` before/after narrative (problem in one sentence, naive
      4-LLM-call trace, compiled 1-LLM-call trace, copy-paste path)
      before the prose "Why ChainWeaver?" section.
#203  New `tests/test_readme_smoke.py` extracts `<!-- smoke-test: run
      -->`-marked Python blocks (allowlist by inline marker per the
      issue's "small allowlist mechanism" criterion) and runs each
      in a fresh subprocess; covers `chainweaver --help` (asserts every
      registered command appears) and `chainweaver validate` against
      the shipped example, and checks the bundled examples listed in
      the README all exist.  New `tests/test_metadata_consistency.py`
      asserts pyproject.toml ↔ chainweaver.__version__ ↔ latest
      CHANGELOG release heading agree.
#208  New `docs/runtime-responsibilities.md` (5 sections + concrete
      `handle_request` example) covering host-app responsibilities:
      when to invoke, trace persistence/redaction, side-effect tools,
      flows-as-named-operations, MCP parity.  Linked from the README
      MCP section; wired into `mkdocs.yml` nav under "Is this for
      me?".  Anti-drift test in `tests/test_runtime_responsibilities_doc.py`.
#209  `pyproject.toml` `[project.urls]` now exposes Documentation,
      Source, Changelog, and Issues alongside Homepage/Bug Tracker so
      the PyPI sidebar and `pip show chainweaver` link users to the
      right places; `keywords` expanded.  README §Installation now
      lists every shipped optional extra (`yaml`, `otel`, `mcp`,
      `contrib`, `langchain`, `llamaindex`, `test`, `docs`, `dev`)
      with the transitive deps each pulls.  Also bumps
      `chainweaver.__version__` from "0.9.0" → "0.10.0" to match the
      release commit in `pyproject.toml` (caught by the new
      metadata-consistency test).
#212  New README "Where ChainWeaver fits in the Weaver Stack"
      subsection placing this repo on the deterministic-execution
      seam of the broader stack (weaver-spec / contextweaver /
      agent-kernel / lessonweaver), with the existing
      `weaver-stack` extra placeholder called out as the future
      install seam.

All four validation commands clean:
- ruff check chainweaver/ tests/ examples/             → All checks passed
- ruff format --check chainweaver/ tests/ examples/    → 152 files OK
- python -m mypy chainweaver/ tests/                   → 125 files, no issues
- python -m pytest tests/ -q --no-cov                  → 1107 passed

Closes #194, #202, #203, #208, #209, #212.
Copilot AI review requested due to automatic review settings May 28, 2026 14:32
Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Performance Alert ⚠️

Possible performance regression was detected for benchmark 'ChainWeaver microbenchmarks'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 1.25.

Benchmark suite Current: 19378b5 Previous: 530174a Ratio
compiled_total_ms_n2_llm100_tool0 0.18995399994992113 ms 0.14817599998195874 ms 1.28
compiled_overhead_ms_n2_llm100_tool0 0.12493199994878523 ms 0.08674199997926735 ms 1.44
compiled_overhead_ms_n5_llm500_tool50 0.36212600002727413 ms 0.21608299999797964 ms 1.68

This comment was automatically generated by workflow using github-action-benchmark.

CC: @dgenio

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens newcomer onboarding by updating top-level docs and package metadata and adding anti-drift tests that exercise README snippets, verify metadata consistency, and ensure the new runtime-responsibilities doc stays wired into the docs site/navigation.

Changes:

  • Add smoke/anti-drift tests for README runnable snippets, CLI help/validate, metadata/version consistency, and the runtime-responsibilities doc presence/structure.
  • Expand README onboarding content (30-second narrative, installation extras table, Weaver Stack positioning) and add the new docs/runtime-responsibilities.md page wired into mkdocs.yml.
  • Update pyproject.toml keywords/URLs and bump chainweaver.__version__ to match the package version.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
tests/test_runtime_responsibilities_doc.py Adds anti-drift assertions for the new runtime-responsibilities doc page and its references.
tests/test_readme_smoke.py Adds opt-in README fenced-python execution + CLI and bundled-example smoke checks.
tests/test_metadata_consistency.py Adds version/URL consistency checks across pyproject.toml, runtime __version__, and CHANGELOG.md.
README.md Updates onboarding narrative, installation extras guidance, Weaver Stack context, and links to runtime responsibilities; adds smoke-test markers.
pyproject.toml Expands keywords and adds [project.urls] entries for user navigation.
mkdocs.yml Adds the runtime responsibilities page to the published nav.
docs/runtime-responsibilities.md Introduces a host-responsibilities guide for embedding ChainWeaver (invocation, traces, side effects, MCP parity).
chainweaver/__init__.py Bumps __version__ to 0.10.0.

Comment thread docs/runtime-responsibilities.md Outdated
Comment thread docs/runtime-responsibilities.md Outdated
Comment thread docs/runtime-responsibilities.md Outdated
Comment thread docs/runtime-responsibilities.md Outdated
Comment thread pyproject.toml Outdated
Comment thread tests/test_readme_smoke.py
Comment thread docs/runtime-responsibilities.md Outdated
… + smoke contract

- docs/runtime-responsibilities.md:
  * Tighten the Redaction bullet: `RedactionPolicy` is dict-only (`redact(dict)`),
    hosts must walk `result.execution_log` themselves and apply the policy per
    step.  Previous wording suggested `RedactionPolicy` could redact
    `ExecutionResult` / `StepRecord` directly (Copilot review thread, line 57).
  * Rewrite the host example to match real APIs: drop the non-existent
    `redactor.redact_execution_result(result)` call (line 182) and the wrong
    `redact_fields=...` constructor (line 197).  Walk `execution_log`, call
    `replace()` on each step with `redact()`-ed `inputs`/`outputs`, instantiate
    with `redact_keys=frozenset({...})` — these match `chainweaver.log_utils`.
  * Replace "compiling tool chains into flows" → "compiling sequences of tool
    calls into flows" so the page does not contradict the enforced domain
    vocabulary ("flow", never "chain"/"pipeline" — see review-checklist.md,
    line 110).
  * Grammar fix: "an `dry_run: bool`" → "a `dry_run: bool`" (line 86).

- pyproject.toml: drop "pipeline" from `keywords` — same domain-vocabulary rule.
  PyPI sidebar should not advertise terminology the repo forbids internally
  (line 22).

- tests/test_readme_smoke.py: tighten the smoke contract so the module
  docstring's "no traceback on stderr" promise is true.  The test now also
  fails if `stderr` contains "Traceback (most recent call last):" even when
  the script exits 0 — catches scripts that swallow exceptions with
  `traceback.print_exc()` (line 19).

Validation: ruff lint + format + mypy + pytest (1107 passed) on Python 3.11.
Phase-3 follow-ups from the in-memory audit on #216 (no comments were
posted on the PR by Phase 2; these address findings A1, N1, N2):

- "See it in 30 seconds" output block (A1, V3-adjacent): the prose
  output is a stylized condensed view rather than verbatim
  `python examples/simple_linear_flow.py` stdout (which prints
  timestamps and the structured step log).  Add a one-paragraph
  caption above the fenced block explaining the relationship, so the
  values stay accurate and the marketing intent stays intact without
  overclaiming format.

- Installation extras table (N1): add a `chainweaver[weaver-stack]`
  row so the table matches `pyproject.toml`'s
  `[project.optional-dependencies]`.  The extra is a placeholder
  today (no transitive dep), but omitting it from the README hides a
  user-visible seam.

- Weaver Stack ecosystem table (N2): drop the `#107` reference from
  the Routing row so the table cross-references exactly the issues
  the PR description's "Scope notes (Mode B)" enumerates
  (#91 / #106 / #89 / #210).

Validation: ruff lint + format + mypy + pytest (1107 passed).
@dgenio dgenio merged commit b517c6c into main May 28, 2026
15 checks passed
@dgenio dgenio deleted the claude/github-issues-triage-NUpLq branch May 28, 2026 17:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment