Skip to content

v0.7.1

v0.7.1: section-slicing API for long articles
@Hosuke Hosuke tagged this 18 Apr 11:20 
Driven by 斯文·太虛間 (fo.siwen.ink) needs: 太虛大師全書 books reach
100k+ chars and the frontend was forced to re-parse #### heads itself
to build TOCs / chapter navigation. This release moves section parsing
upstream so frontend + MCP share one source of truth.

Surface:
- New tools/sections.py: parse_sections(body) → nested tree of
  {level, title, anchor, start, end, children}. Fenced code blocks are
  skipped (CommonMark §4.5: same-char closer ≥ opener length, no info
  string), so ```mermaid blocks containing ## lines never get treated
  as headings even when nested inside longer fence runs.
- New op kb_get_sections (CLI/HTTP/MCP via the unified registry) and
  new HTTP route GET /api/articles/<slug>/sections.
- kb_get gains optional `section=<anchor>` param: returns just that
  section's subtree (heading + content + descendants) via body[start:end].
  Lets MCP clients fetch a single chapter from a 100k+ article without
  paying the full-body context cost. When `section` is omitted, kb_get's
  return shape is unchanged (backwards compatible).

Anchor format: h{level}-{slug-short}-{hash6}, e.g. h4-第三章判教-a3f95c.
- slug-short: title with invisibles / brackets / punctuation / dashes /
  whitespace stripped, truncated to 20 code points (CJK, kana, ASCII
  alphanumerics preserved). Empty after normalize → "_".
- hash6: first 6 hex digits of sha1 over the full normalized ancestor
  chain, joined by U+203A "›". Original spec was 4 hex; bumped to 6
  after Codex pointed out the birthday bound makes 4-hex collisions
  ~50% likely on a 300-section book vs ~0.3% at 6 hex. Collisions still
  append -2, -3 as fallback.
- Stable across (a) trivial whitespace / punctuation / zero-width /
  BiDi-control edits in any title and (b) sibling reordering. Breaks
  on title 字 changes anywhere in the chain — v0.7.2 will paper over
  with a content-hash-driven aliases map.

Codex pre-commit caught five real issues (HIGH x2, MEDIUM x2, LOW x1)
in the first pass — all fixed before this commit:
- HIGH: path-traversal `startswith(str(concepts_dir))` was bypassable
  when concepts_dir shared a string prefix with a sibling directory
  (concepts vs concepts_evil). Switched both /api/articles/<slug> and
  the new /sections route to Path.is_relative_to (component-aware).
- HIGH: kb_get / kb_get_sections ops at the operations layer had no
  traversal guard at all — only the HTTP layer enforced it, leaving
  direct CLI/MCP callers exposed. Both ops now use _safe_concept_path
  with the same is_relative_to check.
- MEDIUM: fenced-code closer was just startswith(fence_open * 3),
  ignoring opener length. A ``` line inside a ````` fence would
  prematurely close it and unhide a fake heading. Now requires same
  fence char, length ≥ opener, no info string.
- MEDIUM: ATX heading regex stripped trailing #'s unconditionally, so
  "## C#" became "C". Fixed per CommonMark §4.2 (closing #+ requires
  preceding whitespace). Also restricted leading indent to 0–3 spaces
  so 4-space indented "## …" lines stay as code, not headings.
- LOW: 4-hex hash collision realism (above). Bumped to 6 hex.

Tests: +36 in tests/test_sections.py covering normalize/anchor/parse/
ops/HTTP edges (164 total, all green). tsc clean, vite build clean,
python imports OK. Codex LGTM on third pass after fixing 3 doc nits
left over from the substantive fixes (hash4 → hash6 in docstrings +
CHANGELOG, _hash4 → _hash_hex helper, example anchor in op schema).

Note: this leaves v0.7.0 (X-LLM-Key per-request override, the
security-gated ask) as a future hole in the version line. Per user
request to ship sections directly as 0.7.1 — X-LLM-Key will land in a
later patch (likely 0.7.3 after sections-aliases ships as 0.7.2).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Assets 2
Loading