Skip to content

v0.8.3 — Failing books held alone with a fix request; `--force` keeps figure links; gap chapters

Latest

Choose a tag to compare

@drpwchen drpwchen released this 13 Nov 20:19

Added

  • A book that fails split validation is held alone, with a fix request. post_convert.py
    used to count a resplit as done when it exited 0 and wrote a manifest; the validator's
    verdict in _resplit_report.json was never read, so a wrongly split book was indexed like
    any other. Now a book recorded as valid: false moves to <corpus>/_held/<book>/
    (ask-your-library skips _-folders, so it is out of search while every other book is handed
    off as usual) with a _fix_request.md: what failed, which _split_config.json keys address
    each check, the exact re-split / validate commands, the done criterion and the rules — a brief
    an AI agent can carry out. It is listed under [FAIL] (exit 1), re-split automatically once its
    markdown or the split config changes, and returns as soon as it validates. Measured on the
    author's corpus: 1 book failed (a two-page-spread scan OCR'd with both pages interleaved line
    by line). Loop and check→key table: docs/corpus-maintenance.md. Specs:
    openspec/changes/archive/split-acceptance-gate/, …/held-book-fix-request/.
  • converter/chapter_check.py: chapter numbers vs each file's own pages. A split can pass
    every check and still number all chapters wrong (off by a constant). Every book resplit in a
    post_convert.py run gets counts of files whose opening confirms / contradicts their
    chapter_number, and a [WARN] when contradicted > confirmed. Report only — it never fails
    the run (in-text cross-references make a few files contradict in many correct books; on the
    author's corpus no book trips the warning). Also a CLI over a whole corpus.

Changed

  • --resplit --force keeps a book's figures* / images folders. post_convert.py links
    each corpus book's figure folder to the converted book (a junction on Windows); a forced
    resplit deleted the old book folder and the link with it, so every image in that book was
    unreachable until the links were recreated by hand (a real figures folder was simply
    lost). Folders resplit never writes (figures*, images, _epub_src, *.bak, …) now move
    into the rebuilt folder — links as links, targets never opened — and the run report lists
    them as kept_dirs. Everything resplit writes is unchanged. Checked on a real book with 476
    linked figures. Spec: openspec/changes/archive/resplit-force-keeps-figures/.
  • A bare-numbered chapter one level off is split as a chapter when it fills a numbering
    gap.
    0.8.0 moved Chapter N entries to the chapter level; 3 Paralysis and Weakness was
    left alone because section numbers look the same. Now, between chapter-level entries a and
    b, entries at one other level numbered exactly a+1 … b−1 (in order, on pages between
    theirs) are chapters too. Adams & Victor's Neurology (chapters 3–26 one Section deeper)
    splits with no config at all: every one of its 1,616 pages gets the same chapter number as
    the 24 hand-written overrides gave. Full corpus resplit before/after: several hundred books, 19,128
    files, 0 changed (books with chapter_number_overrides are still never promoted).
    Spec: openspec/changes/archive/resplit-gap-chapters/.

Docs

  • README (EN and zh-TW): support line after the quickstart pointer, new Cost and Limitations sections.
  • Replaced exact corpus sizes in README and CHANGELOG with "several hundred books".
  • README(英文與繁中):quickstart 後加支持連結、新增「費用」與「限制」段落;README 與 CHANGELOG 中的精確語料庫書數改為概略說法。
  • Added CITATION.cff.
  • docs/corpus-maintenance.md: "When a resplit book fails validation" (the fix loop, check → config keys); README: every book of the author's corpus now passes the validator.