Skip to content

v0.1.0

Latest

Choose a tag to compare

@github-actions github-actions released this 07 Aug 07:13
· 1 commit to main since this release

First release.

Conversion

  • Encoding repair — candidate-scoring detector covering UTF-8/16, GB18030,
    Big5, Shift_JIS, EUC-KR and windows-1252. Scores the decoded result rather
    than guessing from raw bytes, so mojibake is caught even on short files.
  • Hard-wrap re-joining — detects text folded at a fixed column by measuring
    how tightly line widths cluster under a ceiling, then rebuilds the paragraphs
    without inserting spaces into CJK or splitting numbers.
  • Advertising removal — a pattern library (site watermarks, download URLs,
    求推荐票, QQ groups, 本章完) plus a frequency heuristic that catches anything
    short repeating once per chapter. Every deletion is reported; none is silent.
    Rules are deliberately conservative — see Design notes below.
  • Chapter detection — a library of heading patterns scored on the result set
    they produce (ordinal continuity, chapter evenness, coverage, standalone
    lines), so 章/回/话/节, Chapter N, bracketed and bare-number headings all
    work, and 第三章讲的是… inside a paragraph does not become a heading.
  • Two-level structure — 卷/部/篇 detection with 楔子/番外/后记 kept at book
    level rather than buried inside a volume.
  • Metadata inference — from 《书名》作者.txt-style filenames, scraper
    header blocks, config files and CLI flags, with the source of each field shown
    in the report. Language is classified as zh-Hans/zh-Hant/en.
  • Generated covers — deterministic SVG covers rasterised to PNG via resvg,
    falling back to embedded SVG when no CJK font is available.
  • EPUB 3 output — per-chapter XHTML, nav.xhtml plus toc.ncx, Dublin Core
    metadata with file-as sort keys, EPUB 3 collections and calibre series tags,
    landmarks, and a stylesheet that sets structure without pinning font family,
    size or colour.
  • Built-in validation — container invariants, manifest/spine consistency,
    XML well-formedness and unescaped-& detection on every document, with
    optional hand-off to epubcheck when it is installed.
  • inspect subcommand — full diagnostics without writing a file: encoding
    candidates, pattern scores, deleted lines and a structure preview.
  • Batch conversion — directories, recursion, and parallel processing.
  • Profileswebnovel, published, lightnovel, raw.

Online metadata lookup

  • --scrape searches by title and author — either inferred or supplied with
    --title/--author — scores the candidates, and fills in the author, blurb,
    genre tags, completion status, update date and cover. --scrape-url takes a
    book page directly and skips the search.
  • Scraped values are written verbatim and only ever fill gaps; anything set
    in a config file or on the command line wins.
  • Pages are read through the og:novel:* meta convention, which most Chinese
    fiction sites implement, so --scrape-url is not tied to one provider.
  • Off at runtime unless asked for, and a compile-time feature that can be
    removed entirely with --no-default-features.

Distribution

  • Static binaries for Linux (gnu + musl, x86_64/aarch64), macOS (Intel + Apple
    Silicon) and Windows (x86_64/aarch64), plus a multi-arch Docker image.

Design notes

Two capabilities were built, tested against a real 13 MB / 6.4M-character novel,
and then removed because they damaged data in ways their test fixtures did
not reveal. Both are recorded here so they are not reinvented:

  • Release-credit ad rule. A pattern matching lines that ended near 制作 /
    整理 / 校对 deleted roughly 10,000 characters of story from that one book —
    those are ordinary verbs, and 制作 appears on nearly every page of 修仙
    fiction. No rule for bare credit lines remains; only the anchored 本书由…整理
    form. Author asides (求月票 and friends) additionally apply to short lines
    only, so a paragraph that merely starts with one survives intact.
  • Official chapter titles. Adopting titles from a site's catalogue worked on
    the fixture and matched 0 of 1198 chapters on the real book, because real
    catalogues renumber across volumes, duplicate ordinals and skip them. Blurb
    rewriting went the same way: trimming marketing copy meant hand-tuning a
    keyword list against two examples.

The governing rule: a surviving blemish is better than deleted prose. Where
a heuristic cannot be made reliable on unseen input, it is not shipped.

Scraping requests metadata only — not chapter text, and not the chapter list.
It is rate limited to one request per 1.5 s by default, runs sequentially, and
falls back to local inference on any failure, including a site returning an
access challenge, which is never worked around.