Chapters

Chapters

Splitting a comic into one archive per chapter (what readers want and phones can open), correcting boundaries, packing, and staying current. Needs a settled Page Index and Alignment; all of it is also on The Web Page.

python chapters.py chapters "D:/Comics/Uncompressed/MyComic" --archive "https://example.com/archive/"
python chapters.py chapters "D:/Comics/Uncompressed/MyComic"               # from the addresses, or what it remembers
python chapters.py chapters "D:/Comics/Uncompressed/MyComic" --list starts.txt
python chapters.py chapters "D:/Comics/Uncompressed/MyComic" --every 100   # a part every 100 pages
python chapters.py try "D:/Comics/Uncompressed/MyComic" --archive "https://example.com/archive/"
python chapters.py try . --archive "https://example.com/archive/" --like "https://example.com/comic/1"

Only corrections by hand are written without --save. With no source, chapters re-reads whatever source the comic remembers. Packing is below.

Previewing: try shows how many links are this comic's pages and the chapters it would read, using the real code with the archive's order, without walking or saving. --like supplies a page address for an unscraped comic. One huge chapter means no headings: use another source. Stated lengths ("3. A Day Out (4 pages, 5/8/06)") are stripped from names and totalled, and a heading lacking one where the rest have it is flagged as probably another part of the site.

Sources of chapter starts

An archive page

--archive URL. Links are checked against the walked addresses, and titles come from headings above them.

• A link inside its heading reads the same: <h4>1. <a href="c1/p1">First Steps</a> <span>(6 pages)</span></h4>.

• One table per chapter, named in its <th>, reads like <h2> headings, even with no number or "chapter".

• A real heading tag beats a container classed "chapter", which usually holds a description and icon too.

• A container holding a list of page links is never a heading (<div id="chapters">, <table id="chapter12_table">); otherwise a 3,500-page comic reads as 3,500 chapters.

• Drawn headings (<img src="chapter12.png">) are named by alt or filename, but only when named for a chapter, arc, volume, book, part, episode or season followed by a number (partners.png heads nothing).

• Banners are recorded with each chapter as image, when the list shows one: the drawn heading itself, a picture inside the heading beside its name, or a storyline's thumbnail just above its header. A gallery of page thumbnails, or the site's logo in its title, is nobody's banner. try says how many it found. Packing saves them as covers (below).

• Shortcuts are not chapters. Once the archive has listed its chapters in order, a later link that points back into them, such as a "favourite moments" or gallery section below the list, starts nothing. Its page stays in the chapter it was published in.

• Only pages are read. A heading whose one link leads to a page of its own (a chapter's page listing) has that page read for where the chapter starts. Some things are never fetched for this:

  • links to files (pictures, zips, pdfs, videos), and anything that isn't HTML;
  • anything over 8 MB.

  A page several headings share is read only once. Each one read is named in the output as it goes.

• --browser loads a JavaScript-built archive page; --shift moves every boundary, for an archive that labels a chapter after its first page.

The addresses themselves

No source, or --urls to force it over a remembered archive page. For /c4/p7, /mc/4-7 or /comic/issue-4-page-7, every number is tried as the chapter and the best reading wins. Chapters count up from where the comic starts (so dates are not chapters), numbers above the page count are not chapters (so /mc/20211202 is a filler, not chapter twenty million), a one-off address stays in its chapter, and each chapter is named by what most of its pages say.

A file, or by hand

--list FILE: one start address per line, optionally | title. Or add boundaries one at a time with fix (below); the first on an unchaptered comic creates it, marked as chaptered by hand so re-working keeps them.

Every so many pages

--every N cuts a chapterless comic too big to open (a page a day for ten years can be a gigabyte): --every 100 gives pages 1-100, 101-200, and so on.

• Parts are named for what they will hold (Pages 3601-3700), so the growing one keeps its name.
• The size is remembered and the next part starts by itself; changing it needs --force, and old-size archives are left for you to delete. A comic under one part is saved as one part.
• Numbered files need no walk (--prefix names, or 0001.png, even 0001.png.png): thousands of pages split in seconds. Files from 0000.png (another tool's start) are read one on. A missing number is a missing page; an unnumbered file, or two on one number, means it asks for a walk. A comic scraped from page one (primed included) needs no walk either way.
• On the web page, when filenames cannot say which page is which, it stops and asks (explained in the editor and under Recent): keep one of a page on two files (0742.png and 0742.png.png, shown with sizes and whether identical; the other goes to .set aside, not deleted, and in no archive); Discard that walk for a one-page walk begun on the newest page, which would otherwise win over the numbers (renamed aside in the config folder); or Walk it instead.
• A primed comic's one-page archive is given up when the update fills the parts and every page is placed.

Rules for every source

• A result of one chapter over the whole comic is refused (no headings found).
• A chapter runs until the next, so filler, guest art and flash pages stay where published.
• Chapters are numbered in reading order whatever their labels, so year-labelled specials are not sorted last.
• Corrections by hand apply on top of any source.

Putting a boundary right by hand

For a page the site names wrongly, like one cover called 20-an-odd-name among issue-20-covers:

python chapters.py fix "D:/Comics/Uncompressed/MyComic" \
  --at "https://example.com/comic/20-an-odd-name/" --label "Issue 20"

Flag	Effect
--at PAGE + --label NAME	Chapter starts here. An existing name is moved, not duplicated.
--drop	No chapter starts here; its pages join the one before.
--forget / --clear	Take back one / all corrections (then chapters --save restores the original).
--no-repack	Leave archives alone and report them out of date.
(none)	List corrections; corrected chapters show by hand.

Changed chapter archives are rewritten at once, and only those (one boundary in thirty issues rewrites two). Corrections live under chapters.fixes in the metadata, anchored to page addresses, so they survive re-working and the comic growing.

Packing

chapters.py pack <folder> --root "D:/Comics" writes CBZs/SomeAuthor/TheirComic/TheirComic - c001 - Chapter 1.cbz and so on. Each has a ComicInfo.xml with series, chapter and a first-page bookmark, as Kavita, Komga and PerfectViewer read; the chapter count appears only once the comic is marked ended. Only changed chapters are rewritten, a stale ComicInfo alone is appended without rewriting pages, and new pages join the chapter still being published. --replace gives up the single archive only once every page is in exactly one chapter archive at its on-disk size, and turns the single archive off in settings.

• The number in the name, c001, and the ComicInfo Number are the chapter's own number, read from its label:
  • It is the number after Chapter, Ch., Episode, Issue, Book, Arc, Story or #, or the number a label opens with (12. Off to the Movies). So a comic kept from Chapter 391 has c391, and Ch. 10.5 is c010.5.
  • A chapter whose label has no number takes a half step after the one before it: an intermission after 12 is c012.5 (a second in a row is 12.6), and a prologue ahead of Chapter 1 is c000.
  • The comic keeps plain places 1, 2, 3 instead when its labels do not number it in order. That covers titles only, numbers that go back, "Part 3" counted afresh for each story, and a comic cut every so many pages.
  • Packing renames an archive still named by its place to its number, with the cover beside it, rather than writing it again. It records the rename so the Reader keeps where you were in that chapter.
  • Use chapters.py comicinfo <library> --dry-run to list what a library would rename.
• The shelf is the one recorded in chapters.folder once a comic has been packed, so archives you move by hand (fixing that folder in the metadata) stay where you put them. Packed the first time, a comic goes in the shelf folder of its name, My_Comic counting as My Comic, rather than a folder nested inside it.
• Each chapter's banner is saved beside its archive under the same name, TheirComic - c001 - Chapter 1.png, where readers (Komga, Kavita, the Reader) look for a book's cover. It is fetched with the chapter list as the referring page, checked to be a picture, and never written over one already there, so a cover you chose stays. chapters.py covers <folder> --root ... saves them for a comic already packed, packing nothing; read the chapter list again with --save first, so the banners are recorded.

Keeping chapters current

• A scrape adds each saved page to the index, so a chaptered comic never needs re-walking.
• After gaining pages, the updater aligns them and rewrites the chapter archives (in practice, the current one), however the scrape ended short of being stopped by hand. A comic in chapters that has never been packed - set to be cut once it had caught up, say - is packed by the next update even when it finds nothing new. Off: pack_chapters: false or --no-pack-chapters; chapters.py pack catches up later.
• A comic added with a chapter list or cut every so many pages is packed straight after its first scrape, as part of the same job, even if that scrape failed on its last page. The Reader shows the new archives after its next scan, within 15 minutes.
• It then re-reads the remembered chapter list (off: refresh_chapters: false or --no-refresh-chapters). New chapters at the end are taken; a change moving an archived chapter is reported, and taken with chapters.py chapters <folder> --save --force once you have looked.
• A comic cut every N pages is re-cut after every update regardless of refresh_chapters.
• A walk that stopped short (a site refusing a long walk) is not built on: a scrape adds nothing to it, and says so, and Work out chapters carries the walk on from where it stopped.