Repository navigation
backdraft 0.11.0
A week on sizing up a source before spending on it — what it will be called, how much of it a read costs, and whether there is anything in it.
backdraft ingest <source> --dry-runsays what each source would be called before that is permanent: the slug and media type it would take, whether it is already ingested (under which slug, and whether withdrawn), and the numbered slug a taken name would land on. Nothing is fetched, written or minted. A slug is fixed once a token carries it, so this is how to see the default before--slughas to overrule it.- A page read has a budget. One
backdraft readshows 12,000 characters — 200 rows of a sheet — unless--limitsays otherwise, so a scraped article no longer spends a context window in a single call with nothing to say so. A read that stops short closes with the total and the exact command that continues it. The cut never lands inside a chunk, so every token still names text you were shown whole, and ordinary pages print exactly what they did. - A web page's table of contents is its chunks:
backdraft read <slug>on a one-page source listsp1.c1,p1.c2, … with what each opens with, under the name the page gave itself in its<title>— the map for finding where the article starts past the site's navigation. The artifact names a fetched page by that title too, wherever the record carries one. - A source that came back a shell says so wherever it is listed.
backdraft read's document list, that source's table of contents andbackdraft lsall mark itlittle text: N chars, so a registry ingested in another session still tells the writer which sources are login walls and scans. - Every hint the gate prints —
search's,show's, the table of contents',session show's and a short read's continuation — now carries a--sessionyou typed, so following one no longer reads into a different ledger from the one you bind against. ingest --dry-runrefuses a malformed--configthe wayingestdoes, rather than predicting a run that cannot start.
1440 tests.