Repository navigation
The output files are now replaced atomically. Before this release an interrupted run could leave a shorter file where a complete one had been — measured, a 2,084-byte good
out.jsoncame back 0 bytes and invalid JSON after a crash mid-write. If you have ever seen a truncated output or sidecar, that was this.CSV cells beginning
=,+,-,@, tab, CR or LF now carry a leading apostrophe, so a spreadsheet reads them as text rather than as a formula. The JSON is unchanged and still carries the site's own bytes;csv_cells_escapedin<out>.meta.jsonsays how many cells the two files differ in. On this site that number is 0 today, measured across 2,662 stored rows, so no existing CSV changes.
Fixed
- Atomic writes.
write_json,write_csvandwrite_run_metausedopen(path, "w"), which truncates before a byte is written. Each now writes to a temporary file in the same directory (os.replaceis only atomic within one filesystem),fsyncs it and renames over the target. The sidecar matters most: it is the file a consumer branches on, so a truncated one beside good rows reads as a broken run over data that is fine. - CSV formula neutralisation, strings only — escaping a number would turn
-5into text and break every sum written over the column. - Scraper API:
waitForis sent as a JSON object. The string form is refused with HTTP 422 and still billed, so every--wait-*run was a paid exit 5. - Scraper API: the target's HTTP status is read from
http_code, not from the API's own verdict string, so the page classifier now sees a target 403/503. - The end-of-listing check compared against the wrong page number, so
--url '...?p=2' --pages 1threw away 45 good rows as "the end of the listing". - The credential scan was blind to JSON-escaped fixtures — the key-shaped-field rule used bare quotes and matched zero times in the largest file in the repository.
- Donor-repo leftovers, none of which changes behaviour.
Changed
- CI tests Python 3.13. The badge has claimed
3.9 | 3.13while the offline matrix ran 3.9 and 3.12 — the ceiling a reader sees first was the one version nobody ran.
Full notes: CHANGELOG.md