v1.3.0

What's Changed

• Created a How This Works page, inspired by Matt Hendrick's auto-georef (#397)
• Georeferencing
  • Introduced the "snap" step, a geometry-first fit that detects roads in a Sanborn image and
    aligns them with the modern OSM street grid. This is an extremely effective way to take
    decent fits and make them excellent fits. This was a huge win on the score metric, +15 points.
  • Introduced the "reconciler," a final step which considers all candidate fits for all pages as
    a whole and chooses the combination that minimizes an energy function. This was able to absorb
    many previous ad-hoc "gates" that were scattered through the pipeline.
  • "Upside-down" poses are rejected. This was a surprisingly small but effective check. (#324)
  • "street-solve" generates fits directly from the street detections, without requiring GCPs.
    This was my original idea for georeferencing, but it has somewhat limited use. (#175)
• Adjacency: lots of iteration here, including a UI for generating manual truth data.
• Key maps: Lots of iteration here, too.
  • mapsnap now detects lettered pages ("33C")
  • "Cartouches" like "index" and "graphic map of volumes" are detected and ignored (#384)
  • I manually generated truth data for regions. These are still hit and miss.
• Debugger app
  • More views, notably a snap debugger in the volume viewer and a boxes.json view.
  • Volume viewer shows missing pages and shows splits correctly
  • Volume viewer uses compare output as the source of truth
  • Visualization of adjacency data
  • You can take notes on pages directly in the viewer app
• Lots of retrained models
  • Fine-tuned EasyOCR model #279. This replaced old hacks like the underline detector.
  • keymap CRNN: #169
  • Road UNet #348
  • Page content region #355 (not used yet)
• Use the printed scale note ("100 ft per inch") #196/#197
• Tooling and prep for the big LoC run.
  • Move craft before split (#362)
  • loc-mirror (#404) and lots more; I now have a mirror of all of the LoC's imagery at 25% scale on S3 (many thanks to Dan Peebles!)
  • Cache P(road) map as a JPG and move this before split (#412)
  • loc-craft, the GPU pass for craft + roadprob (#413 and many others)
  • Generate per-county OSM road extracts (#407, #408, #419, #430, #433)

Full Changelog

• Improvements to local IIIF viewer by @danvk in #137
• Split key maps: mirror splits onto raw scans and pick the right panel by @danvk in #139
• family_scale: a junk-immune volume reference scale + per-page rung selection by @danvk in #140
• Support letter-only page ids (pa/pb) as key-map candidates by @danvk in #141
• Take notes on images in debug server by @danvk in #142
• Boxes view by @danvk in #143
• Accept any letter in page keys (for Los Angeles) by @danvk in #145
• Show missing pages + misc debugger improvements by @danvk in #146
• Read mapsnap compare output in volume viewer by @danvk in #147
• Add mapsnap score: the land-weighted success metric by @danvk in #148
• Show truth rectangles in volume viewer by @danvk in #149
• Visualize volume-wide adjacency data by @danvk in #150
• Lower aspect-ratio filter and add keymap location outlier check by @danvk in #151
• Geometry-first georeferencing (osm-snap): rescue, arbitration, refinement — score 55.5 → 69.8 by @danvk in #152
• Update express-iiif to 1.8.0 and drop the identity-size workaround by @danvk in #158
• Volume viewer tweaks by @danvk in #157
• Debugger: toggle to show a page's road-probability map by @danvk in #159
• Score truth split panels over their own regions (bad-split sensitivity) by @danvk in #160
• Re-sweep the refinement margin on the region-graded scorer (#153) by @danvk in #161
• Upgrade to ruff 0.16 and fix the newly-enabled rules by @danvk in #162
• Fix two blockers that break snap on any new volume by @danvk in #163
• Reorganize key-map truth data around the volumes it describes by @danvk in #164
• Adjacency-truth labeler in the debugger app by @danvk in #170
• Retrain key-map page-number models on the raw/truth corpus by @danvk in #169
• Full-string page keys: lettered vocabulary from disk through the KeymapLocator by @danvk in #171
• Repair conflicted key-map reads with CTC-margin evidence by @danvk in #172
• Stop filtering page edges in callers the pipeline does not by @danvk in #176
• Volume viewer: take the missing pages from the comparison by @danvk in #177
• Adjacency labeler: label split panels instead of the sheets they split by @danvk in #178
• Speed up the snap channel (issue #155) by @danvk in #174
• Streets-as-constraints georeferencer (issue #168): +1.5 aggregate score by @danvk in #175
• Don't let a failed key-map georef take the volume down by @danvk in #180
• Show unfit (non-truth) pages, filter keymaps by @danvk in #179
• Ignore a key map that re-indexes another sheet's pages by @danvk in #181
• Replace the page-region runaway guard with a flat 2% of the sheet by @danvk in #182
• Pick the key-map segmentation scale per sheet, preferring the finest that works by @danvk in #186
• Keymap region labeler by @danvk in #187
• Run the street-constraint channel in mapsnap fit by @danvk in #189
• fix-truth-splits: repair OIM's frame-mixed split selectors (OIM#402) by @danvk in #190
• Refresh the performance table and gallery from the 2026-07-30 run by @danvk in #193
• snap: rung-flip arbitration — adopt doubled-scale candidates over half-scale fits by @danvk in #195
• ocr: admit the printed scale note into the rotation-0 vocabulary (#12) by @danvk in #196
• Consume the printed scale note: rung authority + misscale demotion by @danvk in #197
• snap: collapse the outer search grid — local challenges + clustered centers by @danvk in #198
• compare: land-area columns and a land-weighted score footer by @danvk in #199
• compare: score truth splits from GCP-verified selectors when panels.json is absent by @danvk in #200
• printed-scale: cold-start calibration + a note-derived rung in snap's ladder by @danvk in #201
• Generate adjacency data in the LOC and OIM pipelines by @danvk in #202
• A printed note may not publish an unsupported off-rung pose by @danvk in #204
• adjacency: letter-aware page keys + read repair for long page numbers by @danvk in #206
• adjacency: one-sided claims as a lower-trust pose-graph tier by @danvk in #207
• adjacency-gate: demote fits that contradict their mutual claims, re-search from the seam by @danvk in #208
• adjacency: a volume-calibrated height floor under every claim by @danvk in #209
• mapsnap craft: CRAFT as its own step, and reorder adjacency before the key-map build by @danvk in #216
• Repair key-map page-number assignments from the printed adjacency graph by @danvk in #217
• Run the key-map repair to convergence, and never draw on an occupied panel by @danvk in #219
• clip_masks: don't let an invalid sliver abort the whole IIIF build by @danvk in #224
• keymap-snap: place pages by matching P(road) against the key map (#211) by @danvk in #225
• debugger: auto-restart the server, and one command to run both halves by @danvk in #232
• streets view: reflect the gate georef actually applies by @danvk in #231
• volume viewer: link a run's own sidecars, not the volume's latest by @danvk in #233
• Skip low-resolution key maps for vocabulary restriction by @danvk in #235
• volume viewer: open page views inline, and fix their image paths by @danvk in #234
• mapsnap archive: record a hand-assembled run by @danvk in #236
• Fix the circular import that broke mapsnap fit by @danvk in #237
• Make subvolumes first-class in the volume viewer (#228) by @danvk in #238
• fit: clear derived georef sidecars before regenerating them (#240) by @danvk in #247
• keymap: make the adjacency assignment repair opt-in (#239) by @danvk in #249
• Precise underline erasure with two-read arbitration (#250, #251) by @danvk in #260
• Scale the stamp-contradiction bar with sheet resolution; retry key-map outliers in the neighborhood (#254, #259) by @danvk in #262
• Three-vote underline arbitration: raw / precise / legacy row-paint by @danvk in #263
• README: 2026-08-07 corpus results (17 volumes, aggregate 74.9%) by @danvk in #266
• viewer: resolve truth page keys from the label, not the service URL alone by @danvk in #268
• Splitter Phase 0/1: OIM-truth harness, and a shape guard in place of the 0.05 floor (#83) by @danvk in #272
• oim-panels: build split truth from OIM's region boundaries (#273) by @danvk in #274
• snap: refine ineligible on deferred/1-GCP incumbents (#277) by @danvk in #280
• Fine-tuned street recognizer behind mapsnap ocr --recognizer-weights (#265) by @danvk in #279
• P(road) adjudicates the yellow band the fill filter spares (#127/#278) by @danvk in #281
• rerun: --recognizer-weights passthrough, forcing full re-OCR (#265) by @danvk in #282
• rerun: --no-fit for two-lane corpus runs by @danvk in #283
• docs: how to run a corpus baseline by @danvk in #284
• mapsnap reconcile: report-only joint pose arbitration, gates passed (#270 v1) by @danvk in #286
• reconcile: delete the sibling factor, measure the keep prior; snap: refine evidence floor by @danvk in #293
• fit --reconcile: publish through the arbiter (#270 phase 2) by @danvk in #297
• Collapse the georef sidecars: verdicts inside the file, one publisher (#270 phase 3) by @danvk in #301
• Sort canonical street matches: fix the pipeline's nondeterminism (#299) by @danvk in #302
• Score every sheet equally, split by paper area by @danvk in #300
• Delete the key-map-outlier gate (#270 phase 3) by @danvk in #307
• Draw the OSM relation a volume's streets came from; download-osm takes a directory by @danvk in #308
• Make a crashed or skipped fit run diagnosable instead of silent by @danvk in #309
• Run oim-panels in the OIM pipeline; retire the template-match truth path (#290) by @danvk in #310
• Buffer the OSM download by 1 km past the relation boundary by @danvk in #311
• Eight small debugger fixes by @danvk in #312
• Default to the fine-tuned recognizer; hash reads for effective pages only by @danvk in #313
• Retire the location-outlier gate and the 1-GCP confirmation veto (#270 phase 4) by @danvk in #314
• Skeleton pages are not key-map expectations (#316 phase 0) by @danvk in #317
• Record per-stage timing in mapsnap fit by @danvk in #321
• Keymap CRNN reads letter suffixes; synthetic training strips; multi-offset inference (#316 phases 1+2) by @danvk in #318
• Run adjacency on splits: per-panel claims, family-relative gate licenses, parent supersession (#135, #256) by @danvk in #323
• Remove the underline erasure path (#264): the fine-tuned recognizer reads through the ink by @danvk in #322
• Snap searches from demoted poses without un-demoting them (#315) by @danvk in #328
• Instrument the RANSAC special cases (#229) by @danvk in #327
• Veto upside-down poses (#324): 0/1,332 truth pages read inverted by @danvk in #329
• README + gallery: 2026-08-14 corpus results by @danvk in #331
• Fit prints the #300 score; guard mis-invocations (#330) by @danvk in #332
• Snap explorer in the volume viewer (#325 phase 1) by @danvk in #326
• compare prints the #300 score with 25-50/50-200ft bands; truth self-compare scores 100% by @danvk in #338
• Street-solve: batch consensus scoring (9x on heavy pages, no semantic changes) by @danvk in #339
• keymap-snap: post-#270 bitrot fixes + neighbor-offset correction (#211) by @danvk in #337
• Debugger: compare colors for uppercase lettered sheets (asheville p3C) by @danvk in #346
• Fit starts with cold snap caches (#342): warm saves 3 min/corpus, costs ~2 points of variance by @danvk in #349
• Embed sidecar keymap data below the megapixel floor (asheville/columbia regression) by @danvk in #347
• Train road UNet with uniform patch sampling by @danvk in #348
• Road UNet v4 swap with recalibrated verification bars by @danvk in #350
• Failed fits seed snap's rescue from candidate-GCP hints and neighbor stamps by @danvk in #336
• RANSAC runner-up poses: emit near-ties, promote past misscale demotions, seed snap challenges by @danvk in #341
• Page content-region model: trainer, visualizer, and trained weights by @danvk in #355
• Promote one-sided adjacency claims closed by a mutual-edge triangle by @danvk in #356
• Rescue sub-floor single-digit claims that a multi-digit claim reciprocates (#353) by @danvk in #357
• Fix CI pyright break; adopt the no-direct-push workflow rules by @danvk in #363
• Split annotations fall back to their panel rectangle, not the page by @danvk in #364
• 2026-08-28 baseline: 20 volumes, matched-18 mean 82.1, corpus mean 79.8 by @danvk in #360
• craft derives split-panel boxes from the parent page by @danvk in #362
• Fix snap panel crash on unscored candidates by @danvk in #367
• Volume viewer greys the selected panel's own off-panel area, not siblings by @danvk in #366
• Snap debugger phase 2: truth_pose and decision records by @danvk in #370
• Snap searches a fitted page's own scale rung by @danvk in #371
• Log admitted street reads that match no street name by @danvk in #374
• Charge snap candidates for street names that match nothing by @danvk in #376
• Compare tables name both the truth page and the generated page by @danvk in #380
• IIIF labels carry our own split marker, never the reference item's by @danvk in #381
• Two-panel sheets number the bottom-left panel first, matching OIM by @danvk in #382
• Key-map splits must cut away edge-flush boxes, or the parent sheet is used by @danvk in #383
• Key-map pipeline reads the sheet's cartouche words into a sidecar by @danvk in #384
• Key-map sheets keep a decision log, raw/.keymap.txt by @danvk in #385
• Find and mask the volume-index insets on key-map sheets by @danvk in #386
• Consumers of key-map reads skip the ones inside a volume-index inset by @danvk in #387
• Debugger shows key-map reads masked as volume-index insets by @danvk in #388
• Key-map corner-box detector cuts boxed regions the divider gate hides by @danvk in #389
• Snap panel shows how select and verif are built, on hover by @danvk in #391
• craft detects parents before deriving their panels; baseline pre-flight notes by @danvk in #394
• Key-map underlay: opacity slider on 'k', warped by thin-plate spline by @danvk in #392
• 2026-09-03 baseline: 20 volumes, mean 79.8 → 80.4; key-map consumers skip split parents by @danvk in #396
• How mapsnap works: a stage-by-stage explainer with figures by @danvk in #397
• Per-page content-region maps and a Page / Region / P(road) toggle in the volume viewer by @danvk in #401
• mapsnap loc-mirror: build the Sanborn mirror from LoC full-resolution JP2s by @danvk in #404
• loc-mirror: use the mirror's pre-rendered 25% JPEGs, falling back to the JP2 by @danvk in #406
• Cache P(road) as a sidecar beside each page, predicted before split by @danvk in #412
• Add loc-craft: the sharded corpus CRAFT + P(road) pass and its fleet scripts by @danvk in #413
• loc-craft: make a run measurable, and let --limit mean work done by @danvk in #414
• loc-craft: fixes from the first pilot (S3 retries, shuffled shard order) by @danvk in #415
• loc-craft: download the next item while the current one computes by @danvk in #416
• loc-craft: launch spot in the cheapest zone that has capacity by @danvk in #417
• loc-craft: list with s3api, which can tell empty from failed by @danvk in #418
• Add loc-keymaps: identify every volume's key map across the mirror by @danvk in #419
• Add loc-raw: fetch the full-resolution key-map sheets the mirror skipped by @danvk in #420
• bootstrap.sh: stop the periodic log uploader before the final upload by @danvk in #421
• mapsnap loc-counties: map LoC Sanborn items to Natural Earth counties by @danvk in #407
• loc-raw: let --list name an s3:// URL by @danvk in #422
• loc-raw: handle TIFF and IIIF sources, and survive a bad sheet by @danvk in #424
• launch.sh: rotate shards across zones instead of piling into the cheapest by @danvk in #425
• Read centerlines from an OSM extract directly; --no-debug-files for osm-to-geojson by @danvk in #408
• Add supervise.sh: relaunch shards that are neither finished nor running by @danvk in #427
• Add osm-counties: cut one OSM extract per Sanborn county from a national dump by @danvk in #426
• Report the zone's error when every zone refuses a launch by @danvk in #428
• Install the bootstrap exit trap before anything that can fail by @danvk in #429
• Map the independent-city Sanborn items to OSM boundary relations by @danvk in #430
• Emit the polygons and work list that osm-counties needs for the cities by @danvk in #431
• Let EC2 place on-demand instances before naming a zone by @danvk in #432
• Cut counties against surveyed OSM boundaries, with no buffer by @danvk in #433
• Add an SQS work queue as an alternative to static sharding by @danvk in #434
• Pin every SQS call to the queue's own region by @danvk in #436
• Reject --extra-args that ends in a valueless flag by @danvk in #437
• HOW-IT-WORKS: current step order, inputs/outputs, and GPU guidance per step by @danvk in #435
• Add loc-fit: the CPU chain per volume, with a provenance record for every page by @danvk in #438

Full Changelog: v1.2.0...v1.3.0