Map Viewer Tool

Map Viewer Tool

rns510_map_viewer.py is a read-only Tkinter GUI that renders the real map disc directly, built specifically to validate and debug the file format findings documented elsewhere in this wiki — not a production navigation UI.

Run with py rns510_map_viewer.py.

What it does

• Pools all 5 rendering layers (eeuz.mp0, .mg1–.mg4) together, with per-layer visibility checkboxes.

• Renders every decoded point as a dot, with an optional "Draw connected roads" mode built on the topology/adjacency resolver.

• Click-to-identify: right-click any rendered point or connected-road edge to see exactly which layer/tile/feature/point it is — the tool used to find and report several real bugs in the topology resolver (ground-truth connectivity data, false-edge reports). A point pick also now shows "nearby streets" — the real street names from the nearest eeu.iof "anchor" record's own eeu.il-based list, if one exists within ~3.3km of the click. A real POI icon is checked first of all — a click on one shows its real name and category in the status bar.

• A simulated Address Entry screen (Country → City → Street, live per-letter narrowing) built on eeu.ctr/eeu.cty/ the road name-search catalog, modeled on the real unit's own on-screen keyboard.

• Discrete, real-hardware-style zoom levels (500km down to 25m) and bezel/hardware-button chrome matching the physical unit's layout.

• Real POIs from EDB/POI/POI.DB3 — 4.73M real points of interest, gas stations to airports, unlocked by cracking the database's own Coordinate field. Zoom-gated by each category's own real, disc-designed MapZoomLevel (airports visible from far away, "downtown area" markers only up close), toggleable with a "Show POIs" checkbox, and drawn with the disc's own real category icons (gas pump, airplane, pharmacy cross, ...) — not plain dots. The same unified search box that already finds roads/cities also searches real POI names (tagged [POI], showing the real category).

• Real Sirius TravelLink POIs (gas stations, from a separate NAR firmware disc's own SIRIUS.DB3), with real per-channel icons decoded from the .DDB channel-art cache — see Sirius TravelLink and North America Discs for the full format crack this feature is built on.

• Live firmware screen-shape reading: File > Open Firmware ISO (real screen shape)... reads a real firmware ISO's own FHDD6.FLI and reshapes the canvas to the unit's own real aspect ratio (letterboxed/pillarboxed) — nothing hardcoded. See "v40 → v41" below.

Architecture

MapData (all data/decode logic, no GUI — directly exercised by test_map_viewer.py) + App (the Tkinter GUI shell). Rendering is a single Pillow-rasterized bitmap per redraw (PIL.Image/ImageDraw, shown via one canvas.create_image()), not one Tkinter canvas item per point/edge — the earlier per-item approach didn't scale past a few thousand points.

Full changelog

This tool has gone through many iterations (v1 → v42, as of this page), each one documented with real before/after numbers, root-caused bugs, and validation evidence. README.md's own §10 "Map viewer (visualization/validation tool)" section is the original, even more detailed source this page condenses.

("v22 → v23"): POI search — completes the POI feature arc (crack → display → icons → click-to-identify → search). The same unified search box that already merges road + city results now also searches real POI names, via a new poi_db_reader.search_pois() (a plain case-insensitive linear scan over a precomputed uppercase name list, ~0.20s per search — fine for an explicit "press Enter" search that already runs in a background task). SearchHit gained a "poi" kind and a proper breadcrumb branch (previously any non-city hit fell through to a hardcoded "ROAD" label, which would have mislabeled a POI). Verified end to end with a real running-app test: searching "MCDONALD" returns real POI hits alongside road/city results; selecting one and jumping lands exactly on its real coordinates, with the breadcrumb correctly reading RESTAURANTS | POI | MCDONALD'S.

Also this session ("v21 → v22"): POI click-to-identify — the natural next gap once real icons landed ("v20 → v21"): clicking one did nothing. A real POI icon is now checked FIRST in the click-to-identify flow (wider pick radius than a road dot, matching a real 34×39 icon's on-screen size), searching the EXACT set the last redraw actually drew after decluttering — so a click can only ever hit something visible — and shows the real name/category in the status bar. Verified with real running-app tests: a click on a real rendered POI reports its name/category correctly (including "(unnamed)" for the many real POIs with no Name), and a click in genuinely empty space correctly falls through with no false positive.

Also this session ("v20 → v21"): real POI category icons, not plain dots. POI.DB3's own icon schema — PoiPartition.Icon_ID → Image_BaseAttributes → Image_ImageBlob_Relation → ImageBlob.ImageData — turned out to be real, standard 34×39 PNG files the whole way through (confirmed via PNG magic bytes), with every category name matching its own linked image's real name ("gas station" → an image named "gas station"). Visually confirmed real, recognizable icons (gas pump, airplane, pharmacy cross) by decoding and inspecting them directly. Pasted into the same rasterized bitmap as road dots (~0.0056ms/paste — a whole-EEU 500km-span redraw with POIs on still takes only 0.150s). A real clutter bug found and fixed on the actual ISO: a dense real area (central Sofia) first rendered as an unreadable wall of overlapping icons — confirmed by inspecting a real screenshot, not assumed — fixed with a grid-occupancy declutter (52px cells) that caps rendered density regardless of true POI count; re-rendered and visually confirmed the fix produces genuinely readable icons.

Also this session ("v19 → v20"): real POI display, unlocked by cracking POI.DB3's own Coordinate field the same session (a 64-bit Morton/ Z-order code). All 4,733,183 real POIs are extracted + decoded in the background (~8.6s total, never blocks opening the disc) and filtered per-viewport with the disc's own real MapZoomLevel visibility design — confirmed directly: a Sofia-area query returns a sensible real category mix (826 restaurants, 811 ATMs, 95 gas stations, ...) at a close zoom, correctly dropping to only long-range categories (airports, etc.) at a continent-wide zoom. Markers rasterize into the same bitmap as road dots (a wide-but-in-range viewport can cover 100k+ POIs, confirmed at 185,520 — individual canvas items at that count would risk the exact freeze "v17 → v18" already fixed for roads); only labels are real canvas items, capped like city labels already are. Verified with a real running-app test: rendered pixels match the POI marker color exactly, and drop to zero after unchecking "Show POIs".

Also this session ("v18 → v19"): 2 real, measured performance fixes. Road-name matching was being fully re-run over the entire pooled feature set on every single pan/zoom, even for tiles already matched by an earlier, overlapping load — now cached per (tile_id, tol_m) (MapData.name_caches), 293x faster on a real repeat-view test. App._to_canvas() was re-querying Tk for the canvas size on every single point/vertex it projects — now cached once per redraw. Both verified against the real ISO with before/after correctness checks (byte-identical results vs. ground truth), not just "didn't crash".

A real, severe bug found and fixed ("v24 → v25"): every app close leaked its entire temp workdir

MapData extracts a working copy of the ISO's own files (all 5 layers, eeu.cty, POI.DB3, a search index) into a real tempfile.mkdtemp() directory per open — often 500MB-3GB+. MapData.close() deletes it correctly, but was previously only ever called when replacing an already-open ISO with a different one — never on normal app shutdown (no WM_DELETE_WINDOW handler existed). Closing the app via its window X button silently leaked the entire workdir, every session. Confirmed directly against a real system: 56 leaked directories, 146GB total, deleted after confirming no viewer instance was running. Fixed with a real close handler (App.on_close(), wired via root.protocol ("WM_DELETE_WINDOW", ...)) that closes self.data before the window exits.

"Show tile boundaries" checkbox ("v25 → v26"), user-requested

Prompted directly by the real cross-tile adjacency gap found this session — a user-reported road connection crossed from one mg4 tile into a completely different one, a class of gap the adjacency resolver has no mechanism for at all. MapData.active_tile_bboxes() computes each currently-pooled tile's own bounding box from its already-decoded points (no extra disk I/O, no real per-tile grid bbox stored on-disc for these layers). Sanity-checked directly against the real cross-tile report: the 2 reported tiles' computed boxes meet almost exactly at the reported point's own coordinate. Draws each box as a thin outline rectangle, rasterized into the same single image as every other overlay (never individual canvas items — a wide viewport can pool 1,000+ tiles). Makes it easy to visually spot a dead-end point sitting right on a tile edge — a strong hint it's a candidate cross-tile connection worth reporting. OFF by default.

"Hide decode garbage" checkbox REMOVED ("v26 → v27"), user-requested

User request (verbatim): "remove the hide decode garbage mechanism, i keep it always off because it doesnt help" — consistent with the checkbox's own history: real-ISO testing had already found the underlying oscillation filter net NEGATIVE for real dense-urban rendering (a real Sofia sample lost 6 whole named roads and badly truncated 13 more while removing only 5.74% of raw points), which is why it defaulted to unchecked before this removal. The checkbox and its handler are gone entirely — nothing in the running app can re-enable the filter anymore. MapData.trim_oscillation/set_trim_ oscillation() remain as a programmatic-only API (still exercised by test_map_viewer.py against the real ISO); decode_features()'s own detector also still exists in research/map_compressed_reader.py for research use. Only the UI surface is gone.

Real bug fixed ("v27 → v28"): max_edge_m=200 was wrongly dropping real mg4/mg3 edges

User-reported ("missing points", then correctly "these 2 must be connected"): a real mg4 edge 1,518m long (a genuine divided-road carriageway transition) was being silently dropped by the adjacency resolver's own max_edge_m=200 default — calibrated only against dense mg2/ mp0 ground truth, never validated against the coarser mg4/mg3 layers where long real edges are normal. Auditing one feature found 28 real edges dropped this way. Fixed with a new MAX_EDGE_M_BY_LAYER dict (mg4/mg3: 5,000m; others unchanged at 200m), passed at both of MapData's own call sites. Verified with a live redraw test: rendered connected-roads line pixels increased at 2 zoom levels (667→1,183, 1,055→1,460) — more real edges now drawn.

A 2nd false-edge bug ("v28 → v29"): the loosened mg3 cap above let a sentinel-value bug finally surface

Widening mg3's edge cap to 5,000m (previous entry) let a different, pre-existing bug through: _shared_value_edges() (research/ map_compressed_reader.py) never excluded the literal topology value 0 as a "not a real link id" sentinel — 5 earlier false edges (all mg1/mg2, 200m cap) happened to be masked by the tight distance cap alone, never by this exclusion actually existing. User right-clicked a real mg3 edge (tile 9237, points 61↔227, 4,555.3m apart, sharing only value 0) that the loose cap no longer caught. Fixed at the source: 0 is now excluded outright as a candidate link id, not relied on the distance cap to catch its symptoms. Re-validated: both human ground-truth tiles unchanged (mg2 20597 16/16, mp0 91124 16/18), all 5 original false edges still excluded, this case now excluded too — by never being proposed as a candidate at all.

The viewer stops extracting files to disk — reads directly from the ISO instead ("v29 → v32", 3 sessions)

A 3-step arc, each step widening what's read directly out of the 6.48GB .ISO instead of being copied to a temp workdir first:

• "v29 → v30": the MAP_COMPRESSED tile layers (mp0/mg1-mg4). pycdlib.get_file_byte_extents() exposes a file's absolute byte range(s) inside the ISO; every file checked on this disc family (including mp0 at 2.14GB) is a single contiguous extent, so a tile decompresses by seeking straight into the ISO — byte-identical to reading an extracted copy (verified both at the raw-reader level and end-to-end: identical geo-index tile counts for every layer, the mg2 20597 ground-truth tile still resolves shift=1/"high"). load_heavy_layer()'s ~2.14GB mp0 copy step is gone entirely; only its (unchanged) geo-index build remains. The road editor (rns510_gui.py) is unaffected — editing needs a real mutable local copy, so its extract-then-edit flow is unchanged.
• "v30 → v31": eeu.cty, eeu.rd/.il/.iof, and eeu.ctr/eeuz.rt (the address-entry panel's own files) — each reader module gained its own small, dependency-free _open() (same duck-typing pattern, deliberately not centralized). rns510_core. MapProject gained a read_only=True flag that resolves files as ISO refs and refuses every edit/save call with a clear error. Verified: after a full load, workdir contains zero files — search, topology, and country-list loading all still work.
• "v31 → v32": POI.DB3 too — the last extracted file. sqlite3 can't seek a byte-window inside a bigger file, but Python 3.11+'s sqlite3.Connection.deserialize() loads a database straight from an in-memory buffer. Verified end-to-end: read 5.0s, deserialize() 1.65s, decoded the same 4,733,183 real POIs and 61 icons as always, workdir confirmed empty afterward. Real, accepted trade-off (confirmed with the user before implementing): POI.DB3's ~1.1GB stays resident in memory for the life of the connection instead of on disk. With this, the entire map viewer reads every file directly from the ISO; only the road editor still extracts anything.

tile_ids_in_bbox() gains 2 more real tests, then one of them is reverted ("v32 → v37", 5 sessions)

• "v32 → v33": tile_ids_in_bbox() (decides which tiles to pool for the current viewport, called on every pan/zoom) used to test only a tile's single geo-index ANCHOR point against the query rectangle — a tile whose anchor sat just outside the viewport but whose real content still extended into it could be silently skipped. Now unions that with a 2nd test: does the tile's own real content bbox OVERLAP the query rectangle (a strict superset of the anchor test wherever both exist, so this can only ADD tiles, never drop any) — confirmed: the same small query that returned 1 tile via anchor-only returned 4 via the union. Built as 2 new background tasks (FAST_LAYERS right after load(); mp0's as a tail step of the existing heavy-layer load, deliberately AFTER mp0_ready, since build_bbox_index() takes minutes vs. build_geo_index()'s tens of seconds) — a missing/incomplete bbox index for any layer just falls back to anchor-only, never an error.
• "v33 → v34": a 3RD test — a tile's native, no-decompression geo- hint (a single coordinate bound, axis-ambiguous by construction, read straight out of the pre-table region with no tile decompression at all) — for fast partial coverage while the real bbox index is still building. Same "only ever adds tiles" philosophy. Built synchronously for FAST_LAYERS inside load() itself (2.7s combined, cheap enough not to need a background task).
• "v34 → v35": mp0_ready reordered to wait on the (much faster) native hint scan instead of the original build_geo_index() — street -level detail usable in ~8s instead of 35-55s+. Timed stage by stage on the real 194,705-tile file: native hints finish at 7.8s (53.7% coverage, mp0_ready now flips here); build_geo_index() completes at 158.9s, build_bbox_index() at 613.2s, both now pure background enhancements.
• "v36 → v37": REVERTED, a real user-reported performance regression, honestly measured and walked back. User report: tile search hung a long time on a first search at a new location. Root cause, measured directly: the native-hint test (either-axis, by construction) inflated pooled tile counts 4.5-10x over anchor-only at a realistic viewport (mg1 41→203, mg2 23→109, mg3 13→54, mg4 6→24) — confirmed structural across box sizes from ~170m to ~21km, not a large-viewport edge case. Every extra tile costs a real decode_tile() call — this was the reported slowdown, directly. Reverted, not patched: back to the 2-test union (anchor + bbox); mp0_ready back to waiting on build_geo_index() (flipping it after only the hint scan would leave mp0 "ready" but functionally near-empty for the ~150s gap until the real geo-index finishes — worse than honestly still loading). scan_native_geo_hints() itself is unchanged and still built (cheap, ~4.35s) for possible future use — it was specifically its use as a blind either-axis viewport filter that didn't hold up at real scale.

A 3rd and 4th false edge found and fixed ("v35 → v36", "v37 → v38")

• "v35 → v36": a contaminated 3-point clique, not a sentinel value this time. Same tile as the prior 2 false-edge fixes (mg3 9237): user-reported points 60↔211, 4,215.8m apart, sharing non-zero link-id value 11 with a 3rd point, 213 — but only the (211, 213) pair is real (23.4m apart); both (60, 211) and (60, 213) are false, all under mg3's loose 5000m cap so the distance filter didn't catch it either. _shared_value_edges() used to treat any clique of size ≤max_group as fully connected (every pair an edge) — wrong when an unrelated point just happens to also carry that value. Fixed: for cliques of size >2, keep only pairs within max(200m, 5× the clique's own shortest pairwise distance), instead of accepting every pair uncritically. Re-validated: both ground-truth tiles unchanged, all 6 previously-confirmed false edges still excluded, this case excluded too, AND the real sibling edge (211, 213) confirmed to survive.
• "v37 → v38": a plain 2-point value collision (shared link-id value 100, no 3rd point, well under mg4's 5000m cap) — user- reported points 98 ("OKOLOVRASTEN PAT", Bulgarian for "Ring Road") ↔126, 772.8m apart. Gap alone isn't a safe signal (real validated edges reach a gap of 21), but cross-referencing gap AGAINST distance reveals a clean split that never overlaps in real data: large-gap real edges always have small distance, large-distance real edges always have tiny gap. This false edge is the one combination that never occurs for real (gap ratio 0.168, distance 772.8m, both large). Fixed: a new conjunctive filter — drop an edge only when BOTH its gap ratio exceeds 0.10 AND its distance exceeds 200m — so a short high-gap edge or a long low-gap edge each survive untouched. Re-validated: both ground-truth tiles unchanged, all 7 previously- confirmed false edges still excluded, this case excluded too, and the real 1,518m interleaved-parallel-chains edge confirmed to survive.

A 2nd, independent POI layer: real Sirius TravelLink data ("v38 → v40")

• "v38 → v39": a byproduct of the firmware-side investigation (see Sirius TravelLink and North America Discs) — a real, 158,088-POI Sirius TravelLink database from a factory SWL disc, sharing enough of POI.DB3's own schema that poi_db_reader.load_poi_cache()/load_poi_partitions() work against it completely UNCHANGED. New: File > Load Sirius POIs... (a plain file picker — SIRIUS.DB3 is a standalone SQLite file, no extraction involved), a Show Sirius POIs checkbox, its own declutter grid (independent of the regular-POI one). Appended into the same rendered_pois list regular POIs use, so click-to-identify picks them up with zero extra code. Verified: the real Northern- California gas-station POIs ("BLOXHAMS SHELL", "CHEVRON", "UNION 76") decode and render correctly.
• "v39 → v40", immediate follow-up: real per-category icons for Sirius POIs too, not plain dots — load_poi_icons() works UNCHANGED against SIRIUS.DB3 since the same Image_BaseAttributes/ ImageBlob_BaseAttributes tables exist in its schema. Visually confirmed: partition 1 ("Fuel Station") decodes to a real, clean gas -pump icon. App._get_sirius_poi_icon() (identical shape to the regular-POI version), falls back to the plain teal dot only if a partition has no resolvable icon.

The canvas matches the real firmware's own screen shape — read live, nothing hardcoded ("v40 → v41")

Investigated what's recoverable from FHDD6.FLI's own Java UI code — 123 real embedded ZIP/JAR headers and several PNG/GIF-tagged byte regions exist, but none panned out as directly usable "menu look" assets. A real overclaim caught and corrected, user-pushed ("why not"): an early pass claimed "156 entries, confirmed via a real, parseable Central Directory" — wrong, trusted too quickly. A real sequential walk (not just the EOCD's own summary fields) found only 1 entry genuinely self-consistent; only 4 real PK\x01\x02 markers exist in a 30,000-byte window near the EOCD, nowhere near 156 — this archive is fragmentary, not a complete JAR (and even an intact one would only yield more compiled bytecode, never image/layout data). The 3 PNG hits are false positives (no valid IEND); the GIF-tagged regions have a real header and palette but an invalid block marker right after — some proprietary format borrowing GIF's header convention, not real GIF data; PIL correctly refuses it.

What WAS solidly recoverable and is now implemented: the real screen resolution, 800×480, confirmed 2 independent ways here (a real java.awt.Rectangle-shaped widget-bounds string, and 11 of the GIF-tagged regions' own palette dimensions agreeing) — a 3rd and 4th independent confirmation came later, see Theft Protection and SWL Disc: Boot Chain and Build System. New, fully dynamic, nothing hardcoded: rns510_iso.find_file() walks a real disc's own directory tree for a filename pattern (necessary since EU vs. North-America discs genuinely differ in folder layout — see Sirius TravelLink and North America Discs); swl_5238_reader.find_screen_resolution() scans raw FHDD6.FLI bytes for every widget-bounds string and returns the most common (width, height) pair (a real Counter, correctly ignoring a non-full-screen 100×50 logo widget found the same way); File > Open Firmware ISO (real screen shape)... runs both in a background task and reshapes the canvas (letterboxed/pillarboxed, live on resize). Tested against all 3 real firmware ISOs on hand: 2 resolve to exactly (800, 480); the 3rd (the newest, largest NAR build) returns None — a real, honest per-build limitation (that build's compiled UI code just doesn't contain the resolution-revealing string), not a bug.

A real, still-open splash-screen lead followed further, same session: H_PQEE.FRG's per-variant "remainder" region (past the shared BSP stream, see HOST BSP and Vehicle Coding) is 772,577 bytes; 800×480×2 (RGB555) = 768,000 — a promisingly close match to the real leftover 4,577 bytes (already known to hold a real VW part number/"RNS-MID" text). Rendered as raw 800×480 BGR555 (the same convention validated for the Sirius .DDB format): a clean, regularly-repeating textured/banded image, NOT random noise, but not recognizable either. FFT autocorrelation found a broad periodicity plateau near 700-742px, not one sharp value. Checked for zlib compression directly — zero real zlib headers anywhere in the remainder. 3 more real attempts, made DEFINITIVE: no plausible width/height header field in the first 64 bytes; RGB565 at 4 candidate widths produced the same banding, just a different hue (ruling out "right structure, wrong color format"); the decisive test — swept EVERY candidate width from 100 to 1600px and measured real row-to-row similarity (a standard, width-agnostic raw-image- forensics technique) — a true fixed-stride image shows one sharp minimum at its real width; this data shows none (the best 20 candidates differ by under 2% from each other). This rules out "raw, fixed-stride, 2-bytes-per-pixel bitmap, at ANY width" as a category, not just the specific widths/formats already tried — a materially stronger negative result. Whatever this data is, it needs either a fundamentally different structural hypothesis or independent ground truth (a real photo of an actual unit's boot screen), not further blind guessing.

2 separate search boxes — "Address" and "POI" ("v41 → v42")

The single search box (road+city+POI all at once, MapData.search_combined()) is replaced in the UI by 2 separate boxes, each with its own Entry + Search button, stacked (Address: above POI:), sharing ONE results list below. New MapData methods, search_address() (roads+cities) and search_all_pois() (regular POIs + Sirius POIs together) — factored out of search_combined()'s own logic, left completely UNCHANGED (test_map_viewer.py's existing tests still call it directly). SearchHit gains a 4th real kind, "sirius" (renders [Sirius] instead of [POI]). Verified directly: search_address("SOFIA") returns 20 real hits; search_all_ pois("MCDONALD") returns real POI.DB3 matches; a loaded Sirius database searched for "CHEVRON" returns 5,682 real [Sirius]-tagged hits. A window screenshot confirms the 2-row layout renders as intended.

Road search results now zoom to the real segment size, not a flat default

SearchHit.jump_span_deg() previously had a real branch for "city" hits (scaled to roughly fit the city's own bounding box, clamped to [0.01, 2.0] degrees) but "road" hits just fell through to the caller's flat DEFAULT_JUMP_SPAN_DEG (0.3°, ~33km) regardless of whether the matched road was a short residential street or a long highway stretch — double-clicking ANY road result zoomed out to roughly the same county-sized area.

A later session's crack of eeu.rd's delta_long/delta_lat (each matched segment's own real bounding-box width/height, now exposed directly on core.RoadRecord) made a real fix possible: "road" hits now get their own branch, mirroring the city one — span = max(delta_lon, delta_lat) * 2.0, clamped to [0.01, default_span]. Since the real median segment diagonal is only ~350m, almost every road search result now zooms in tight enough to actually see the matched street (floor-clamped to 0.01°, ~1.1km); a genuinely long route segment still gets a proportionally wider span, up to the old flat default as a ceiling. Verified against real search results from the reference disc (core.MapProject.search("VIA") and others) — real named streets now produce ~0.01-0.012° spans instead of the old flat 0.3° for every single one.

Opening the same disc twice no longer re-pays the slow "Building geo-index..." cost

A still later session cracked the real on-disk structure behind the slow build_geo_index()/build_bbox_index() progress messages — a genuine k-d tree, found via DBAL.OUT disassembly (see MAP_COMPRESSED Container Format's own "Geo-index prefix" section for the full crack) — but 2 real gaps (the split coordinate's own numeric scale, and kd_pdecend/ kd_rdecend turning out to be a full rectangle-intersection range query rather than a simple point descent) meant swapping it in as the viewer's primary tile lookup right now would risk silently WRONG tile resolution. Not worth the risk for a speed win.

Shipped instead: a real, safe fix for the actual user-visible complaint. build_geo_index()/build_bbox_index() are unchanged; MapData now has a persistent geo_index_cache_dir (next to the script, NOT self.workdir, which is a fresh tempdir every run) and both call sites go through new build_geo_index_cached()/ build_bbox_index_cached() wrappers — a pickle cache keyed by the ISO's own identity (path + size + mtime) and the exact byte span being indexed, written atomically (tmp file + rename) so a killed process never leaves a corrupt cache entry. Re-opening the SAME disc a second time skips the decode-every-tile cost entirely.

Verified, not just implemented: cached and freshly-built results are bit-for-bit identical for both indexes. Real measured numbers against the reference ISO: a full MapData.load() dropped from 37.76s to 6.89s on a second run; a single layer's geo_index alone dropped from 1.30s to 0.010s (130x) once cached.

The "2 real gaps" above turned out to be a 4-byte reader bug — the k-d tree is now the primary bbox source

A still later session resolved both gaps the previous section declined to risk: cross-referencing eeu.mod's own schema (the disc's real data dictionary, see eeu.mod) found a 4-byte maxPclId field sitting between mapHeader and the k-d tree's own node array that the existing reader had never skipped — a plain off-by-4-bytes bug, not an external-schema wall, even though it produced symptoms (disconnected nodes, implausible edge values) that looked exactly like one. Fixing it makes a full tree traversal reach 100% of nodes and leaves on every layer tested, and edge turns out to use the SAME /100000-degree convention as every other coordinate on this disc — see MAP_COMPRESSED Container Format's "Geo-index prefix" section for the full validation numbers (median ~0.003-0.013° agreement against independently-decompressed content boxes, a real point-lookup matching 200/200 self-consistency checks).

MapData._build_layer_bbox_index() now tries mcr. build_kdtree_bbox_index() FIRST — a single structural tree traversal with NO tile decompression at all — falling back to the previous section's build_bbox_index_cached() only if the k-d tree can't be parsed for that layer (e.g. eeuz.fea, a different table format entirely) or comes back empty. Verified end-to-end against the reference ISO: load_bbox_indexes() (all 4 FAST_LAYERS) now takes ~2.1s total with 100% tile coverage on every layer (mg4 6,110/6,110, mg3 16,503/16,503, mg2 34,209/34,209, mg1 63,996/63,996 — previously 5,935-63,970, content-decode-dependent), and the full test suite (test_map_viewer.py) run end-to-end against both the old and new code produces identical results up through the point where an unrelated, pre-existing test failure (a canvas-pixel ground- truth check, confirmed present on the unmodified baseline too — not a regression from this change) halts the script in both cases. mp0 (the heaviest layer, 194,705 tiles) also goes through the new path as part of load_heavy_layer()'s own bbox-index step, reaching 194,705/194,705 (was 194,650/194,705) in under a second (previously up to several minutes).

Confirmed highways now render in their own color, using eeu.rd's newly-cracked highway bit

The connected-roads overlay previously colored every named road edge the same way (ROAD_COLOR_MAJOR, gold) regardless of real importance — the only signal available was n_vertices (a named run's point count), a crude proxy the module's own docstring already flagged as such ("no reliable road-class field exists"). Now that eeu.rd's roadInfo byte is fully cracked, a confirmed highway (bit 0 of that byte) gets its own color (HIGHWAY_COLOR, a strong green) and a thicker line — a REAL per-road signal, not a heuristic.

research/road_naming.py's RdCache now also vectorizes the roadInfo byte and exposes highway_names() — the set of real road names with at least one highway-flagged record, built with the exact same garbage-byte-clearing + numpy.unique() dedup technique distinct_names() already used. MapData.highway_names builds and caches this lazily (same policy as street_name_index_global), and the edge-coloring code in _redraw() checks it before falling back to the old name-length-based coloring.

Deliberately NOT touched: the per-vertex dot rendering. An earlier session fixed every dot to a single, unambiguous DOT_COLOR (red) at the user's own explicit request ("make the points red dots not gray ones") — this is a separate rendering path (the connected-roads line overlay, which already used multiple status colors) and doesn't revisit that decision.

Verified end-to-end against the real ISO: 578,946/8,809,081 records (6.57%) flagged, matching this project's own earlier-measured 5.9% route-like-name baseline closely; real route-numbered names (SS24, K4309, L606.3) correctly appear in highway_names(), an ordinary street name (MONGINEVRO) does not.

eeu.iof "nearby streets" lookup now fires on search-result jumps too, not just map clicks

MapData.nearby_streets_for_point() (the real eeu.iof/eeu.il "nearby street names" anchor lookup) already existed and was wired into the click-to-identify path (on_canvas_click() -> _append_nearby_streets_row()) — but searching for a road BY NAME and double-clicking the result to jump there (on_go_to_result()) never triggered it, even though the exact same data (rd_cache/search_project, already resident) is equally available at that point. Wired it in for "road"-kind hits, right after _jump_to() kicks off — the lookup only needs already-loaded in-memory arrays, so it runs synchronously rather than racing _jump_to()'s own async done() status-bar update.

(This investigation started from a still later session's firmware disassembly of eeu.iof's type byte — see that file's own wiki page — which turned out NOT to need any UI change: the newly-cracked type==1 "provisional count, recomputed at runtime" case covers only 7 of 31,318 real anchors (0.02%), confirmed directly against the reference disc, far too rare to justify special-casing. The actual gap found instead was this missing search-path wiring, unrelated to the type byte itself.)

Verified end-to-end against the real ISO: searching "MONGINEVRO" and jumping to one of its results resolves the real eeu.iof anchor "VITTORIO EMANUELE II" 1.7km away, with its full real cross-street list (VIA ROMA, SP233, SS24, VIA GIUSEPPE GARIBALDI, ...).

Real bugs found and fixed along the way, beyond the version-by-version entries above

• _initial_scale() outlier sensitivity. A single decoded feature can legitimately be a long chain whose far end is a meaningful fraction of a degree away from the searched point. Letting one such outlier set the initial auto-fit zoom collapsed a real Tirana-area load to a near-country-wide scale, at which point far-away-but- genuinely-large cities (Bari, Ancona, Split — real Adriatic port cities, just hundreds of km away) out-ranked Tirana's own label. Fixed by clipping the auto-fit's bounding-box calculation to a generous multiple (2.5×) of the requested jump span — only affects the one-time initial zoom choice, not what gets drawn once framed.
• A minority of eeu.cty records have an implausibly huge bounding box. Found while tuning zoom-dependent city importance: a small number of records — so far only observed among postal-code-tagged sub-entries, e.g. "185 45, PEIRAIAS" (real Piraeus is a compact Athens-area port city) — have a bbox spanning tens of degrees (one observed case ~34 deg², most of the southern Aegean/Balkans for what should be one postal code's neighborhood), sitting right next to sibling entries for the SAME place with normal, tight boxes. Not a decode-offset bug (surrounding fields on the same record decode sanely, neighboring records at adjacent offsets are fine) — a genuine data-quality quirk in a minority of source records. Since "rank by bbox area" is this project's own established zoom- importance proxy (see eeu.cty), an unfiltered ranking lets these degenerate records win every low-zoom label slot by sheer spurious size. Mitigated with an optional max_area cap on CtyCache.query() (the map viewer passes CITY_MAX_AREA_DEG2 = 2.0, comfortably above any genuine city/admin area observed — Timișoara's is ~0.1) — the reader itself still returns the raw, unfiltered records, a filtering consumer opts in.
• A related, subtler case: even with the above fix, a WIDE viewport can still rank a real (not degenerate) but much bigger surrounding admin area's box above the actual place at the view's center — observed directly: a rural municipality ("ZALL BASTAR") northeast of Tirana has a bbox that also legitimately covers central Tirana's coordinates. App._ensure_center_city_included() looks up whichever real .cty record most TIGHTLY contains the exact center point (smallest bbox among all containing records) and makes sure that one specifically is represented.
• A corrupt fallback-path anchor near Iași: on a real mg3 tile, find_tile_anchor()'s "bruteforce" fallback path produced an outright corrupt anchor/feature thousands of km from the tile's true location. Defensively guarded (MAX_FEATURE_DRIFT_DEG=3°, drops any feature straying that far from its own tile's anchor) rather than root-caused — the underlying fallback path in research/map_compressed_reader.py could use tighter verification.
• Long straight "spike" lines near Sofia, Bulgaria, root-caused and fixed at the source: 2 decoded features oscillating 24-33km between two near-fixed latitude bands, well within the 3° drift guard above (so that guard alone didn't catch it). Root cause: decode_features() running a block's declared point_count past where its real geometry ends, into misread non-coordinate bytes. Fixed by calling decode_features(raw, declen, trim_oscillation=True) in MapData.decode_tile(), which removes both Sofia spikes cleanly.
• A follow-up near Turin, Italy found the same oscillating-overrun signature slipping through the original single-cutoff fix in 2 ways: a 2nd cluster hiding inside what the old code reported as a "clean, kept" prefix, and a cluster the old code missed entirely because it only ever looked for the FIRST one. decode_features()'s detector and reassembly were redesigned (multi-cluster scan + split into surviving sub-features instead of truncating). Re-running the same jump-detection check that originally found the Sofia bug, over the same Turin-area viewport: single-step jumps >1km across all pooled features dropped from 551 to 399 (max jump 10.4km→8.1km, total "wasted" jump distance roughly halved) — a real, measured improvement, though not total elimination: one specific tile's corruption remains uncaught and is still visible among the remaining jumps in this same viewport.

Known limitations (explicit, not oversights)

• tile_ids_in_bbox()/find_tile_for_coord() are anchor-inside-box/ nearest-anchor, not exact tile-boundary containment (no tile extent was ever recovered) — a coordinate near a tile edge might miss a technically-adjacent tile's content until the viewport nudges further.
• Road "importance" styling (thicker/darker for longer named runs) is a vertex/run-length proxy, not a real road-class field.
• decode_features() still can't split one rendered chain back into per-named-road segments at the geometry level (the topology table that would allow this is still uncracked) — name_features() only ATTACHES names to vertex ranges of the geometry as already decoded, it doesn't re-cut the polyline. Roads render as unconnected per-vertex dots by default, so this no longer produces a visible wrong-line artifact — the underlying data gap is unchanged, only its visual symptom is gone (see Tile Geometry and Topology).
• The eeu.cty huge-bbox data quirk above is capped for display, not explained — a future session could investigate which record types/fields predict it.
• mp0 (the densest layer, build_geo_index() measured 34-55s on the reference disc) is not opened eagerly — it loads in the background right after "Open Map ISO" and folds into the same pooled view automatically once ready. Before that finishes, the view shows only whatever the 4 fast layers have — not an error, not a block — which can still look near-empty in an area that's sparse in every fast layer but has real content in mp0 (measured: a tight Sofia-area box has 0 tiles in mg1-mg4 but 1 tile/2 features/426 points in mp0). A real, un-eliminated rough edge: there was no way found to make mp0's own geo-index build meaningfully faster (build_geo_index() already reads every tile's exact byte range once, no wasted I/O), so a background window of some length is inherent to the current approach.

Non-GUI functional test: test_map_viewer.py (py test_map_viewer.py) — covers viewport-bbox math, in-memory caches, unified/split search, dynamic area loading, real names/cities actually appearing in decoded data, actual canvas rendering (item counts + label text) driven with real ISO data, a GUI-construction smoke test, and multi-layer pooling behavior end to end (including a real-ISO scale sweep proving monotonic/additive point-count growth and cache reuse on re-zooming). test_core.py covers the separate editing tool (rns510_gui.py).