Show the city at ~25s instead of ~3min on a big cold load - #150
Merged
Conversation
A subtree grows away from its origin along the mirrored axis, so its rect list can never be symmetric about 0 — but isMirrorInvariant only learned that after an O(n²) scan over up to ~1000 rects. An extent check settles those cases in one pass, and hoisting the mirrored coordinates out of the inner loop cuts the remaining work. 93k-file layout: isMirrorInvariant 128ms → 35ms. Refs #147 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
On linux the history walk is ~135s of a ~180s cold load, and it ran before the tree walk, so the first manifest waited on all of it. Nothing the skeleton draws needs it: the packer keys off structure and per-file size, which is exactly what layout_signature hashes. The walk now records filesystem dates, the skeleton goes out right after it (~10s), and _apply_git_dates overlays history onto the tree and its directory rollups when that walk finishes. The skeleton also carries no commits, which are 318 MB of the 358 MB linux payload and which nothing in it reads — so the commit-derived passes in _wrap_manifest, previously run once per manifest, now happen once. The signature pre-walk moves behind the cache check too: it stat-walks the whole tree to build a key that scan_tree computes anyway, so a no-cache scan paid for the walk twice. Refs #147 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The city app moved under app/src/, constants/ is gone, and layoutV4.ts no longer exists — layout now lives in app/src/city/layout/. The bit-identical snapshot contract still holds, so it moves with the path. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The overlay waited for the whole stream, so a finished city sat hidden behind the git history walk — minutes of it on a big repo, for data that only draws trees and the timeline. Per-file metadata now runs before the history walk and gets its own emit, so the scan streams structure, then real building heights, then history. Each manifest declares what is still to come in `pending`, and the overlay lifts once a manifest with no pending metadata has painted. On linux that puts the city on screen at roughly 30s instead of ~170s, with trees and the timeline filling in when history lands. Refs #147 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…gnal The reveal check read MANIFEST, which still holds the previous repo's finished manifest when a new load starts — so on any repo switch the "metadata resolved" test passed instantly, the reveal latched, and the overlay never showed. Two latches and a misnamed helper were papering over that stale read. The pump now records the applied manifest's `pending` on SCAN_PROGRESS, written after the apply so it can never run ahead of REBUILD_STATUS, and absent until this load's first partial applies so nothing stale can leak in. The overlay reaction is back to one state bit: hold while the applied manifest still owes metadata, lift when the real-heights paint lands, stay down while history streams behind the live city. Refs #147 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cold-loading a big repo took ~3 minutes with nothing on screen. The city is now visible at ~25s, and the scan keeps streaming behind it.
All numbers are
torvalds/linux(93,455 files, ~1.43M commits), scanned withno_cache=trueagainst a clone already on disk, same machine.Before / after
What changed
The git history walk no longer blocks the first paint. It was ~135s of the load and ran before anything else, but the packer only needs structure + per-file byte size (exactly what
layout_signaturehashes). The scan now emits in three stages: skeleton (structure, fs dates, no commits) → metadata (real per-file lines, the emit worth revealing on) → complete (history: commits, git dates)._apply_git_datesoverlays history dates onto the tree and its directory rollups when the walk finishes; both signatures are date-free, so the frontend keeps the layout it packed from the skeleton.The skeleton carries no commits. They're 317.9 MB of the 358.5 MB payload (89%) and nothing the skeleton draws reads them. First payload drops 358 MB → 38.6 MB, and the commit-derived passes in
_wrap_manifest(compute_repo_stats,_annotate_same_day_totalsover 1.43M commits) run once instead of twice.Each manifest declares
pendingstages (["metadata","history"]→["history"]→[]), so consumers know which fields are provisional instead of guessing. Cache schema bumped to v23.The overlay lifts on real heights, not on stream end. The pump records the applied manifest's
pendingonSCAN_PROGRESS(written after the apply, so it can never run ahead ofREBUILD_STATUS); the overlay reaction holds while metadata is owed, lifts when the real-heights paint lands, and stays down while history streams behind the live city. Trees + timeline pop in when history arrives (~170s).Redundant work removed. The signature stat-walk of all 93k files now runs only when there's a cache to check (a
no_cachescan paid for it twice —scan_treecomputes the same digest as it walks)._build_tree'spending_entries.pop(0)was O(entries²) per directory; now a cursor.isMirrorInvariantrejects asymmetric extents in O(n) before its O(n²) pair search (128ms → 35ms at 93k — the one layout-solver win that survived measurement; the issue records everything that didn't).Verification
_collect_git_historyruns (verified red without the reordering) and that skeleton/finallayout_signaturematch.Out of scope
The complete manifest is still 358 MB because it carries all 1.43M commits; slimming that changes the timeline/decorations contract — split out as its own issue.
Closes #147
🤖 Generated with Claude Code