Skip to content

Complete manifest is 89% commits (318 MB on Linux): cap, split, or slim delivery #151

Description

@thalida

After #150, the first paint no longer waits on commits — but the complete manifest still carries the entire history: 317.9 MB of the 358.5 MB Linux payload (89%) is the commits array (~1.43M entries). It's gzipped on the asyncio event loop (sse_compression.py:62, blocks the loop for seconds at this size) and JSON.parsed on the browser's main thread in one go.

Who actually reads commits

  • Trees + fireflies (decorations): one tree per commit, capped at TREE_MAX_CELLS = 100k cells — so decorations can never use more than a fraction of 1.43M entries anyway.
  • The almanac / stats panes: superlatives and counts, already computed server-side into stats by compute_repo_stats.
  • Capture shots: sorts by author count to find a placed tree.
  • The Timeline feature has its own endpoint (/api/timeline) and bundle — it does NOT read Manifest.commits.

Directions to discuss (not decided)

  • Cap what ships: decorations only consume up to the tree cap; send the most-recent N (or a sampled spread) plus the server-computed aggregates the panes need. Changes what "one tree per commit" means on mega-repos — it's already capped, so maybe nothing visible changes.
  • Split delivery: a separate /api/commits fetch (or a fourth SSE stage) so the manifest stays slim and history loads fully in the background, parsed off the main thread in a worker.
  • Slim the entry: authors[], subject, same_day_total per entry are the bulk; the decorations need sha/date/files, little else.

This touches the decorations + almanac contract, so it needs a decision on intended behavior for very deep histories, not just a mechanical change. Numbers and stage timeline: #147, #150.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions