Point-window cache (T13) + resolvability gate (#225)#227
Draft
masenf wants to merge 3 commits into
Draft
Conversation
…dows Fixes #225: zooming out from a drilled window brought fixed-rate "sampled points" back over the density surface — individual marks at a zoom where real points are sub-pixel, misrepresenting the dataset. Now: - Interactive density_view replies ship no point sample at all. The mean-color density surface (LOD doc §2) stands alone above the drill budget; real points still arrive the moment a window fits it. The retained first-payload sample (the standalone re-bin worker's CPU source) draws only below a resolvable-count gate: estimated in-view count <= LOD_DIRECT_POINT_BUDGET (lodOverlayResolvable, T9). Datasets under the budget keep the hybrid look everywhere. Point caching (LOD doc T13) so pan/zoom stops re-shipping the same points: - Kernel: a points-tier reply ships the largest ALIGNED window around the view whose exact count fits the budget (lod.aligned_window: bounds snap outward to a power-of-two grid over the trace's extent, per dimension; DRILL_PAD_TARGETS ladder, pyramid-count-gated, exact-verified, DRILL_PAD_SPAN_CAP guards the §16 re-encode ladder). Consecutive pans resolve to the SAME window; lod_blend stays keyed on the view's own count. Nonlinear axes keep the exact view window. - Client: a reply for a new window retires the previous exact drill into a bounded per-trace cache (lodRetireDrill/LOD_POINT_CACHE_WINDOWS) instead of overwriting its buffers — so does a drill dying outside its window — and any later view covered by a cached window promotes it back alpha-continuously with zero round-trips (lodPromoteCachedDrill). Outgrown windows free geometry-only (T11 rule, §27). - Kernel keeps a bounded drill-subset history (Trace.drill_history, DRILL_HISTORY_KEEP) so picks against a promoted retired window still translate exactly; expired seqs drop the pick, data changes clear it. - Client suppresses identical density_view requests (same window, screen, unchanged data) and accepts the suppressed duplicate's in-flight reply per-trace instead of killing it with the bumped seq — ending the re-request loop at gesture end. Spec: lod-architecture T9 rewritten + new T13, wire-protocol density_view/points/pick contracts, dossier constants table, CHANGELOG, README representation labels, docs ladder table. Tests: new test_drill_window_padding (alignment, budget, span cap, nonlinear skip, history clear), test_density_sample_resolvability_gate and test_drill_point_window_cache (headless-Chromium client probes), elision test extended for duplicate suppression; drill tests updated to the padded-window contract.
masenf
marked this pull request as draft
July 23, 2026 12:50
Greptile SummaryThis PR revises the density LOD path and exact-point drill behavior. The main changes are:
Confidence Score: 5/5This PR appears safe to merge. No verified issues were found in the reviewed client LOD, kernel interaction, or Python density paths. The changed behavior has targeted browser and unit coverage for the main new flows. No files require special attention.
 T-Rex LogsWhat T-Rex did
Important Files Changed
Reviews (3): Last reviewed commit: "The aggregate never refines: density req..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
…yered draw Field HAR from the 100M live drilldown (PR #227 review): every pan/zoom step at 200-450% zoom re-requested a ~2.7 MB full-screen density grid — including back-to-back windows differing by sub-pixel amounts — and the frame dropped to the blurriest cached texture whenever a pan left the freshest window. Three fixes, all recorded under LOD doc T13: - Kernel: pyramid-served grids are clamped to the source cells the finest level resolves under the window (interaction._pyramid_source_shape, ceil(base*frac)+1 per axis) — composing to full screen resolution just upsamples blocky cells the client's texture filtering reproduces from the source-resolution grid at a fraction of the bytes. The attainable per-axis cell size rides the reply as density.min_cell; exact and spatial grids (true full-detail bins) omit it. The HAR's 2.7 MB replies drop to ~0.5-1.4 MB in its zoom range and ~200 KB deeper. - Client request elision (lodDensityCacheServes): a cached texture that contains the view and is as detailed as anything the kernel could return — one texel per screen pixel, or already at min_cell (zooming into a source-limited texture cannot sharpen it) — answers with no round-trip. Guarded so the kernel stays in charge where it can do better: estimated in-view count must exceed budget x LOD_DENSITY_ELIDE_EST_FACTOR, and the cached window may be at most LOD_DENSITY_ELIDE_MAX_AREA_RATIO x the view area (uniform-density estimates overshoot in sparse corners). Entries without recorded counts never elide; lodRememberDensity now supersedes an unpinned same-window twin so elision always reads current facts. - Sub-texel request dedup: the identical-request memo compares windows numerically within half an output texel per edge (gesture-end and settle emit sub-pixel twins; a grid shifted below half a texel is the same picture) instead of exact string equality. - Finer-detail layering (lodDensityDetailForView): when no fine cached window contains the view, the smallest cached texture overlapping >= 5% of it draws on top of the broad backdrop after every crossfade layer, so the already-fetched region stays sharp while the request for the rest is in flight — a resolution seam beats uniform blur. Spec: lod-architecture T13 extended, wire-protocol density_view/density contracts (min_cell, clamped grids, elision), dossier constants table, CHANGELOG. Tests: test_density_wire_economy (source-shape math, min_cell presence/ absence, 20M-point clamp integration), test_density_request_economy (headless-Chromium probe: source-resolution elision + guards, sub-texel dedup, fine-over-broad layering).
Field feedback on PR #227 (100M live drilldown), superseding the previous commit's adequacy-elision approach with a simpler rule: the density texture already covering the view STANDS — however blurry — until the view could plausibly resolve into real points. Intermediate-zoom blur is an accepted, recorded cost; the kernel's sharper answers (exact re-bins, drill-in) stay reachable because the points band still requests. - Client gate (lodAggregateStands): a density_view goes out only when the estimated in-view count sits within LOD_POINTS_REQUEST_BAND (4x) of the direct budget. Two estimators, LOWER wins so an over-estimate can never strand a view in blur: the smallest cached density window containing the view (recorded count, area-scaled; seeded by the home texture's first-payload count) and the retained deterministic sample counted in-view and re-weighted (lodSampleViewCount) — the sample follows the data's actual distribution, so sparse tails reach their points where uniform-density area-scaling over-estimates by orders of magnitude. Traces with no recorded counts always request. - Kernel-attached clients never draw the retained sample at ANY zoom (field capture: 8k arbitrary sample rows at full alpha over a view the kernel serves real points for): resolvable views get exact points, and the sample's kernel-mode job is now purely CPU-side estimation. The standalone client keeps drawing it below the resolvability gate — its only point representation. - Reverted the fine-over-broad detail layer from the previous commit (field capture: a stale darker rectangle) — density textures alpha-composite, so the overlap double-counts opacity, and per-window normalization makes the seam a brightness step. Recorded in T13; the tier draws one texture per frame. Dropped the density.min_cell reply field with it (no reader remains); the source-resolution grid clamp and the sub-texel request dedup stay. Spec: lod-architecture T9/T13 revised (never-refines rule, sample-based estimator, recorded reversal), wire-protocol, dossier constants (LOD_POINTS_REQUEST_BAND replaces the elision guards), config comment, CHANGELOG, docs ladder table. Tests: request-economy probe rewritten to the stands-rule (band requests, sub-texel dedup, single-texture frames), resolvability-gate probe gains kernel-mode assertions, wire-economy tests drop min_cell.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements two interconnected LOD improvements: a per-trace LRU cache of retired exact point windows (T13) that eliminates re-requests on pan/zoom sequences, and a resolvability gate (#225) that prevents sample overlays from drawing above the direct point budget where they misrepresent the dataset.
Key Changes
Point-Window Cache (T13)
LOD_POINT_CACHE_WINDOWS = 3constant to bound retired window memorylodRetireDrill(),lodPromoteCachedDrill(), and supporting functions to manage the per-trace LRU cache of exact point windowslodWindowServesView()as a shared containment check for both live drill and cached windowsResolvability Gate (#225)
lodOverlayResolvable()function that gates sample overlay drawing on estimated in-view countoverlay.sample.visible * (viewArea / windowArea) <= LOD_DIRECT_POINT_BUDGETdensity_viewreplies no longer ship point samples at all (only the first-payload sample is retained for the standalone re-bin worker)lodSampleForView()to apply the gate before considering overlay candidatesPadded Aligned Windows (T13 support)
aligned_window()and_padded_drill_window()to compute the widest aligned window that fits the budgetDRILL_PAD_SPAN_CAP) to keep deep zooms inside the §16 re-encode ladderDrill Subset History
drill_historydict toTraceto remember recent shipped subsets (bounded byDRILL_HISTORY_KEEP)drill_history()returns the shipped subset for a givendrill_seq, or None when expired (drops the pick)clear_drill_history()forgets subsets after data changesIdentical-Request Suppression (T13)
_densityRequestDup()to detect when the same window at the same screen size is already in flight or answeredClient-Side Changes
lodClearDrillState()consolidates drill lifecycle reset (used by drop, retire, promote)lodFreeDrillBuffers()frees all GPU buffers for a drill objectlodSameWindow()compares windows with floating-point toleranceNotable Implementation Details
(span * 1e-9 + 1e-300)to handle floating-point slopvisiblecount on the sample metadata; overlays without recorded counts (hand-built/legacy specs) keep drawinghttps://claude.ai/code/session_01XQT3v7NdL4dmC6S3g8HjJt