feat(map): pull extreme outliers closer + centered computing overlay#5665
feat(map): pull extreme outliers closer + centered computing overlay#5665MarkusNeusinger merged 5 commits intomainfrom
Conversation
Closes #5664. Outlier handling - New custom d3-force `outlier-squash` that activates only beyond the 95th-percentile distance from the cluster centroid. Inside the threshold the inner geometry is untouched; outside, distances are compressed via a sigmoid-like map (asymptote = R + k) so outliers remain visibly separate but bounded. This stops a couple of far-flung null-bucket specs from collapsing zoomToFit's framing without needing stronger global gravity (which would crush clusters). - Tuning: percentile 0.95, k = 200 graph-units, strength 0.18. Loading state - Replaced the small bottom-right "arranging" pill with a centered spinner + label + slim progress bar. Backdrop is dimmed and blurred so users register that the canvas is in a transient state without losing the underlying preview. The overlay fades out on opacity transition rather than unmounting, so re-derives (filter / weight changes) fade it back in cleanly. - Progress bar driven by a throttled tick counter (every 6 ticks, ~10 Hz) wired to onEngineTick - accurate and avoids re-rendering at the full simulation frequency. Lint cleanup in the same file - Refactored four set-state-in-effect violations to render-time patterns (last-non-null ref for panel content, "store previous prop" pattern for gate re-arm, JSX gating for the pin marker, inline reset in the search input handler). - Filled in the missing nodeById deps on the pin RAF effect and panelData memo. https://claude.ai/code/session_01XcRMey5z2WQNhZJQnX2Vtu
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
There was a problem hiding this comment.
Pull request overview
This PR refines the /map experience by making the initial layout feel more intentional: it adds a custom force to keep extreme nodes from stretching the viewport and replaces the old corner “arranging” indicator with a centered loading overlay. It fits into the map page’s broader goal of making the force-directed graph understandable and usable for non-technical users.
Changes:
- Added a custom
outlier-squashd3 force to compress only the farthest nodes without changing the inner cluster layout. - Reworked the settling/loading state into a centered spinner + progress overlay that fades instead of unmounting.
- Refactored several
MapPagestate flows away from effect-driven updates and adjusted the existing settling-overlay test.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
app/src/pages/MapPage.tsx |
Adds the outlier-squash force, throttled progress tracking, centered settling overlay, and related state-management refactors. |
app/src/pages/MapPage.test.tsx |
Updates the settling-overlay test to assert accessibility-visible status behavior during fade-out. |
| dists[i] = Math.hypot(node.x - cx, node.y - cy); | ||
| } | ||
| const sorted = dists.slice().sort((a, b) => a - b); | ||
| const R = sorted[Math.floor(sorted.length * percentile)]; |
| fg.d3Force( | ||
| 'outlier-squash', | ||
| outlierSquashForce( | ||
| OUTLIER_THRESHOLD_PERCENTILE, | ||
| OUTLIER_SQUASH_K, | ||
| OUTLIER_SQUASH_STRENGTH, | ||
| ), |
Addresses Copilot review feedback on #5665: 1. Off-by-one in the percentile cutoff. With Math.floor(n * p) and p = 0.95, any graph of n <= 20 nodes gets R = max(distances), so the strict comparison r > R is never true and the squash force silently no-ops. This is exactly when filtered subsets land — the cases that most need outlier handling. Switched to floor((n - 1) * p) (numpy's linear/lower percentile interpolation), which keeps at least the single most-outlying node above R for any n >= 2. Also early-return for n < 2 so single-node subsets don't index into a length-0 array. 2. Test coverage for the new force math. Exported outlierSquashForce and added six unit tests covering: - empty / single-node graphs (no-ops) - inner-cluster geometry preservation - off-by-one regression guard for small graphs - bounded velocity correction at extreme distances (asymptote check) - linear scaling of correction with alpha (cooling) 3. Test coverage for the new UX surface: - onRenderFramePre registers the outlier-squash force on the d3-force simulation (mock now captures the (name, fn) pairs passed to fg.d3Force). - onEngineTick advances the progress bar's aria-valuenow once the throttle threshold is crossed. 4. The hand-rolled progress bar now exposes role="progressbar" with aria-valuemin/max/now/label so it's announceable to assistive tech and discoverable from tests. https://claude.ai/code/session_01XcRMey5z2WQNhZJQnX2Vtu
- Couple d3-force alpha decay to COOLDOWN_TICKS so the loading overlay doesn't fade out before the progress bar reaches 100 %. - Drop COOLDOWN_TICKS 450 -> 300 and bump alphaMin 0.001 -> 0.01 so the engine stops where motion stops being perceptible (no dead tail). - Outlier squash K 200 -> 120: pull extreme outliers further toward the threshold ring (~40 % tighter asymptote) without touching inner cluster geometry. - Rebrand the loading text from "Arranging map..." to "map.simulate()" to match the any.plot() function-notation style. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
| // collapses to a dot because of one runaway outlier" zoomToFit problem | ||
| // without needing stronger global gravity (which would crush clusters). | ||
| const OUTLIER_THRESHOLD_PERCENTILE = 0.95; // distance percentile beyond which compression starts | ||
| const OUTLIER_SQUASH_K = 120; // graph-units of extra room outliers can use beyond R; smaller = harder squash |
There was a problem hiding this comment.
The actual code at line 76 reads const OUTLIER_SQUASH_K = 200; — it matches the PR description exactly. There's no 120 literal anywhere in the file (grep OUTLIER_SQUASH /home/user/anyplot/app/src/pages/MapPage.tsx confirms). I think this comment was generated against a phantom value that doesn't exist in the diff.
Generated by Claude Code
There was a problem hiding this comment.
Correction: my prior reply was wrong. I had stale local state and didn't see commit 293e1d1 (fix(map): tighten cooldown timing, squash outliers harder, brand overlay) which intentionally lowered OUTLIER_SQUASH_K from 200 → 120 ("~40 % tighter asymptote"). Copilot is right that the code now reads 120 while the PR description still says 200 — that's a stale doc. The user's tuning is deliberate; I'll update the PR description to reflect the actual shipped value rather than reverting the code.
Generated by Claude Code
| color: 'var(--ink)', | ||
| }} | ||
| > | ||
| map.simulate() |
There was a problem hiding this comment.
The actual label rendered at line 1553 is Arranging map… — plain English, no jargon. There's no map.simulate() string in the file (grep confirms). Same as the previous comment, I believe this is a phantom — the diff doesn't contain that text. The plain-language UX goal is already met.
Generated by Claude Code
There was a problem hiding this comment.
Correction: my prior reply was also wrong on this thread — same stale-state mistake. The label was indeed renamed to map.simulate() in commit 293e1d1, which the message describes as deliberate branding ("Rebrand the loading text from 'Arranging map…' to 'map.simulate()' to match the any.plot() function-notation style"). I won't apply your suggestion since this is an explicit brand choice by the maintainer, but flagging here so they can confirm/override if they want a more user-friendly label.
Generated by Claude Code
| const [prevGraphData, setPrevGraphData] = useState(graphData); | ||
| if (graphData !== prevGraphData) { | ||
| setPrevGraphData(graphData); | ||
| if (graphData.nodes.length > 0) { | ||
| setSettled(false); | ||
| setTickProgress(0); | ||
| tickCountRef.current = 0; | ||
| } |
Addresses the third Copilot review comment on #5665: the existing overlay tests only exercise the initial cooling phase, so a regression where a weights/threshold change stops bringing the computing overlay back would have slipped through. The new test: - Renders MapPage and waits for the canvas mount. - Asserts the progress bar is initially reachable (gate visible). - Calls onEngineStop() — settled flips true, progressbar parent gets aria-hidden=true, queryByRole filters it out. - Opens the weights panel and bumps the first slider. That changes weights state, graphData useMemo re-derives with a new identity, and the prevGraphData mismatch in render must reset settled and tickProgress so the gate re-arms. - Asserts the progress bar is reachable again with aria-valuenow=0. https://claude.ai/code/session_01XcRMey5z2WQNhZJQnX2Vtu
| let cx = 0, cy = 0, n = 0; | ||
| for (const node of nodes) { | ||
| if (node.x == null || node.y == null) continue; | ||
| cx += node.x; | ||
| cy += node.y; | ||
| n++; | ||
| } | ||
| if (n === 0) return; | ||
| cx /= n; | ||
| cy /= n; |
| const [prevGraphData, setPrevGraphData] = useState(graphData); | ||
| if (graphData !== prevGraphData) { | ||
| setPrevGraphData(graphData); | ||
| if (graphData.nodes.length > 0) { | ||
| setSettled(false); | ||
| setTickProgress(0); | ||
| tickCountRef.current = 0; | ||
| } |
Closes #5664.
Summary
Two UX improvements to the
/mapview, both motivated by feedback thatnon-technical users misread the initial state as "broken":
outlier-squash, only activates beyond the 95th-percentile distancefrom the cluster centroid. Inside the threshold the inner geometry is
bit-identical to before; outside, distances are compressed via a
sigmoid-like map
r' = R + (r - R) / (1 + (r - R)/k)so outliers stayvisibly separate but bounded (asymptote = R + k). This avoids the
alternative — stronger global gravity — which would also crush the
inner cluster.
"arranging" pill is replaced by a centered spinner + branded label
(
map.simulate(), matching theany.plot()function-notation style)is driven by
onEngineTick(throttled to ~10 Hz). The overlay fadeson
opacityinstead of unmounting, so subsequent re-derives (filter /weight slider) fade it back in.
Technical notes
/ collide / center) so its velocity correction is the last word per
tick. Multiplies by
alpha, so the correction tapers off naturallyover
cooldownTicks.OUTLIER_THRESHOLD_PERCENTILE = 0.95,OUTLIER_SQUASH_K = 120(tightened from 200 in293e1d1— ~40%tighter asymptote, pulls extreme outliers further toward the
threshold ring),
OUTLIER_SQUASH_STRENGTH = 0.18. Allconst; not user-toggled perthe issue's open question (always-on).
293e1d1):COOLDOWN_TICKSreduced 450 →300,
d3AlphaMinraised 0.001 → 0.01 so the engine stops wheremotion stops being perceptible, and
d3AlphaDecayis derived fromthose two so the simulation stops exactly when the progress bar
reaches 100%.
MUI's
LinearProgressand avoids an extra import; updates throttledto once every 6 ticks via a ref +
setTickProgress.floor((n - 1) * p)(numpy linearinterpolation) so small filtered subsets don't degenerate to "no
outliers" — fixed in
e87bf1cafter a Copilot review caught theoff-by-one with
floor(n * p).react-hooks/set-state-in-effectviolations and two
exhaustive-depswarnings inMapPage.tsx. Eachwas refactored to a render-time pattern (last-non-null ref for the
hover panel, "store previous prop" for the gate re-arm, JSX gating
for the pin marker, inline reset in the search input handler).
Test plan
yarn type-check— cleanyarn lint src/pages/MapPage.tsx— zero findings on this fileyarn test— full suite green; newoutlierSquashForceunittests cover empty/single-node graphs, inner-cluster preservation, the
off-by-one regression, bounded correction at extreme distances, and
alpha scaling. Two new integration tests cover
onRenderFramePreregistering the custom force and
onEngineTickadvancingaria-valuenowon the progress bar.yarn build— succeeds/map, confirm centered spinner + progress barappears immediately and disappears smoothly when layout settles
for the new cooling phase
visibly outside the main cluster but inside the viewport on first
framing
https://claude.ai/code/session_01XcRMey5z2WQNhZJQnX2Vtu