explorer: 'Samples in View' shows the 5000 fetch budget, not real count + URL round-trip doesn't reproduce view

[rdhyee]

Investigated 2026-05-11 starting from this URL:
https://isamples.org/explorer.html#v=1&lat=34.9957&lng=33.6798&alt=15212&heading=360.0&mode=point

User-visible symptoms:

1. The "Samples in View" stat box reads exactly 5000 — a suspiciously round number.
2. Pan and zoom from that starting state, copy the URL, paste into a different browser → the view that comes back is not the view that was captured.

Both symptoms have concrete root causes. Treating as a single issue because they were investigated together and overlap; happy to split if a maintainer prefers.

---

Part 1 — "Samples in View" is the fetch budget, not the real in-view count

Root cause

explorer.qmd:418

DEFAULT_POINT_BUDGET = 5000

explorer.qmd:1530-1538 — the point-mode viewport query:

SELECT pid, label, source, latitude, longitude, place_name, result_time
FROM read_parquet('${lite_url}')
WHERE latitude BETWEEN ${padded.south} AND ${padded.north}
  AND longitude BETWEEN ${padded.west} AND ${padded.east}
  ${sourceFilterSQL('source')}
  ${facetFilterSQL()}
LIMIT 5000

explorer.qmd:1557 — the UI counter:

updateStats('Samples', cachedData.length, cachedData.length, ..., 'Samples in View', 'Samples in View');

cachedData is the result of the LIMIT 5000 query. The counter therefore tops out at 5000 by construction. In dense regions it does not represent "samples in view" — it represents "samples we chose to load from a slightly-padded box around the view."

Ground-truth numbers for the Cyprus URL

Direct DuckDB query against https://data.isamples.org/isamples_202601_samples_map_lite.parquet, centered on the URL's lat=34.9957, lng=33.6798:

Viewport (degrees half-extent)	Actual samples
±0.10°	23,421
±0.20°	23,803
±0.50°	24,305
±1.00°	37,869

So when the UI says "5,000 Samples in View" at Cyprus, the truth is 23,000+ even within a viewport tighter than the explorer's 30%-padded fetch box. Counter is wrong by ~5x in this region.

Secondary smells in the same query path

• No ORDER BY before LIMIT 5000 → which 5000 rows are returned is undefined. DuckDB-on-parquet is probably stable file-order in practice, but it's not a contract.
• Label says "in View" but fetch uses a padded box (30% larger; explorer.qmd:1514-1522). Even if we set aside the cap, the count is loosely defined.
• The displayed count never shrinks to "true visible" as the user pans inside the cached padded box — renderSamplePoints plots all of cachedData, including rows outside the actual viewport.

What's at Cyprus (for context)

The 23K samples in the box are one dense cluster around lat 34.98, lng 33.71, all OPENCONTEXT. That's the Polis excavations project (Excavations at Polis had 52,762 OC records per the Open Context facet API). So the cap is hiding a single very-dense site, not a diffuse distribution.

Suggested fix directions (for discussion, not prescriptive)

1. Cheapest: change the label from "Samples in View" to "Samples Loaded (max N)" so the counter no longer lies. Wire the budget value into the label.
2. Show the real count separately: a fast SELECT count(*) against the same WHERE clause (no LIMIT) is cheap on the lite parquet via DuckDB-WASM range reads. Display two numbers: real count and rendered count.
3. Adaptive budget / aggregation: if real count > budget, fall back to a server-side aggregation or show "23,421 samples — too dense to render individually" with a UI affordance to drill in.
4. Add ORDER BY pid (or similar) so the LIMIT 5000 subset is at least deterministic across browsers and sessions.

---

Part 2 — URL state round-trip doesn't reproduce the view

What we know about the write path

Camera-change handler (explorer.qmd:1965-2030):

viewer.camera.changed.addEventListener(() => {
    if (timer) clearTimeout(timer);
    timer = setTimeout(async () => {
        // ... mode/resolution decisions ...
        if (!viewer._suppressHashWrite) {
            history.replaceState(null, '', buildHash(viewer));
        }
    }, 600);
});

buildHash (explorer.qmd:651-671) encodes only: v=1, lat, lng, alt, optional heading (only if abs(heading % 360) > 1), optional pitch (only if not nadir), optional mode=point, optional pid or h3. The query-string state (?search=, ?sources=, ?material=, etc.) is written by a separate writeQueryState() function (explorer.qmd:494-526) on filter changes, not by the camera handler.

Hypotheses (need empirical confirmation per the EXPLORER_STATE.md state-contract framing, #164)

1. Copy mid-debounce. 600ms debounce means a user who pans/zooms and immediately copies the URL gets stale state. Easy to confirm: pan, wait 2s, then check the address bar.
2. Heading normalization drops 360.0. buildHash only writes heading if abs(heading % 360) > 1 (line 661). The deep-link URL Raymond used has heading=360.0, but after one camera-handler tick that value is normalized to 0 and the param is dropped. If the user's view depends on heading != 0, the next URL write silently discards it.
3. Cold-cache point-mode latency dominates. Deep-link to mode=point triggers the res8 + samples_map_lite fetch path that takes 60–90s on a cold cache (the same path explorer: 60–90 s 'no dots' window on cold-cache deep-link to point mode (DuckDB-WASM 1.24.0 falls back to full HTTP read) #190 / PR explorer: surface 'Fetching sample index…' during cold-cache boot→point-mode wait (#190 fix 2) #191 worked around). In a fresh browser the view "looks different" might mean "samples haven't loaded yet."
4. 5000-cap non-determinism (Part 1). Even when both browsers finish loading, the displayed 5000-sample subset is undefined without ORDER BY. The two browsers might render different 5000 dots.
5. _suppressHashWrite could stay stuck. Hashchange handler sets it true (line 2127) and clears it after a 2000ms timeout (lines 2140-2145). If a user chains hashchanges (back/forward repeatedly) faster than 2000ms, the flag may stay set across writes. Edge case; less likely in Raymond's flow but worth ruling out.

Cleanest discriminating test

After pan/zoom and a 2-second pause:

• If the address bar contains the current camera state → problem is on the load side. Suspects: (3) cold-cache latency, (4) 5000-cap subset roulette, (5) stuck suppress flag.
• If the address bar still has stale state → problem is on the write side. Suspects: (1) longer-than-600ms thing keeping _suppressHashWrite true, or (2) heading-normalization dropping a param the user needed.

Relationship to other work

• Explorer state contract: URL/DOM/widget-state inventory + search-as-global-filter decision #164 Explorer state contract: this issue is exactly the kind of empirical fodder Explorer state contract: URL/DOM/widget-state inventory + search-as-global-filter decision #164's state inventory needs. Add a row for each URL-encoded field including "what triggers it" and "what swallows it" (e.g., heading-normalization-on-write is a swallow the inventory should capture).
• explorer: 60–90 s 'no dots' window on cold-cache deep-link to point mode (DuckDB-WASM 1.24.0 falls back to full HTTP read) #190 / PR explorer: surface 'Fetching sample index…' during cold-cache boot→point-mode wait (#190 fix 2) #191: confirms the cold-cache cost on point-mode deep-link is a known multi-second delay; hypothesis (3) is plausible.
• explorer: map position and size jump between views because surrounding elements shift #199: layout-stability work that's in-flight separately.

---

Repro (Cyprus, 2026-05-11)

URL: https://isamples.org/explorer.html#v=1&lat=34.9957&lng=33.6798&alt=15212&heading=360.0&mode=point
Observed: "Samples in View: 5,000"
Real count in tight ±0.1° box: 23,421 (one dense OPENCONTEXT cluster, probably Polis)

DuckDB reproduction (no auth needed):

import duckdb
con = duckdb.connect()
con.execute("INSTALL httpfs; LOAD httpfs;")
url = 'https://data.isamples.org/isamples_202601_samples_map_lite.parquet'
con.execute(f"""
    SELECT count(*) FROM read_parquet('{url}')
    WHERE latitude BETWEEN 34.8957 AND 35.0957
      AND longitude BETWEEN 33.5798 AND 33.7798
""").fetchone()
# (23421,)

[rdhyee]

Empirical investigation via Playwright (2026-05-11)

Ran a 6-iteration round-trip test against the live site. Harness: tests/playwright/url_roundtrip_investigation.js. Drives Context A through deterministic camera.flyTo sequences from the Cyprus deep-link, then opens each resulting URL in a fresh Context B and probes at 5s/15s/30s. Full log: /tmp/url_roundtrip_log.json.

Findings

#	Pan/zoom step	URL-write OK?	Round-trip OK?	Notes
1	no-op (same target)	n/a	✅	baseline
2	pan NE only	❌ URL stale	❌ B at old lat/lng	Δ0.019° / Δ0.020° = exactly the missed move
3	zoom in (alt 8000), heading 0	✅	❌ B stuck in cluster mode	URL has mode=point, B never enters point
4	heading 45° at zoom	✅	✅	B enters point mode by 30s
5	pan + zoom out + heading 90°	✅	✅	B enters point mode by 30s
6	heading 360° (= 0 after %360)	✅	❌ B stuck in cluster mode	same failure shape as iter 3

Two distinct bugs confirmed

Bug A: camera-handler URL write is not always applied.
Iter 2 moved the camera (snapshot confirms lat=35.0150, lng=33.7000, mode=cluster) but location.href was unchanged from the original deep-link. Pasting the URL elsewhere reproduces the OLD position. By iter 3 the URL had caught up.

Code path: explorer.qmd:1965-2030. The camera-changed handler awaits loadRes(...) / tryEnterPointModeIfNeeded() (lines 1981, 1987, 1997, 2006, 2011) before reaching the history.replaceState at line 2027. If those awaits are still pending — which on a cold or partially-warm cache is the common case during early panning — the URL write is deferred until they settle. The next camera move then writes the now-correct lat/lng but the intermediate state never appeared in the URL.

Effect: when a user pans and immediately copies the URL during cluster→point boot or any resolution change, the URL is stale by one or more steps.

This matches hypothesis #1 from the original report but is more specific than "600ms debounce" — the real culprit is the await chain inside the debounced callback, not the debounce duration.

Bug B: hashchange/boot does not restore mode=point when the heading URL param is absent.

Iter 3 URL: #v=1&lat=35.0150&lng=33.7000&alt=8000&mode=point (no heading; was 0 after %360, dropped by line 661 normalization).
Iter 6 URL: #v=1&lat=34.9800&lng=33.6500&alt=25000&mode=point (no heading; was 360 → 0 → dropped).

In both, Context B never reached point mode (probed at 5s/15s/30s, all mode=cluster).

By contrast, iters 4 and 5 (with heading=45.0 and heading=90.0 in the URL) reached point mode by 30s.

Hypothesis: hashchange handler at explorer.qmd:2130-2137:

viewer.camera.flyTo({
    destination: Cesium.Cartesian3.fromDegrees(state.lng, state.lat, state.alt || 20000000),
    orientation: {
        heading: Cesium.Math.toRadians(state.heading),
        pitch: Cesium.Math.toRadians(state.pitch)
    },
    duration: 1.5,
});

When the URL omits heading, state.heading is undefined, Cesium.Math.toRadians(undefined) returns NaN. Cesium's camera.flyTo may silently fail or settle into an invalid orientation, and the post-flight setTimeout at line 2140 reads the hash again to decide on enterPointMode(false) — the NaN orientation could interfere with what mode the engine reports during this window.

Need to confirm the exact mechanism, but the empirical signal is clean: heading param present ⇒ round-trip works; heading param absent ⇒ mode hydration silently fails.

Why "sometimes works, sometimes doesn't" (Raymond's symptom)

The two bugs explain the intermittency cleanly:

• If a pan/zoom triggers a loadRes await but the user copies the URL before it settles → bug A → URL stale.
• If a pan/zoom results in heading ≈ 0 or 360 → bug B → URL captures-but-fails-to-restore.
• If both go right (heading != 0 AND no pending awaits) → URL round-trips correctly.

Suggested fix shape (sketch, for discussion)

For bug A: write the URL hash before awaiting loadRes/tryEnterPointModeIfNeeded. The camera state is known synchronously from viewer.camera; we don't need to wait for tile/cluster loads to write where the camera is. Move the if (!viewer._suppressHashWrite) history.replaceState(...) block to before the mode-transition awaits, not after.

For bug B: either always write heading (even 0), or in the hashchange handler default to state.heading ?? 0 before toRadians. Probably do both — write-side and load-side defensively.

Both fixes are small, isolated changes in explorer.qmd. Each suitable for a focused PR with a Playwright regression assertion based on this harness.

Other observations (lower priority)

• Context A's initial snapshot reports mode=cluster even though waitReadyAndPointMode returned true moments earlier — the OJS-cell mode variable and viewer._globeState.mode can be transiently out of sync during boot. Worth a deeper look but not blocking these two bugs.
• The "Samples in View = 5000" issue (Part 1 of this issue) is independent — the harness saw cachedDataLen=null throughout because OJS exposes cachedData differently than the viewer cell does; my snapshot didn't capture it but the in-DOM #sSamples text confirmed it stays at the corpus total in cluster mode.