explorer: 60–90 s 'no dots' window on cold-cache deep-link to point mode (DuckDB-WASM 1.24.0 falls back to full HTTP read)

[rdhyee]

Symptom

Deep-link a viewer URL whose altitude lands inside point mode (alt < 120 km) and the viewport stays empty for ~60–90 seconds on first visit before any dots appear. At wider altitudes the page looks fine because the global res4 cluster set (38K dots) is already painted from Phase 1.

Repros (live deploy, current HEAD):

• Wider, works fast — lots of dots: https://isamples.org/explorer.html#v=1&lat=34.9515&lng=33.2166&alt=136823&h3=882da6b2e1fffff
• Zoomed in, empty for ~85 s — no dots: https://isamples.org/explorer.html#v=1&lat=34.9954&lng=33.7052&alt=62054&heading=360.0&mode=point&h3=882da6b2e1fffff

Reported by Raymond Yee, debugged via Playwright against prod 2026-05-09 / 2026-05-10.

Timing (live console log, alt=62054 URL, cold cache)

12:52:12  Phase 1: 38406 clusters in 192ms              (res4 global, 580 KB)
12:52:13  WARN  falling back to full HTTP read for: …samples_map_lite.parquet  (60 MB)
12:52:13  WARN  falling back to full HTTP read for: …isamples_202601_h3_summary_res8.parquet
12:53:37  Res8: 175653 clusters in 458ms                (≈85 s after boot)
12:53:37  Entered point mode
12:53:37  Point mode: 47 samples in 22ms                (dots finally appear)

The state machine is doing the right thing — zoomWatcher honors deep-link altitude, fires enterPointMode(), calls loadViewportSamples(). The wait is all in the network.

Root cause — DuckDB-WASM 1.24.0 client behavior, not the CDN

Initial hypothesis was that data.isamples.org Range support was broken. Live captures (curl + Playwright) show the server is fine:

• GET with Range: bytes=0-1023 → HTTP/2 206, Content-Range: bytes 0-1023/62631623 ✔
• OPTIONS preflight returns 204 with Access-Control-Allow-Headers: Range and the right Access-Control-Expose-Headers ✔
• HEAD returns 200 with full Content-Length + Accept-Ranges: bytes. (RFC 7233 §3.1 lets a server ignore Range on HEAD; this is correct.)

The actual sequence emitted by the DuckDB-WASM worker (per Codex's Playwright capture):

HEAD samples_map_lite.parquet            Range: bytes=0-
RESP 200, content-length=62631623, accept-ranges=bytes
console: falling back to full HTTP read
GET  samples_map_lite.parquet            (no Range header)
RESP 200, content-length=62631623        ← entire 60 MB pulled

So DuckDB-WASM 1.24.0 treats the HEAD-with-Range 200 (no Content-Range) as failure, logs falling back to full HTTP read, and re-fetches the file with no Range header — pulling all 60 MB on cold cache. The file is cache-control: public, max-age=31536000, immutable, so reload visits hit disk cache and look instant; cold visits eat the full 60 MB.

The pinned version is in Quarto's bundled OJS runtime, not the project:

• docs/site_libs/quarto-ojs/quarto-ojs-runtime.js:228 — const duckdb = dependency("@duckdb/duckdb-wasm", "1.24.0", "+esm");

Why the wider URL looks fine

At alt=136,823, zoomWatcher stays in cluster mode and only needs the 2.5 MB res8 file. The same wasteful "fallback then full read" still happens, but 2.5 MB completes in 1–2 s and the 175K res8 clusters paint immediately. The zoomed-in URL trips the much larger samples_map_lite.parquet fetch, which is why the symptom is altitude-dependent.

Recommended fixes (ordered)

1. Move off Quarto's bundled DuckDB-WASM 1.24.0. Either:

   • bump the Quarto bundle if a newer Quarto ships a fixed runtime, or
   • bypass DuckDBClient.of() and instantiate a newer DuckDB-WASM directly from CDN inside explorer.qmd's OJS cells, so we control the version regardless of Quarto. Either path requires verifying the wasm worker no longer logs falling back to full HTTP read on the same files. Worth a quick spike against @duckdb/duckdb-wasm@1.28.x (or current latest) before committing.

2. Surface the loading state during the boot→point-mode gap. Today updatePhaseMsg('Loading individual samples...', 'loading') only fires from inside loadViewportSamples (explorer.qmd:1451), but the camera handler already awaits loadRes(8, h3_res8_url) first (explorer.qmd:1838). The user gets no signal during the slow initial load. A "Fetching sample index…" message earlier in the chain converts "page is broken" into "page is busy" — and is worth shipping independently of fix (1), since 60 MB on a slow connection will always be a multi-second wait the user needs to see.

3. Independent of (1)/(2): re-verify halo/terrain rendering once samples load. After the slow load completed in my repro, "5,000 samples" rendered but were hard to spot at the test viewport size — possibly intersects explorer: halo crescents / disappearing dots over hilly terrain (root cause not yet identified) #185.

Diagnostic command (corrected)

# Use GET, not HEAD — many servers (correctly per RFC 7233) ignore Range on HEAD,
# so HEAD is not a reliable pass/fail check.
curl -s -o /dev/null -D - -H 'Range: bytes=0-1023' \
     https://data.isamples.org/isamples_202601_samples_map_lite.parquet | head -10
# Expect: HTTP/2 206 + Accept-Ranges: bytes + Content-Range: bytes 0-1023/<size>
# Confirmed 206 as of 2026-05-10 — server is not the bug.

Refs

• Distinct from explorer: halo crescents / disappearing dots over hilly terrain (root cause not yet identified) #185 (halo crescents over hilly terrain — different root cause, different repro).
• PR explorer: persist selected cluster identity in URL via &h3=<cell> (Phase 1) #186 (h3 URL state) and explorer: extract freshSelectionToken() primitive (closes #187 step 1) #188 (freshSelectionToken) are correctness fixes around this same boot path; this issue is about the latency of the same path.
• Original issue framing (server-side Range hypothesis) is preserved in the comment thread for history.

[rdhyee]

Quick follow-up from curl probes against data.isamples.org:

• curl -I -H "Range: bytes=0-1023" https://data.isamples.org/isamples_202601_samples_map_lite.parquet returns HTTP/2 200 with the full content-length: 62631623, even though accept-ranges: bytes is present. So the HEAD version of the suggested probe is not a reliable pass/fail test.
• An actual ranged GET succeeds: curl -L -r 0-1023 -D ... -o ... https://data.isamples.org/isamples_202601_samples_map_lite.parquet returns HTTP/2 206, content-length: 1024, and content-range: bytes 0-1023/62631623; the downloaded body is 1024 bytes.
• A browser-style CORS preflight also looks OK: OPTIONS with Origin: https://isamples.org, Access-Control-Request-Method: GET, and Access-Control-Request-Headers: range returns 204 and includes access-control-allow-headers: Range plus access-control-expose-headers: Content-Length, Content-Range, Accept-Ranges, ETag.

So as of 2026-05-10 02:24 UTC, the object/CDN is serving plain byte-range GETs correctly. If DuckDB-WASM still logs falling back to full HTTP read, the next thing to inspect is the exact browser request DuckDB emits and receives: HEAD vs GET behavior, request header casing, cache mode, service worker/cache interaction, or whether Cloudflare served a stale/non-range cached variant during the original repro.

I would also update the diagnostic command in the issue from curl -I -H "Range: ..." to a GET-based check, since HEAD can return 200 here while GET correctly returns 206.

[rdhyee]

Following up on @codex's curl probes (thanks). Captured the actual HEAD response that the duckdb-wasm worker receives in a fresh browser session; it confirms the server side is fine and shifts the search to the wasm decision logic.

What the live HEAD response looks like (not just curl)

Captured via Playwright on https://data.isamples.org/isamples_202601_samples_map_lite.parquet:

HTTP/2 200
content-length: 62631623
accept-ranges: bytes
access-control-allow-origin: *
access-control-allow-headers: Range
access-control-expose-headers: Content-Length, Content-Range, Accept-Ranges, ETag
cache-control: public, max-age=31536000, immutable
etag: "6b096425a0bc6635934831ab57b3276d-2"
content-type: application/vnd.apache.parquet

So:

• HEAD = 200 (correct — RFC 7233 §3.1: a server MAY ignore Range on a HEAD; many do).
• accept-ranges: bytes is advertised on HEAD ✔
• All required CORS exposure headers are present ✔

Curl confirms (matching @codex's earlier result, re-run from a different network):

$ curl -s -o /dev/null -D - -H "Range: bytes=0-1023" -H "Origin: https://isamples.org" \
       https://data.isamples.org/isamples_202601_samples_map_lite.parquet | head
HTTP/2 206
content-length: 1024
content-range: bytes 0-1023/62631623
accept-ranges: bytes
...

So why does duckdb-wasm still log falling back to full HTTP read?

Two things to disentangle:

1. The warning is per-file and fires regardless of the actual subsequent fetch behavior. I see it for every parquet on data.isamples.org — including the 580 KB res4 file that loads in 192 ms. So the warning by itself isn't a smoking gun for the 60–90 s wait — it's a worker-side log that the wasm runtime emitted at file-handle-open time.

2. The 60–90 s wait reproduces on cold cache, not warm. The file is served cache-control: public, max-age=31536000, immutable. First-ever visit pays the cost; reload visits are instant. My first repro took 85 s; the most recent reload of the same URL completed in <15 s. That fits a one-time 60 MB transfer that's then served from disk cache, regardless of whether ranges were used.

So the user-visible problem is "cold-cache visits to a deep-link in point mode are slow" — and we still don't know whether duckdb-wasm 1.24.0 is downloading the whole file on cold cache or doing many sequential range reads that happen to add up to ~60 MB / ~85 s.

What would resolve this

• Capture worker network in devtools during a cold-cache repro (DevTools → Network → enable "Other origins" + service workers; or chrome://net-export). One sample of the actual GETs the worker emits for samples_map_lite.parquet decides whether ranges are honored end-to-end inside the wasm worker.
• Try upgrading duckdb-wasm. The explorer pins 1.24.0 via the Observable CDN URL; 1.28.x has reworked range / HEAD handling. Worth a one-line bump to see if the warning + behavior changes.
• Independent of either, the user-experience fix (explorer: 60–90 s 'no dots' window on cold-cache deep-link to point mode (DuckDB-WASM 1.24.0 falls back to full HTTP read) #190 fix 2) — surface "Fetching sample index…" during the boot→point-mode gap — should ship anyway, since 60 MB on a slow connection will always be a multi-second wait the user needs to see.

I'll update the issue body's diagnostic command to use a GET with -r 0-1023 rather than HEAD -H "Range", per @codex's note that HEAD is not a reliable pass/fail check.

[rdhyee]

Restructured the issue body per @codex's re-review. Summary of what changed and why:

• Title now leads with DuckDB-WASM 1.24.0 falls back to full HTTP read instead of Range-request fallback for samples_map_lite.parquet. The original framing implied the CDN was at fault; the actual evidence (Codex's request capture: HEAD 200 → console: falling back → GET no-Range 200) points squarely at the wasm runtime.
• Root cause rewritten. Server-side ranges work end-to-end (GET with Range returns 206; preflight returns 204; cold curl reproduces this). The falling back to full HTTP read warning is duckdb-wasm's misinterpretation of the HEAD-with-Range 200 response, after which it re-fetches the file with no Range header at all. The pinned version lives in Quarto's bundled OJS runtime (docs/site_libs/quarto-ojs/quarto-ojs-runtime.js:228), not the project itself.
• Recommended fixes reordered. Primary fix is now "move off Quarto's bundled DuckDB-WASM 1.24.0" (either bump Quarto or bypass DuckDBClient.of() with a direct CDN import). The UX loading-message fix is preserved as fix (2), with the precise file:line refs (explorer.qmd:1838 awaits loadRes(8) before enterPointMode(); Loading individual samples... only fires inside loadViewportSamples at explorer.qmd:1451).
• infrastructure label removed. This is a client-runtime + UX bug, not a data.isamples.org ops issue.
• The two earlier comments (initial diagnosis, server-side investigation) are intentionally kept on the thread so the path from "blame the CDN" → "blame the wasm runtime" stays auditable.

Cheapest next experiment: spike @duckdb/duckdb-wasm@1.28.x (or current latest) instantiated directly in explorer.qmd, against the same alt=62054 URL, and watch the worker-side network for whether falling back to full HTTP read still fires.

[rdhyee]

Spike result: version bump kills the warning, but UBIGINT representation broke

Ran the cheap experiment locally on a spike/duckdb-wasm-1.28 worktree: replaced the db cell so ?duckdb=<version> query-param bypasses Quarto's DuckDBClient.of() and instantiates @duckdb/duckdb-wasm directly from jsdelivr. Default path (no override) is unchanged.

Cold-cache run against the same alt=62054 deep-link URL:

Run	duckdb-wasm	falling back to full HTTP read count	Page renders?
Baseline (no ?duckdb=)	1.24.0 (Quarto bundle)	5 (one per parquet)	✅ — but stuck at H3 Res4
?duckdb=latest	1.33.1-dev45.0 (jsdelivr)	0	❌ — runtime error (see below)

So fix path #1 in the issue body is directionally correct: a newer wasm runtime stops misinterpreting the HEAD-with-Range 200 and the warning goes away end-to-end. Network log on the ?duckdb=latest run shows duckdb-wasm sending a real HEAD then a GET for h3_summary_res4.parquet (no spurious full-file refetch).

But it's not free. The ?duckdb=latest run errors at quarto-ojs-runtime.js:5430:

RuntimeError: Cannot mix BigInt and other types, use explicit conversions

…thrown from inside the OJS reactivity layer, not our code. Newer duckdb-wasm changed the JS representation of UBIGINT (and likely other 64-bit ints) — it now hands back BigInt rather than Number. OJS does arithmetic on the returned values and bombs. This blew up phase1, the facet-summary cell, zoomWatcher, and tableView — basically every cell that consumes a query result.

Practical implications for the issue:

• The Range-fallback warning is a wasm-runtime bug, not a CDN bug — that part of the diagnosis is now load-bearing.
• A "just bump the version" fix will not work as a one-liner; it requires adding explicit Number()/BigInt-handling at every consumer of UBIGINT columns (h3_cell_dec, sample_count, source_count, etc.) and revisiting how OJS reactivity touches those values.
• The cheaper interim ship is still fix (2) — surface the loading state during the boot→point-mode gap. That makes the cold-cache wait honest without depending on the version bump.

Branch spike/duckdb-wasm-1.28 has the bypass-and-A/B switch if anyone wants to repro or build on it. Not opening a draft PR yet — wanted the spike result on the thread first.