Skip to content

explorer: rewrite on progressive_globe foundation for speed + add results table#153

Merged
rdhyee merged 4 commits intoisamplesorg:mainfrom
rdhyee:explorer/progressive-speed
May 1, 2026
Merged

explorer: rewrite on progressive_globe foundation for speed + add results table#153
rdhyee merged 4 commits intoisamplesorg:mainfrom
rdhyee:explorer/progressive-speed

Conversation

@rdhyee
Copy link
Copy Markdown
Contributor

@rdhyee rdhyee commented May 1, 2026

Summary

Rewrites tutorials/isamples_explorer.qmd on the progressive_globe.qmd foundation so the search/filter explorer gets the same near-instant globe rendering as the Progressive Globe page. Also adds a Specimen Type facet, a results table below the globe, and ?sources= URL-param sync.

Live preview: https://rdhyee.github.io/isamplesorg.github.io/tutorials/isamples_explorer.html?sources=OPENCONTEXT

Speed wins (lifted from progressive_globe)

  • Progressive H3 cluster phases — res4 (580 KB) paints the globe in ~1 s, res6 / res8 swap in by camera altitude with hysteresis (no flicker on zoom boundaries).
  • Point mode is viewport-bounded — only at < 120 km altitude does the explorer switch to individual sample primitives, capped at 5K points and queried with a 30 %-padded bounding box that's cached for smooth panning.
  • Globe is decoupled from OJS reactivity — Cesium viewer is mounted eagerly, lives outside the reactive graph, and updates imperatively. Filter changes call `loadRes()` / `loadViewportSamples()` instead of invalidating a `sampleData` cell that rebuilds the whole point set.
  • Two `PointPrimitiveCollection`s (`h3Points`, `samplePoints`) toggle via `.show`, never destroy/re-create.
  • Source filter pushes down to H3 cluster files — the H3 tier files carry `dominant_source` per cell, so source filtering works at every zoom level. (Material / sampled-feature / specimen filters only apply at point zoom and the results table; that's the documented limitation, surfaced via the existing `#facetNote` element.)

Explorer-specific additions on top

  • Specimen Type (`object_type`) filter populated from the facet summaries, alongside Material and Sampled Feature. `facets_url` is the v2 file (which carries the `object_type` URI string column).
  • `?sources=A,B` URL query param is read on initial load to pre-check the source legend before phase 1 runs, and is kept in sync on every source toggle. Camera state continues to live in the URL hash (existing progressive_globe pattern).
  • Results table below the globe (`### Results`) — shows the first 200 samples matching the current filters via `lite_url` (joined to `facets_url` when facet filters are active). Refreshes on every filter change. Independent of camera position.
  • Frontmatter title / intro / "See Also" framing match the explorer's role; preloads `samples_map_lite.parquet` in addition to the H3 res4 + facet summaries.

What I dropped

  • Old reactive `sampleData` / `whereClause` / cross-filter cache machinery.
  • `v1` / `v2` `explorerVersion` toggle and the `ORDER BY RANDOM()` query path on the wide parquet.
  • `maxSamples` slider (progressive_globe hard-codes `POINT_BUDGET = 5000`; we can re-add later if there's demand).
  • SKOS `prefLabel` lookup against `vocab_labels.parquet` — URI-tail fallback is what progressive_globe uses and the labels read fine.
  • The "Setup" / "Database & Queries" / "Sample Card" / "Debug Info" section structure (these existed because the page was code-tutorial-styled; the rewrite is app-styled like progressive_globe).

Test plan

  • `quarto render tutorials/isamples_explorer.qmd` succeeds
  • Playwright smoke test: 0 JS exceptions, 0 console errors, 0 network failures
  • Deployed to rdhyee fork preview — visual confirmation that globe paints res4 clusters, side panel populates, and results table fills with first 200 samples matching `?sources=OPENCONTEXT`
  • Reviewer eyeballs preview URL above and confirms perceived speed matches `progressive_globe.html`
  • Reviewer confirms the results table is the right shape (columns Source / Label / Place / Lat / Lon, 200-row cap) — easy to adjust

Compatibility

  • Bookmarkable URLs of the form `?sources=OPENCONTEXT` continue to work (still pre-check the source filter on load).
  • Other URL params (`q`, `material`, `context`, `object_type`, `view`) used by the prior explorer are no longer honored — they were not documented externally as far as I know. Worth confirming.
  • Internal links in the site (`navbar`, `index.qmd`, `how-to-use.qmd`, `about.qmd`) point to `progressive_globe.html` for "Interactive Explorer"; only `_quarto.yml`'s "How to Use → Search Explorer" sidebar entry and `progressive_globe.qmd`'s "See Also" link target this page. Both still resolve to the same URL after this change.

🤖 Generated with Claude Code

rdhyee and others added 4 commits April 30, 2026 17:09
@rdhyee @claude
explorer: rewrite on progressive_globe foundation for speed + add table
58c4d15 
Replaces the OJS-reactive explorer (sampleData → renderPoints → Cesium
re-add chain) with the progressive_globe architecture, then layers the
explorer's facet UI and a results table on top.

Speed wins from progressive_globe:
- Globe loads H3 res4 (580 KB) in ~1 s, then swaps to res6 / res8 as
  the camera zooms; only at <120 km does it switch to individual point
  primitives, viewport-bounded with 30% padding cache.
- Cesium viewer is mounted eagerly and lives outside the OJS reactive
  graph — filter changes call loadRes() / loadViewportSamples() rather
  than invalidating sampleData and rebuilding the whole point set.
- Two PointPrimitiveCollections (h3Points, samplePoints) toggled via
  .show, never destroyed/re-created.
- Source filter applied at H3 level (works on the cluster files);
  material / sampled-feature / specimen filters apply at point zoom
  and to the results table.

Explorer-specific additions on top of progressive_globe:
- Specimen Type (object_type) filter alongside Material and Sampled
  Feature, populated from facet_summaries; switches facets_url to
  the v2 file (which carries object_type).
- ?sources=A,B query string is read on load to pre-check the source
  legend before phase 1 runs, and is kept in sync as the filter
  changes (camera state continues to live in the URL hash).
- "### Results" table below the globe shows the first 200 samples
  matching the current filters via lite_url (joined to facets_url
  when facet filters are active). Refreshes on every filter change.
- Frontmatter title, intro, and "See Also" framing match the
  explorer's role; preloads samples_map_lite.parquet.

Removes the old reactive sampleData / whereClause / cross-filter cache
machinery, the v1/v2 explorerVersion toggle, the maxSamples slider
(progressive_globe hard-codes POINT_BUDGET=5000), and the SKOS
prefLabel lookup (URI-tail fallback is good enough for facet labels).

Smoke-tested locally: 0 JS exceptions, 0 console errors, 0 network
failures; globe paints res4 clusters and the results table fills in
from lite_url.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@rdhyee @claude
explorer: retain prior filter UX (counts, prefLabels, slider, clear)
598282c 
Brings the previous explorer's filter UX back on top of the
progressive_globe-based rewrite (isamplesorg#153):

- Source legend now shows per-source counts loaded from facet_summaries
  (matches the (12,345)-style badges on the prior live page).
- Material / Sampled Feature / Specimen Type checkboxes use SKOS
  prefLabels via vocab_labels.parquet (~60 KB lookup) with URI-tail
  fallback. Full URI is preserved as the checkbox value so the SQL
  filter matches sample_facets_v2's URI-string columns exactly.
- Filter semantics flip back to the prior pattern: empty = no filter
  (show everything); selecting items = include only those. Reduces
  side-panel noise at startup (was rendering hundreds of pre-checked
  rows).
- Max Samples slider (1K-100K, default 25K) drives both the globe's
  point-mode budget and the results-table query LIMIT. Live label
  updates while dragging; query refreshes on release (debounced 300 ms).
- "Clear Filters" button next to Share resets to a clean state by
  reloading to the bare pathname (matches the prior <a href="?">).
- Results table is now scrollable (max-height 600 px) and capped at
  200 rendered rows so a 100K-LIMIT query doesn't blow up DOM.
  Meta line reports `first N of M+` when the LIMIT is hit.

Smoke test: 0 JS exceptions, 0 console errors, 0 network failures.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@rdhyee @claude
explorer: address Codex review — search composes facets; cluster docs…
a9407e3 
… honest

P2 fixes from Codex review of isamplesorg#153:

1. doSearch() now composes facet filters (Material / Sampled Feature /
   Specimen Type) via JOIN sample_facets_v2 on pid, mirroring the
   pattern used by loadViewportSamples and refreshResultsTable. Was
   only honoring text + source filter, so search could surface and
   fly to samples outside the active facet selection — inconsistent
   with the rest of the explorer.

2. "How It Works" prose no longer claims source filtering "applies at
   every zoom level." Now states explicitly that at cluster zoom the
   H3 tier files carry one row per (h3_cell, dominant_source), so
   source filtering keeps cells whose *dominant* source matches; mixed
   cells where the unchecked source isn't dominant are still drawn
   (and contribute their full sample_count to the legend). Per-source
   filtering kicks in fully at sample zoom and in the table/search.
   Per-source-per-cell H3 builds would fix this properly but require
   a different parquet build — out of scope.

3. Updated the data-tier table to reflect Max Samples slider (default
   25K, max 100K) instead of the stale 5K hardcoded value.

Smoke test still PASS (0 exceptions, 0 errors, 0 network fails).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@rdhyee @claude
explorer: correct OpenContext example in cluster-filter caveat
09f33a6 
The cluster H3 filter is `WHERE dominant_source IN (checked)`, so
"OpenContext present but not dominant" does not keep a cell visible
when OpenContext is unchecked — the cell's dominant source is what
gets matched. Reworked the prose to spell out both consequences:
(1) cell whose dominant is X disappears entirely when X unchecked,
even if it contains samples from still-checked sources; (2) cell
whose dominant is X stays with full sample_count when X checked,
even if some samples in it are from unchecked sources. Codex P3.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@rdhyee rdhyee merged commit bd0a094 into isamplesorg:main May 1, 2026
1 check passed
This was referenced May 1, 2026
Closed
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@rdhyee