From 58c4d156c3e804988931ae4054262df7b4c0f4ad Mon Sep 17 00:00:00 2001 From: Raymond Yee Date: Thu, 30 Apr 2026 17:09:40 -0700 Subject: [PATCH 1/4] explorer: rewrite on progressive_globe foundation for speed + add table MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replaces the OJS-reactive explorer (sampleData → renderPoints → Cesium re-add chain) with the progressive_globe architecture, then layers the explorer's facet UI and a results table on top. Speed wins from progressive_globe: - Globe loads H3 res4 (580 KB) in ~1 s, then swaps to res6 / res8 as the camera zooms; only at <120 km does it switch to individual point primitives, viewport-bounded with 30% padding cache. - Cesium viewer is mounted eagerly and lives outside the OJS reactive graph — filter changes call loadRes() / loadViewportSamples() rather than invalidating sampleData and rebuilding the whole point set. - Two PointPrimitiveCollections (h3Points, samplePoints) toggled via .show, never destroyed/re-created. - Source filter applied at H3 level (works on the cluster files); material / sampled-feature / specimen filters apply at point zoom and to the results table. Explorer-specific additions on top of progressive_globe: - Specimen Type (object_type) filter alongside Material and Sampled Feature, populated from facet_summaries; switches facets_url to the v2 file (which carries object_type). - ?sources=A,B query string is read on load to pre-check the source legend before phase 1 runs, and is kept in sync as the filter changes (camera state continues to live in the URL hash). - "### Results" table below the globe shows the first 200 samples matching the current filters via lite_url (joined to facets_url when facet filters are active). Refreshes on every filter change. - Frontmatter title, intro, and "See Also" framing match the explorer's role; preloads samples_map_lite.parquet. Removes the old reactive sampleData / whereClause / cross-filter cache machinery, the v1/v2 explorerVersion toggle, the maxSamples slider (progressive_globe hard-codes POINT_BUDGET=5000), and the SKOS prefLabel lookup (URI-tail fallback is good enough for facet labels). Smoke-tested locally: 0 JS exceptions, 0 console errors, 0 network failures; globe paints res4 clusters and the results table fills in from lite_url. Co-Authored-By: Claude Opus 4.7 (1M context) --- tutorials/isamples_explorer.qmd | 2299 +++++++++++++++++++------------ 1 file changed, 1393 insertions(+), 906 deletions(-) diff --git a/tutorials/isamples_explorer.qmd b/tutorials/isamples_explorer.qmd index 18820f8..736560f 100644 --- a/tutorials/isamples_explorer.qmd +++ b/tutorials/isamples_explorer.qmd @@ -1,1027 +1,1514 @@ --- title: "iSamples Interactive Explorer" -categories: [parquet, spatial, interactive] +subtitle: "Search, filter, and explore 6.7 million material samples" +categories: [parquet, spatial, h3, performance, isamples] +sidebar: false +# No TOC: this page is an app, not an article. The right-hand TOC sidebar +# (#quarto-margin-sidebar) was overlapping .side-panel and silently +# intercepting clicks on the Source filter checkboxes — see issue #127. +toc: false format: html: - echo: false - toc: true - toc-depth: 3 include-in-header: text: | + - - - --- -Search and explore **6.7 million physical samples** from scientific collections worldwide. + + + + +::: {.callout-note collapse="true"} +## How It Works -::: {.callout-note} -### Serverless Architecture -This app uses a **two-tier loading strategy**: a 2KB pre-computed summary loads instantly for facet counts, while the full ~280 MB Parquet file is queried on demand. **Cross-filtering** keeps counts accurate — selecting a source updates material/context/specimen counts to reflect only that source's samples. All powered by DuckDB-WASM in your browser — no server required! +1. **Instant** (<1s): Pre-aggregated H3 res4 summary (580 KB) → 38K colored circles +2. **Zoom in**: Automatically switches to res6 (112K) then res8 (176K) clusters +3. **Zoom deeper** (<120 km): Individual sample points from 60 MB lite parquet +4. **Click**: Cluster info or individual sample card with full metadata +5. **Search**: Find samples by name — results fly to the location on the globe + +Circle size = log(sample count). Color = dominant data source. ::: + +
+ + +
+
+ +
+
+
+
+
+
Loading...Resolution
+
0Clusters Loaded
+
0Samples Loaded
+
-Load Time
+
+
+
+ + + + +
+
+
+
+Material ▾ +
+ +
+
+
+Sampled Feature ▾ +
+ +
+
+
+Specimen Type ▾ +
+ +
+ +
+ + +
+
+Loading H3 global overview... +
+
+
+
Click a cluster or sample on the globe
+
+
+
+
+ +### Results {#results} + +
+
Loading samples matching your filters...
+
+
+ ```{ojs} -// Imports - use dynamic import to avoid CORS issues -duckdbModule = import("https://cdn.jsdelivr.net/npm/@duckdb/duckdb-wasm@1.28.0/+esm") +//| output: false +Cesium.Ion.defaultAccessToken = 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJqdGkiOiIwNzk3NjkyMy1iNGI1LTRkN2UtODRiMy04OTYwYWE0N2M3ZTkiLCJpZCI6Njk1MTcsImlhdCI6MTYzMzU0MTQ3N30.e70dpNzOCDRLDGxRguQCC-tRzGzA-23Xgno5lNgCeB4'; ``` ```{ojs} -// Version gate. Append ?v=2 to the URL to opt into the lite-backed -// rewrite (samples_map_lite.parquet instead of wide.parquet, lazy -// description fetch on click, no ORDER BY RANDOM(), lazy Cesium mount). -explorerVersion = new URLSearchParams(location.search).get('v') === '2' ? 'v2' : 'v1' - -// Data source configuration. -// wide_url uses the /current/ alias so we pick up the latest enriched build -// (with OpenContext thumbnails); the alias 302-redirects to the dated file. -wide_url = "https://data.isamples.org/current/wide.parquet" -lite_url = "https://data.isamples.org/isamples_202601_samples_map_lite.parquet" -parquet_url = explorerVersion === 'v2' ? lite_url : wide_url - -// Pre-computed facet summaries (2KB - loads instantly) -facet_summaries_url = "https://data.isamples.org/isamples_202601_facet_summaries.parquet" - -// Pre-computed cross-filter cache (6KB - instant single-filter lookups) -cross_filter_url = "https://data.isamples.org/isamples_202601_facet_cross_filter.parquet" - -// Facets file for on-the-fly multi-filter queries (63MB - URI strings, not BIGINT FKs) -sample_facets_url = "https://data.isamples.org/isamples_202601_sample_facets_v2.parquet" - -// Vocabulary labels (~60KB) — maps SKOS concept URIs to human-readable prefLabels. -// Built by scripts/build_vocab_labels.py from isamplesorg/vocabularies TTLs. See #148. -vocab_labels_url = "https://data.isamples.org/vocab_labels.parquet" - -// Source color scheme — imported from canonical palette (issue #113) -// Use a path-relative URL so this works under both the production custom -// domain (isamples.org/tutorials/...) and a project-pages fork preview -// (rdhyee.github.io/isamplesorg.github.io/tutorials/...). +//| echo: false +//| output: false + +// === Constants === +R2_BASE = "https://data.isamples.org" +h3_res4_url = `${R2_BASE}/isamples_202601_h3_summary_res4.parquet` +h3_res6_url = `${R2_BASE}/isamples_202601_h3_summary_res6.parquet` +h3_res8_url = `${R2_BASE}/isamples_202601_h3_summary_res8.parquet` +lite_url = `${R2_BASE}/isamples_202601_samples_map_lite.parquet` +// Stable alias that 302-redirects to the current enriched wide parquet +// (isamples_YYYYMM_wide.parquet). Gets OpenContext thumbnails populated. +wide_url = `${R2_BASE}/current/wide.parquet` +// v2 carries object_type alongside material and context (URI-string columns). +facets_url = `${R2_BASE}/isamples_202601_sample_facets_v2.parquet` +facet_summaries_url = `${R2_BASE}/isamples_202601_facet_summaries.parquet` + +// Canonical palette — see issue #113. Path-relative so this works under +// both isamples.org (custom domain at root) and project-pages fork +// previews (rdhyee.github.io/isamplesorg.github.io/...). _palette = await import(new URL('../assets/js/source-palette.js', document.baseURI).href) SOURCE_COLORS = _palette.SOURCE_COLORS +SOURCE_NAMES = _palette.SOURCE_NAMES + +// === Source URL: resolve pid to original repository === +function sourceUrl(pid) { + if (!pid) return null; + // All sources resolve via n2t.net: + // ARK pids (OpenContext, GEOME, Smithsonian) → n2t.net/ark:/... + // IGSN pids (SESAR) → n2t.net/IGSN:... + return `https://n2t.net/${pid}`; +} -// Cesium colors -function getCesiumColor(source) { - const hex = SOURCE_COLORS[source]; - return hex ? Cesium.Color.fromCssColorString(hex) : Cesium.Color.GRAY; +// === Source Filter: get active sources and build SQL clause === +function getActiveSources() { + const checks = document.querySelectorAll('#sourceFilter input[type="checkbox"]'); + return Array.from(checks).filter(c => c.checked).map(c => c.value); } -``` -```{ojs} -// Parse URL params for bookmarkable searches -initialParams = { - const params = new URLSearchParams(window.location.search); - return { - q: params.get("q") || "", - sources: params.get("sources")?.split(",").filter(s => s) || [] - }; +function sourceFilterSQL(col) { + const active = getActiveSources(); + if (active.length === 0) return ' AND 1=0'; // nothing checked = show nothing + if (active.length === 4) return ''; // all checked = no filter + const list = active.map(s => `'${s}'`).join(','); + return ` AND ${col} IN (${list})`; } -``` -## Search & Filters +// === Material/Context Filters === +function getCheckedValues(containerId) { + const checks = document.querySelectorAll(`#${containerId} input[type="checkbox"]`); + return Array.from(checks).filter(c => c.checked).map(c => c.value); +} -```{ojs} -// Search input -viewof searchInput = Inputs.text({ - placeholder: "Search samples (e.g., pottery, basalt, Cyprus...)", - value: initialParams.q, - submit: "Search" -}) -``` +function hasFacetFilters() { + const mat = getCheckedValues('materialFilterBody'); + const ctx = getCheckedValues('contextFilterBody'); + const ot = getCheckedValues('objectTypeFilterBody'); + const matTotal = document.querySelectorAll('#materialFilterBody input[type="checkbox"]').length; + const ctxTotal = document.querySelectorAll('#contextFilterBody input[type="checkbox"]').length; + const otTotal = document.querySelectorAll('#objectTypeFilterBody input[type="checkbox"]').length; + // Active if some (but not all) are checked, or if none are checked + return (mat.length > 0 && mat.length < matTotal) + || (ctx.length > 0 && ctx.length < ctxTotal) + || (ot.length > 0 && ot.length < otTotal); +} -
-
+function facetFilterSQL() { + let sql = ''; + const mat = getCheckedValues('materialFilterBody'); + const matTotal = document.querySelectorAll('#materialFilterBody input[type="checkbox"]').length; + if (mat.length > 0 && mat.length < matTotal) { + const list = mat.map(s => `'${s}'`).join(','); + sql += ` AND f.material IN (${list})`; + } else if (mat.length === 0 && matTotal > 0) { + sql += ' AND 1=0'; + } + const ctx = getCheckedValues('contextFilterBody'); + const ctxTotal = document.querySelectorAll('#contextFilterBody input[type="checkbox"]').length; + if (ctx.length > 0 && ctx.length < ctxTotal) { + const list = ctx.map(s => `'${s}'`).join(','); + sql += ` AND f.context IN (${list})`; + } else if (ctx.length === 0 && ctxTotal > 0) { + sql += ' AND 1=0'; + } + const ot = getCheckedValues('objectTypeFilterBody'); + const otTotal = document.querySelectorAll('#objectTypeFilterBody input[type="checkbox"]').length; + if (ot.length > 0 && ot.length < otTotal) { + const list = ot.map(s => `'${s}'`).join(','); + sql += ` AND f.object_type IN (${list})`; + } else if (ot.length === 0 && otTotal > 0) { + sql += ' AND 1=0'; + } + return sql; +} -### Filters +// === URL State: encode/decode globe state in hash fragment === +function parseNum(val, def, min, max) { + if (val == null) return def; + const n = parseFloat(val); + if (!Number.isFinite(n)) return def; + if (min != null && n < min) return min; + if (max != null && n > max) return max; + return n; +} -```{ojs} -facetSummariesWarning -``` +function readHash() { + const params = new URLSearchParams(location.hash.slice(1)); + return { + v: parseInt(params.get('v')) || 0, + lat: parseNum(params.get('lat'), null, -90, 90), + lng: parseNum(params.get('lng'), null, -180, 180), + alt: parseNum(params.get('alt'), null, 100, 40000000), + heading: parseNum(params.get('heading'), 0, 0, 360), + pitch: parseNum(params.get('pitch'), -90, -90, 0), + mode: params.get('mode') || null, + pid: params.get('pid') || null, + }; +} -**Source** +function buildHash(v) { + const cam = v.camera; + const carto = cam.positionCartographic; + const params = new URLSearchParams(); + params.set('v', '1'); + params.set('lat', Cesium.Math.toDegrees(carto.latitude).toFixed(4)); + params.set('lng', Cesium.Math.toDegrees(carto.longitude).toFixed(4)); + params.set('alt', Math.round(carto.height).toString()); + const heading = Cesium.Math.toDegrees(cam.heading) % 360; + const pitch = Cesium.Math.toDegrees(cam.pitch); + if (Math.abs(heading) > 1) params.set('heading', heading.toFixed(1)); + if (Math.abs(pitch + 90) > 1) params.set('pitch', pitch.toFixed(1)); + const gs = v._globeState; + if (gs.mode === 'point') params.set('mode', 'point'); + if (gs.selectedPid) params.set('pid', gs.selectedPid); + return '#' + params.toString(); +} -```{ojs} -// Source checkboxes with counts - uses pre-computed summaries for instant load -viewof sourceCheckboxes = { - const counts = facetsByType.source; - const options = counts.map(r => r.value); - - return Inputs.checkbox(options, { - value: initialParams.sources.filter(s => options.includes(s)), - format: (x) => { - const r = counts.find(s => s.value === x); - const color = SOURCE_COLORS[x] || SOURCE_COLORS.default; - const count = r ? Number(r.count).toLocaleString() : "0"; - return html` - - ${x} (${count}) - `; - } - }); +// === Helpers: update DOM imperatively (no OJS reactivity) === +function updateStats(phase, points, samples, time, pointsLabel, samplesLabel) { + const s = (id, v) => { const e = document.getElementById(id); if (e) e.textContent = v; }; + s('sPhase', phase); + s('sPoints', typeof points === 'string' ? points : points.toLocaleString()); + s('sSamples', typeof samples === 'string' ? samples : samples.toLocaleString()); + if (time != null) s('sTime', time); + if (pointsLabel) s('sPointsLbl', pointsLabel); + if (samplesLabel) s('sSamplesLbl', samplesLabel); } -``` -**Material** +function updatePhaseMsg(text, type) { + const m = document.getElementById('phaseMsg'); + if (!m) return; + m.textContent = text; + if (type === 'loading') { m.style.background = '#e3f2fd'; m.style.color = '#1565c0'; } + else { m.style.background = '#e8f5e9'; m.style.color = '#2e7d32'; } +} -```{ojs} -// Material filter - loaded from pre-computed summaries -viewof materialCheckboxes = { - const counts = facetsByType.material; - const options = counts.map(r => r.value); - return Inputs.checkbox(options, { - value: [], - format: (x) => { - const r = counts.find(s => s.value === x); - const count = r ? Number(r.count).toLocaleString() : "0"; - return html` - ${prettyLabel(x)} (${count}) - `; +function updateClusterCard(info) { + const el = document.getElementById('clusterSection'); + if (!el) return; + if (!info) { + el.innerHTML = '
Click a cluster or sample on the globe
'; + return; } - }); + const color = SOURCE_COLORS[info.source] || '#666'; + const name = SOURCE_NAMES[info.source] || info.source; + el.innerHTML = `

Selected Cluster

+
+
+ ${name} + H3 res${info.resolution} +
+
+ ${info.count.toLocaleString()} samples +
+
+ ${info.lat.toFixed(4)}, ${info.lng.toFixed(4)} +
+
`; } -``` -**Sampled Feature** +function updateSampleCard(sample) { + const el = document.getElementById('clusterSection'); + if (!el) return; + const color = SOURCE_COLORS[sample.source] || '#666'; + const name = SOURCE_NAMES[sample.source] || sample.source; + const placeParts = sample.place_name; + const placeStr = Array.isArray(placeParts) && placeParts.length > 0 + ? placeParts.filter(Boolean).join(' › ') + : ''; + const srcUrl = sourceUrl(sample.pid); + el.innerHTML = `

Sample

+
+
+ ${name} +
+
+ ${sample.label || sample.pid || 'Unnamed'} +
+
+ ${sample.lat.toFixed(5)}, ${sample.lng.toFixed(5)} +
+ ${placeStr ? `
${placeStr}
` : ''} + ${sample.result_time ? `
Date: ${sample.result_time}
` : ''} + ${srcUrl ? `` : ''} +
Loading full details...
+
`; +} -```{ojs} -// Context filter - loaded from pre-computed summaries -viewof contextCheckboxes = { - const counts = facetsByType.context; - const options = counts.map(r => r.value); - return Inputs.checkbox(options, { - value: [], - format: (x) => { - const r = counts.find(s => s.value === x); - const count = r ? Number(r.count).toLocaleString() : "0"; - return html` - ${prettyLabel(x)} (${count}) - `; +function updateSampleDetail(detail) { + const el = document.getElementById('sampleDetail'); + if (!el) return; + if (!detail) { + el.innerHTML = 'Detail query failed'; + return; } - }); + const desc = detail.description + ? (detail.description.length > 300 ? detail.description.slice(0, 300) + '...' : detail.description) + : ''; + el.innerHTML = `${desc ? `
${desc}
` : ''}`; } -``` -**Specimen Type** - -```{ojs} -// Object type filter - loaded from pre-computed summaries -viewof objectTypeCheckboxes = { - const counts = facetsByType.object_type; - const options = counts.map(r => r.value); - return Inputs.checkbox(options, { - value: [], - format: (x) => { - const r = counts.find(s => s.value === x); - const count = r ? Number(r.count).toLocaleString() : "0"; - return html` - ${prettyLabel(x)} (${count}) - `; +function updateSamples(samples) { + const el = document.getElementById('samplesSection'); + if (!el) return; + if (!samples || samples.length === 0) { + el.innerHTML = ''; + return; + } + let h = `

Nearby Samples (${samples.length})

`; + for (const s of samples) { + const color = SOURCE_COLORS[s.source] || '#666'; + const name = SOURCE_NAMES[s.source] || s.source; + const placeParts = s.place_name; + const desc = Array.isArray(placeParts) && placeParts.length > 0 + ? placeParts.filter(Boolean).join(' › ') + : ''; + const sUrl = sourceUrl(s.pid); + h += `
+
+ ${sUrl ? `${s.label || s.pid}` : `${s.label || s.pid}`} + ${name} +
+ ${desc ? `
${desc}
` : ''} +
`; } - }); + el.innerHTML = h; } -``` - -```{ojs} -html`Clear All Filters` -``` - -**Max Samples** -```{ojs} -viewof maxSamples = Inputs.range([1000, 100000], { - value: 25000, - step: 1000 -}) -``` - -
-
- -```{ojs} -// Update URL without reloading -{ - const params = new URLSearchParams(); - if (searchInput) params.set("q", searchInput); - if (sourceCheckboxes?.length) params.set("sources", sourceCheckboxes.join(",")); - if (materialCheckboxes?.length) params.set("material", materialCheckboxes.join(",")); - if (contextCheckboxes?.length) params.set("context", contextCheckboxes.join(",")); - if (objectTypeCheckboxes?.length) params.set("object_type", objectTypeCheckboxes.join(",")); - - const newUrl = params.toString() ? `?${params.toString()}` : window.location.pathname; - if (window.location.search !== `?${params.toString()}`) { - history.replaceState(null, "", newUrl); - } +// === Results table (below globe) === +// Renders the first N samples matching current filters, regardless of camera. +// Stable, sourceless ordering (no ORDER BY RANDOM) keeps it cheap on lite_url. +function renderResultsTable(rows) { + const el = document.getElementById('resultsTable'); + if (!el) return; + if (!rows || rows.length === 0) { + el.innerHTML = '
No samples match the current filters.
'; + return; + } + const head = `SourceLabelPlaceLatLon`; + const body = rows.map(r => { + const color = SOURCE_COLORS[r.source] || '#666'; + const name = SOURCE_NAMES[r.source] || r.source; + const placeParts = r.place_name; + const place = Array.isArray(placeParts) && placeParts.length > 0 + ? placeParts.filter(Boolean).join(' › ') + : ''; + const lat = (r.latitude != null) ? r.latitude.toFixed(4) : ''; + const lng = (r.longitude != null) ? r.longitude.toFixed(4) : ''; + const sUrl = sourceUrl(r.pid); + const labelHtml = sUrl + ? `${r.label || r.pid}` + : (r.label || r.pid); + return ` + ${name} + ${labelHtml} + ${place} + ${lat} + ${lng} + `; + }).join(''); + el.innerHTML = `${head}${body}
`; } -``` -
- -
- SESAR - OpenContext - GEOME - Smithsonian -
- -
-Loading data... -
- -```{ojs} -// Show result count -html`
- Showing ${sampleData.length.toLocaleString()} of ${Number(totalCount).toLocaleString()} matching samples -
` +function updateResultsTableMeta(text, isLoading) { + const el = document.getElementById('resultsTableMeta'); + if (!el) return; + el.textContent = text; + el.style.color = isLoading ? '#1565c0' : '#555'; +} ``` -
-
- -### Results - ```{ojs} -// Full-width results table -Inputs.table(sampleData, { - columns: ['source', 'label', 'latitude', 'longitude'], - header: { - source: 'Source', - label: 'Label', - latitude: 'Lat', - longitude: 'Lon' - }, - format: { - source: (x) => html`${x}`, - latitude: (x) => x?.toFixed(4), - longitude: (x) => x?.toFixed(4) - }, - rows: 20 -}) -``` +//| echo: false +//| output: false -```{ojs} -// Initialize DuckDB-WASM +// === DuckDB === db = { - performance.mark('explorer-db-start'); - const bundle = await duckdbModule.selectBundle(duckdbModule.getJsDelivrBundles()); - - const worker_url = URL.createObjectURL( - new Blob([`importScripts("${bundle.mainWorker}");`], {type: 'text/javascript'}) - ); - const worker = new Worker(worker_url); - const logger = new duckdbModule.ConsoleLogger(duckdbModule.LogLevel.WARNING); - - const instance = new duckdbModule.AsyncDuckDB(logger, worker); - await instance.instantiate(bundle.mainModule, bundle.pthreadWorker); - URL.revokeObjectURL(worker_url); - - // Create views for convenience. v1 reads the full wide parquet directly; - // v2 reads the 60 MB lite file (no description, no row_id, source is - // already named 'source' not 'n'). - const conn = await instance.connect(); - if (explorerVersion === 'v2') { - await conn.query(` - CREATE VIEW samples AS - SELECT pid, label, source, latitude, longitude, place_name - FROM read_parquet('${parquet_url}') - `); - } else { - await conn.query(`CREATE VIEW samples AS SELECT * FROM read_parquet('${parquet_url}')`); - } - // Slim facets view with correct URI-string columns for cross-filtering - await conn.query(`CREATE VIEW sample_facets AS SELECT * FROM read_parquet('${sample_facets_url}')`); - await conn.close(); - - performance.mark('explorer-db-end'); - performance.measure('explorer_db', 'explorer-db-start', 'explorer-db-end'); - return instance; -} - -// Helper function to run queries -async function runQuery(sql) { - const conn = await db.connect(); - try { - const result = await conn.query(sql); - return result.toArray().map(row => { - const obj = row.toJSON(); - // Convert BigInt to Number - for (const key in obj) { - if (typeof obj[key] === 'bigint') obj[key] = Number(obj[key]); - } - return obj; - }); - } finally { - await conn.close(); - } + performance.mark('duckdb-init-start'); + const instance = await DuckDBClient.of(); + performance.mark('duckdb-init-end'); + performance.measure('duckdb_init', 'duckdb-init-start', 'duckdb-init-end'); + return instance; } ``` ```{ojs} -mutable facetSummariesError = null -``` +//| echo: false +//| output: false -```{ojs} -// Tier 1: Load pre-computed facet summaries (2KB, instant) -facetSummaries = { - mutable facetSummariesError = null; - performance.mark('explorer-facets-start'); - try { - const rows = await runQuery(`SELECT * FROM read_parquet('${facet_summaries_url}')`); - performance.mark('explorer-facets-end'); - performance.measure('explorer_facets', 'explorer-facets-start', 'explorer-facets-end'); - return rows; - } catch (e) { - console.error("Facet summaries load error:", e); - mutable facetSummariesError = e; - return []; - } -} +// === Cesium Viewer (created once, never re-created) === +viewer = { + performance.mark('viewer-init-start'); + const v = new Cesium.Viewer("cesiumContainer", { + timeline: false, + animation: false, + baseLayerPicker: false, + fullscreenElement: "cesiumContainer", + terrain: Cesium.Terrain.fromWorldTerrain() + }); -``` + // URL deep-link state (must be set before globalRect/once block reads it) + v._globeState = { mode: 'cluster', selectedPid: null }; + v._initialHash = readHash(); + v._suppressHashWrite = true; // cleared after zoomWatcher initializes + v._suppressTimer = null; + + const globalRect = Cesium.Rectangle.fromDegrees(-180, -60, 180, 80); + Cesium.Camera.DEFAULT_VIEW_RECTANGLE = globalRect; + Cesium.Camera.DEFAULT_VIEW_FACTOR = 0.5; + const ih = v._initialHash; + const once = () => { + if (ih.lat != null && ih.lng != null) { + v.camera.setView({ + destination: Cesium.Cartesian3.fromDegrees(ih.lng, ih.lat, ih.alt || 20000000), + orientation: { + heading: Cesium.Math.toRadians(ih.heading), + pitch: Cesium.Math.toRadians(ih.pitch) + } + }); + } else { + v.camera.setView({ destination: globalRect }); + } + v.scene.postRender.removeEventListener(once); + }; + v.scene.postRender.addEventListener(once); + + // Two separate point collections: clusters and individual samples + v.h3Points = new Cesium.PointPrimitiveCollection(); + v.scene.primitives.add(v.h3Points); + + v.samplePoints = new Cesium.PointPrimitiveCollection(); + v.scene.primitives.add(v.samplePoints); + v.samplePoints.show = false; // hidden until point mode + + // Hover tooltip — works for both clusters and samples + v.pointLabel = v.entities.add({ + label: { + show: false, showBackground: true, font: "13px monospace", + horizontalOrigin: Cesium.HorizontalOrigin.LEFT, + verticalOrigin: Cesium.VerticalOrigin.BOTTOM, + pixelOffset: new Cesium.Cartesian2(15, 0), + disableDepthTestDistance: Number.POSITIVE_INFINITY, text: "", + } + }); -```{ojs} -facetSummariesWarning = { - if (!facetSummariesError) return null; - return html`
- Facet summaries failed to load. Filter counts may be missing. Try refreshing. -
`; -} + new Cesium.ScreenSpaceEventHandler(v.scene.canvas).setInputAction((movement) => { + const picked = v.scene.pick(movement.endPosition); + if (Cesium.defined(picked) && picked.primitive && picked.id) { + v.pointLabel.position = picked.primitive.position; + v.pointLabel.label.show = true; + const meta = picked.id; + if (typeof meta === 'object' && meta.type === 'sample') { + v.pointLabel.label.text = `${meta.label || meta.pid}`; + } else if (typeof meta === 'object' && meta.count) { + v.pointLabel.label.text = `${meta.source}: ${meta.count.toLocaleString()} samples`; + } else { + v.pointLabel.label.text = String(meta); + } + } else { + v.pointLabel.label.show = false; + } + }, Cesium.ScreenSpaceEventType.MOUSE_MOVE); + + // Click handler — routes to cluster card or sample card + new Cesium.ScreenSpaceEventHandler(v.scene.canvas).setInputAction(async (e) => { + const picked = v.scene.pick(e.position); + if (!Cesium.defined(picked) || !picked.primitive || !picked.id) return; + const meta = picked.id; + + if (typeof meta === 'object' && meta.type === 'sample') { + // --- Individual sample click --- + updateSampleCard(meta); + v._globeState.selectedPid = meta.pid; + history.pushState(null, '', buildHash(v)); + // Clear nearby list + const sampEl = document.getElementById('samplesSection'); + if (sampEl) sampEl.innerHTML = ''; + + // Stage 2: lazy-load full description from wide parquet + try { + const detail = await db.query(` + SELECT description + FROM read_parquet('${wide_url}') + WHERE pid = '${meta.pid.replace(/'/g, "''")}' + LIMIT 1 + `); + if (detail && detail.length > 0) { + updateSampleDetail(detail[0]); + } else { + updateSampleDetail({ description: '' }); + } + } catch(err) { + console.error("Detail query failed:", err); + updateSampleDetail(null); + } + + } else if (typeof meta === 'object' && meta.count) { + // --- Cluster click --- + updateClusterCard(meta); + v._globeState.selectedPid = null; + history.pushState(null, '', buildHash(v)); + + const sampEl = document.getElementById('samplesSection'); + if (sampEl) sampEl.innerHTML = '
Loading nearby samples...
'; + + const delta = meta.resolution === 4 ? 2.0 : meta.resolution === 6 ? 0.5 : 0.1; + try { + const facetActive = hasFacetFilters(); + const facetSQL = facetActive ? facetFilterSQL() : ''; + let nearbyQuery; + if (facetActive) { + nearbyQuery = ` + SELECT l.pid, l.label, l.source, l.latitude, l.longitude, l.place_name + FROM read_parquet('${lite_url}') l + JOIN read_parquet('${facets_url}') f ON l.pid = f.pid + WHERE l.latitude BETWEEN ${meta.lat - delta} AND ${meta.lat + delta} + AND l.longitude BETWEEN ${meta.lng - delta} AND ${meta.lng + delta} + ${sourceFilterSQL('l.source')} + ${facetSQL} + LIMIT 30 + `; + } else { + nearbyQuery = ` + SELECT pid, label, source, latitude, longitude, place_name + FROM read_parquet('${lite_url}') + WHERE latitude BETWEEN ${meta.lat - delta} AND ${meta.lat + delta} + AND longitude BETWEEN ${meta.lng - delta} AND ${meta.lng + delta} + ${sourceFilterSQL('source')} + LIMIT 30 + `; + } + const samples = await db.query(nearbyQuery); + updateSamples(samples); + } catch(err) { + console.error("Sample query failed:", err); + if (sampEl) sampEl.innerHTML = '
Query failed — try again.
'; + } + } + }, Cesium.ScreenSpaceEventType.LEFT_CLICK); -// Extract facet counts by type from pre-computed summaries (baseline) -facetsByType = { - const grouped = { source: [], material: [], context: [], object_type: [] }; - for (const row of facetSummaries) { - const ft = row.facet_type; - if (grouped[ft]) { - grouped[ft].push({ value: row.facet_value, count: Number(row.count), scheme: row.scheme }); - } - } - // Sort each by count descending - for (const key of Object.keys(grouped)) { - grouped[key].sort((a, b) => b.count - a.count); - } - return grouped; -} -``` + // Timing: viewer ready (mount complete, pre-first-render) + performance.mark('viewer-init-end'); + performance.measure('viewer_init', 'viewer-init-start', 'viewer-init-end'); -```{ojs} -// Load SKOS prefLabels for vocabulary URIs (#148). Tiny lookup (~60KB); -// fallback to last URI segment if a URI isn't covered. -vocabLabels = { - try { - const rows = await runQuery( - `SELECT uri, pref_label FROM read_parquet('${vocab_labels_url}') WHERE lang = 'en'` - ); - const m = new Map(); - for (const r of rows) m.set(r.uri, r.pref_label); - return m; - } catch (e) { - console.warn("vocab_labels load failed; falling back to URI tails:", e); - return new Map(); - } -} + // Timing: first Cesium frame painted (globe visible, may be pre-cluster) + const firstGlobeFrame = () => { + performance.mark('first-globe-frame'); + v.scene.postRender.removeEventListener(firstGlobeFrame); + }; + v.scene.postRender.addEventListener(firstGlobeFrame); -prettyLabel = function (uri) { - if (uri == null) return ""; - const hit = vocabLabels.get(uri); - if (hit) return hit; - // Fallback: last non-empty path segment of the URI - const s = String(uri); - if (!/^https?:\/\//.test(s)) return s; - const parts = s.replace(/[#?].*$/, "").split("/").filter(Boolean); - return parts.length ? parts[parts.length - 1] : s; + return v; } ``` ```{ojs} -// Cross-filter: build WHERE clause excluding one facet dimension -// Queries the sample_facets view (URI strings, correct column names) -function buildCrossFilterWhere(excludeFacet) { - const conditions = []; - - // Text search participates in cross-filtering - if (searchInput?.trim()) { - const term = searchInput.trim().replace(/'/g, "''"); - conditions.push(`( - label ILIKE '%${term}%' - OR description ILIKE '%${term}%' - OR CAST(place_name AS VARCHAR) ILIKE '%${term}%' - )`); - } - - if (excludeFacet !== 'source') { - const sources = Array.from(sourceCheckboxes || []); - if (sources.length > 0) { - const sourceList = sources.map(s => `'${s}'`).join(", "); - conditions.push(`source IN (${sourceList})`); +//| echo: false +//| output: false + +// === PHASE 1: Load H3 res4 globally (instant) === +phase1 = { + performance.mark('p1-start'); + + // Read ?sources=A,B from query string and pre-check the matching legend + // checkboxes BEFORE running the cluster query (so phase 1 honors the + // bookmarkable filter). Camera state still lives in the hash. + { + const initialSources = (new URLSearchParams(location.search).get('sources') || '') + .split(',').map(s => s.trim()).filter(Boolean); + if (initialSources.length > 0) { + document.querySelectorAll('#sourceFilter input[type="checkbox"]').forEach(cb => { + cb.checked = initialSources.includes(cb.value); + }); + document.querySelectorAll('#sourceFilter .legend-item').forEach(li => { + const cb = li.querySelector('input'); + li.classList.toggle('disabled', !cb.checked); + }); + } } - } - if (excludeFacet !== 'material') { - const materials = Array.from(materialCheckboxes || []); - if (materials.length > 0) { - const matList = materials.map(m => `'${m.replace(/'/g, "''")}'`).join(", "); - conditions.push(`material IN (${matList})`); - } - } + const data = await db.query(` + SELECT h3_cell, sample_count, center_lat, center_lng, + dominant_source, source_count + FROM read_parquet('${h3_res4_url}') + WHERE 1=1${sourceFilterSQL('dominant_source')} + `); - if (excludeFacet !== 'context') { - const contexts = Array.from(contextCheckboxes || []); - if (contexts.length > 0) { - const ctxList = contexts.map(c => `'${c.replace(/'/g, "''")}'`).join(", "); - conditions.push(`context IN (${ctxList})`); + const scalar = new Cesium.NearFarScalar(1.5e2, 1.5, 8.0e6, 0.5); + let totalSamples = 0; + + for (const row of data) { + const count = row.sample_count; + totalSamples += count; + const size = Math.min(3 + Math.log10(count) * 4, 20); + viewer.h3Points.add({ + id: { count, source: row.dominant_source, lat: row.center_lat, lng: row.center_lng, resolution: 4 }, + position: Cesium.Cartesian3.fromDegrees(row.center_lng, row.center_lat, 0), + pixelSize: size, + color: Cesium.Color.fromCssColorString(SOURCE_COLORS[row.dominant_source] || '#666').withAlpha(0.8), + scaleByDistance: scalar, + }); } - } - if (excludeFacet !== 'object_type') { - const objectTypes = Array.from(objectTypeCheckboxes || []); - if (objectTypes.length > 0) { - const otList = objectTypes.map(o => `'${o.replace(/'/g, "''")}'`).join(", "); - conditions.push(`object_type IN (${otList})`); - } - } + // Cache cluster data for viewport counting + viewer._clusterData = Array.from(data); + viewer._clusterTotal = { clusters: data.length, samples: totalSamples }; - return conditions.length > 0 ? conditions.join(" AND ") : "1=1"; -} -``` + performance.mark('p1-end'); + performance.measure('p1', 'p1-start', 'p1-end'); + const elapsed = performance.getEntriesByName('p1').pop().duration; -```{ojs} -// Detect whether any filter is active (triggers cross-filter queries) -hasActiveFilters = { - const hasSearch = searchInput?.trim()?.length > 0; - const hasSources = (sourceCheckboxes || []).length > 0; - const hasMaterials = (materialCheckboxes || []).length > 0; - const hasContexts = (contextCheckboxes || []).length > 0; - const hasObjectTypes = (objectTypeCheckboxes || []).length > 0; - return hasSearch || hasSources || hasMaterials || hasContexts || hasObjectTypes; + updateStats('H3 Res4', data.length, totalSamples, `${(elapsed/1000).toFixed(1)}s`, 'Clusters Loaded', 'Samples Loaded'); + updatePhaseMsg(`${data.length.toLocaleString()} clusters, ${totalSamples.toLocaleString()} samples. Zoom in for finer detail.`, 'done'); + console.log(`Phase 1: ${data.length} clusters in ${elapsed.toFixed(0)}ms`); + + return { count: data.length, samples: totalSamples }; } ``` ```{ojs} -// Cross-filtered facet counts: use pre-computed cache for single-filter, -// fall back to on-the-fly queries against sample_facets for multi-filter -crossFilteredFacets = { - if (!hasActiveFilters) return null; // Use pre-computed summaries when no filters - - // Count how many facets have active filters - const activeSources = Array.from(sourceCheckboxes || []); - const activeMaterials = Array.from(materialCheckboxes || []); - const activeContexts = Array.from(contextCheckboxes || []); - const activeObjectTypes = Array.from(objectTypeCheckboxes || []); - const hasSearch = searchInput?.trim()?.length > 0; - - const activeFilterCount = [activeSources, activeMaterials, activeContexts, activeObjectTypes] - .filter(a => a.length > 0).length; - - // Try pre-computed cache: exactly one facet active, exactly one value, no text search - const singleValueFacet = ( - !hasSearch && activeFilterCount === 1 && - [activeSources, activeMaterials, activeContexts, activeObjectTypes] - .every(a => a.length <= 1) - ); - - if (singleValueFacet) { +//| echo: false +//| output: false + +// === Load facet summaries and populate filter checkboxes === +facetFilters = { + if (!phase1) return; try { - const conditions = ["filter_source IS NULL", "filter_material IS NULL", - "filter_context IS NULL", "filter_object_type IS NULL"]; - if (activeSources.length === 1) - conditions[0] = `filter_source = '${activeSources[0].replace(/'/g, "''")}'`; - else if (activeMaterials.length === 1) - conditions[1] = `filter_material = '${activeMaterials[0].replace(/'/g, "''")}'`; - else if (activeContexts.length === 1) - conditions[2] = `filter_context = '${activeContexts[0].replace(/'/g, "''")}'`; - else if (activeObjectTypes.length === 1) - conditions[3] = `filter_object_type = '${activeObjectTypes[0].replace(/'/g, "''")}'`; - - const sql = ` - SELECT facet_type, facet_value AS value, count - FROM read_parquet('${cross_filter_url}') - WHERE ${conditions.join(" AND ")} - `; - const rows = await runQuery(sql); - - if (rows.length > 0) { - const results = { source: [], material: [], context: [], object_type: [] }; - for (const r of rows) { - if (results[r.facet_type]) { - results[r.facet_type].push({ value: r.value, count: Number(r.count) }); - } + const summaries = await db.query(` + SELECT facet_type, facet_value, count + FROM read_parquet('${facet_summaries_url}') + ORDER BY facet_type, count DESC + `); + + const grouped = { material: [], context: [], object_type: [] }; + for (const row of summaries) { + if (grouped[row.facet_type]) { + // Extract short label from URI + const shortLabel = row.facet_value.split('/').pop() || row.facet_value; + grouped[row.facet_type].push({ value: shortLabel, fullUri: row.facet_value, count: row.count }); + } } - return results; - } - } catch (e) { - console.warn("Pre-computed cache miss, falling back to on-the-fly:", e); - } - } - - // Fallback: on-the-fly queries against the slim sample_facets view - const facetConfig = [ - { key: 'source', column: 'source', exclude: 'source' }, - { key: 'material', column: 'material', exclude: 'material' }, - { key: 'context', column: 'context', exclude: 'context' }, - { key: 'object_type', column: 'object_type', exclude: 'object_type' }, - ]; - - const results = {}; - - const queries = facetConfig.map(async ({ key, column, exclude }) => { - const where = buildCrossFilterWhere(exclude); - const sql = ` - SELECT ${column} AS value, COUNT(*) AS count - FROM sample_facets - WHERE ${where} AND ${column} IS NOT NULL - GROUP BY ${column} - ORDER BY count DESC - `; - try { - const rows = await runQuery(sql); - results[key] = rows.map(r => ({ value: r.value, count: r.count })); - } catch (e) { - console.warn(`Cross-filter query failed for ${key}:`, e); - results[key] = null; - } - }); - await Promise.all(queries); - return results; -} -``` - -```{ojs} -// Update facet count labels in-place when cross-filtered counts change -// This avoids re-rendering checkboxes (which would reset user selections) -{ - if (!crossFilteredFacets) { - // No active filters — restore baseline counts and remove dimming - for (const facetKey of ['source', 'material', 'context', 'object_type']) { - const baseline = facetsByType[facetKey] || []; - const countMap = new Map(baseline.map(r => [r.value, r.count])); - document.querySelectorAll(`.facet-count[data-facet="${facetKey}"]`).forEach(el => { - const value = el.getAttribute('data-value'); - const count = countMap.get(value) ?? 0; - el.textContent = `(${Number(count).toLocaleString()})`; - el.style.opacity = '1'; - }); + const renderFilter = (bodyId, items) => { + const body = document.getElementById(bodyId); + if (body && items.length > 0) { + body.innerHTML = items.map(it => + `` + ).join(''); + } + }; + + renderFilter('materialFilterBody', grouped.material); + renderFilter('contextFilterBody', grouped.context); + renderFilter('objectTypeFilterBody', grouped.object_type); + + console.log(`Facet filters loaded: ${grouped.material.length} materials, ${grouped.context.length} contexts, ${grouped.object_type.length} object types`); + } catch(err) { + console.warn("Facet summaries failed to load:", err); } - return; - } - - for (const [facetKey, rows] of Object.entries(crossFilteredFacets)) { - if (!rows) continue; - const countMap = new Map(rows.map(r => [r.value, r.count])); - - document.querySelectorAll(`.facet-count[data-facet="${facetKey}"]`).forEach(el => { - const value = el.getAttribute('data-value'); - const count = countMap.get(value) ?? 0; - el.textContent = `(${Number(count).toLocaleString()})`; - el.style.opacity = count === 0 ? '0.4' : '1'; - }); - } + return "loaded"; } ``` ```{ojs} -// Build WHERE clause from current filters (Tier 2: queries full parquet only when filtering) -// Source filter uses the wide parquet's `n` column directly. -// Material/context/object_type filters use the sample_facets view (URI strings) -// via a subquery, since the wide parquet stores these as BIGINT foreign keys. -whereClause = { - const conditions = ["latitude IS NOT NULL"]; - - // v1 reads the multi-entity-type wide parquet, so filter to sample records. - // v2 reads lite which is already samples-only. - if (explorerVersion !== 'v2') { - conditions.unshift("otype = 'MaterialSampleRecord'"); - } - - // Text search. v1 can search description (column exists in wide); - // v2 can't (description is not in lite — lazy-fetched on sample click). - if (searchInput?.trim()) { - const term = searchInput.trim().replace(/'/g, "''"); - if (explorerVersion === 'v2') { - conditions.push(`( - label ILIKE '%${term}%' - OR CAST(place_name AS VARCHAR) ILIKE '%${term}%' - )`); - } else { - conditions.push(`( - label ILIKE '%${term}%' - OR description ILIKE '%${term}%' - OR CAST(place_name AS VARCHAR) ILIKE '%${term}%' - )`); +//| echo: false +//| output: false + +// === Zoom watcher: H3 cluster mode + individual sample point mode === +zoomWatcher = { + if (!phase1) return; + if (!facetFilters) return; // wait for facet checkboxes + + // --- State --- + let mode = 'cluster'; // 'cluster' or 'point' + let currentRes = 4; + let loading = false; + let requestId = 0; // stale-request guard + // clusterDataCache stored on viewer._clusterData (set by phase1 and loadRes) + + // Hysteresis thresholds to avoid flicker + const ENTER_POINT_ALT = 120000; // 120 km → enter point mode + const EXIT_POINT_ALT = 180000; // 180 km → exit point mode + const POINT_BUDGET = 5000; + + // Viewport cache: avoid re-querying same area + let cachedBounds = null; // { south, north, west, east } + let cachedData = null; // array of rows + + // --- H3 cluster loading (existing logic) --- + let loadResGen = 0; // generation counter to discard stale results + const loadRes = async (res, url) => { + const gen = ++loadResGen; // claim a generation + loading = true; + updatePhaseMsg(`Loading H3 res${res}...`, 'loading'); + + try { + performance.mark(`r${res}-s`); + const data = await db.query(` + SELECT h3_cell, sample_count, center_lat, center_lng, + dominant_source, source_count + FROM read_parquet('${url}') + WHERE 1=1${sourceFilterSQL('dominant_source')} + `); + + if (gen !== loadResGen) return; // stale — a newer call superseded this one + viewer.h3Points.removeAll(); + const scalar = new Cesium.NearFarScalar(1.5e2, 1.5, 8.0e6, 0.3); + let total = 0; + + for (const row of data) { + total += row.sample_count; + const size = Math.min(3 + Math.log10(row.sample_count) * 3.5, 18); + viewer.h3Points.add({ + id: { count: row.sample_count, source: row.dominant_source, lat: row.center_lat, lng: row.center_lng, resolution: res }, + position: Cesium.Cartesian3.fromDegrees(row.center_lng, row.center_lat, 0), + pixelSize: size, + color: Cesium.Color.fromCssColorString(SOURCE_COLORS[row.dominant_source] || '#666').withAlpha(0.85), + scaleByDistance: scalar, + }); + } + + // Cache for viewport counting + viewer._clusterData = Array.from(data); + viewer._clusterTotal = { clusters: data.length, samples: total }; + + performance.mark(`r${res}-e`); + performance.measure(`r${res}`, `r${res}-s`, `r${res}-e`); + const elapsed = performance.getEntriesByName(`r${res}`).pop().duration; + + // Show viewport count immediately + const bounds = getViewportBounds(); + const inView = countInViewport(bounds); + updateStats(`H3 Res${res}`, `${inView.clusters.toLocaleString()} / ${data.length.toLocaleString()}`, inView.samples.toLocaleString(), `${(elapsed/1000).toFixed(1)}s`, 'Clusters in View / Loaded', 'Samples in View'); + updatePhaseMsg(`${data.length.toLocaleString()} clusters, ${total.toLocaleString()} samples. ${res < 8 ? 'Zoom in for finer detail.' : 'Zoom closer for individual samples.'}`, 'done'); + + currentRes = res; + console.log(`Res${res}: ${data.length} clusters in ${elapsed.toFixed(0)}ms`); + } catch(err) { + console.error(`Failed to load res${res}:`, err); + updatePhaseMsg(`Failed to load H3 res${res} — try zooming again.`, 'loading'); + } finally { + loading = false; + } + }; + + // --- Get camera viewport bounds --- + function getViewportBounds() { + const rect = viewer.camera.computeViewRectangle(viewer.scene.globe.ellipsoid); + if (!rect) return null; + return { + south: Cesium.Math.toDegrees(rect.south), + north: Cesium.Math.toDegrees(rect.north), + west: Cesium.Math.toDegrees(rect.west), + east: Cesium.Math.toDegrees(rect.east) + }; } - } - - // Source filter. v1 uses the wide parquet's `n` column; v2 uses `source`. - const sources = Array.from(sourceCheckboxes || []); - if (sources.length > 0) { - const sourceList = sources.map(s => `'${s}'`).join(", "); - const col = explorerVersion === 'v2' ? 'source' : 'n'; - conditions.push(`${col} IN (${sourceList})`); - } - - // Facet filters: build a subquery against sample_facets to get matching PIDs - const facetConditions = []; - const materials = Array.from(materialCheckboxes || []); - if (materials.length > 0) { - const matList = materials.map(m => `'${m.replace(/'/g, "''")}'`).join(", "); - facetConditions.push(`material IN (${matList})`); - } - const contexts = Array.from(contextCheckboxes || []); - if (contexts.length > 0) { - const ctxList = contexts.map(c => `'${c.replace(/'/g, "''")}'`).join(", "); - facetConditions.push(`context IN (${ctxList})`); - } - const objectTypes = Array.from(objectTypeCheckboxes || []); - if (objectTypes.length > 0) { - const otList = objectTypes.map(o => `'${o.replace(/'/g, "''")}'`).join(", "); - facetConditions.push(`object_type IN (${otList})`); - } - - if (facetConditions.length > 0) { - conditions.push(`pid IN (SELECT pid FROM sample_facets WHERE ${facetConditions.join(" AND ")})`); - } - - return conditions.join(" AND "); -} -``` - -```{ojs} -// Source counts now come from pre-computed facet summaries (Tier 1) -// No longer scans the full parquet file on every page load -sourceCounts = facetsByType.source -``` -```{ojs} -// Get total count matching current filters -totalCount = { - performance.mark('explorer-count-start'); - const query = `SELECT COUNT(*) as count FROM samples WHERE ${whereClause}`; - try { - const rows = await runQuery(query); - performance.mark('explorer-count-end'); - performance.measure('explorer_count', 'explorer-count-start', 'explorer-count-end'); - return rows[0]?.count || 0; - } catch (e) { - return 0; - } -} -``` + // --- Count clusters visible in current viewport (from cached array) --- + function countInViewport(bounds) { + const cache = viewer._clusterData; + if (!bounds || !cache || cache.length === 0) return { clusters: 0, samples: 0 }; + const { south, north, west, east } = bounds; + const wrapLng = west > east; // dateline crossing + let clusters = 0, samples = 0; + for (const row of cache) { + if (row.center_lat < south || row.center_lat > north) continue; + if (wrapLng ? (row.center_lng < west && row.center_lng > east) : (row.center_lng < west || row.center_lng > east)) continue; + clusters++; + samples += row.sample_count; + } + return { clusters, samples }; + } -```{ojs} -// Load sample data -sampleData = { - const statusDiv = document.getElementById('loading_status'); - if (statusDiv) { - statusDiv.style.display = 'block'; - statusDiv.textContent = 'Loading samples...'; - } - - performance.mark('explorer-samples-start'); - try { - // v2: read from lite (60 MB), no description (fetched lazily on click), - // no row_id, no ORDER BY RANDOM(). LIMIT returns whatever rows the - // scan encounters first — biased toward row order but ~20x faster - // than RANDOM() on a columnar file. - // v1: original query against the 278 MB wide file. - const query = explorerVersion === 'v2' ? ` - SELECT - pid, - label, - '' AS description, - latitude, - longitude, - source, - place_name - FROM samples - WHERE ${whereClause} - LIMIT ${maxSamples} - ` : ` - SELECT - row_id, - pid, - label, - COALESCE(description, '') as description, - latitude, - longitude, - n as source, - place_name - FROM samples - WHERE ${whereClause} - ORDER BY RANDOM() - LIMIT ${maxSamples} - `; + // --- Check if viewport is within cached bounds --- + function isWithinCache(bounds) { + if (!cachedBounds || !bounds) return false; + return bounds.south >= cachedBounds.south && + bounds.north <= cachedBounds.north && + bounds.west >= cachedBounds.west && + bounds.east <= cachedBounds.east; + } - const data = await runQuery(query); + // --- Load individual samples for current viewport --- + async function loadViewportSamples() { + const myReqId = ++requestId; + const bounds = getViewportBounds(); + if (!bounds) return; - performance.mark('explorer-samples-end'); - performance.measure('explorer_samples', 'explorer-samples-start', 'explorer-samples-end'); + // If viewport is within cached area, just re-render from cache + if (isWithinCache(bounds) && cachedData) { + renderSamplePoints(cachedData, bounds); + return; + } - if (statusDiv) { - statusDiv.textContent = `Loaded ${data.length.toLocaleString()} samples`; - setTimeout(() => { statusDiv.style.display = 'none'; }, 2000); + // Fetch with 30% padding for smooth panning + const latPad = (bounds.north - bounds.south) * 0.3; + const lngPad = (bounds.east - bounds.west) * 0.3; + const padded = { + south: bounds.south - latPad, + north: bounds.north + latPad, + west: bounds.west - lngPad, + east: bounds.east + lngPad + }; + + updatePhaseMsg('Loading individual samples...', 'loading'); + + try { + performance.mark('sp-s'); + const facetActive = hasFacetFilters(); + const facetSQL = facetActive ? facetFilterSQL() : ''; + let query; + if (facetActive) { + query = ` + SELECT l.pid, l.label, l.source, l.latitude, l.longitude, + l.place_name, l.result_time, f.material, f.context + FROM read_parquet('${lite_url}') l + JOIN read_parquet('${facets_url}') f ON l.pid = f.pid + WHERE l.latitude BETWEEN ${padded.south} AND ${padded.north} + AND l.longitude BETWEEN ${padded.west} AND ${padded.east} + ${sourceFilterSQL('l.source')} + ${facetSQL} + LIMIT ${POINT_BUDGET} + `; + } else { + query = ` + SELECT pid, label, source, latitude, longitude, + place_name, result_time + FROM read_parquet('${lite_url}') + WHERE latitude BETWEEN ${padded.south} AND ${padded.north} + AND longitude BETWEEN ${padded.west} AND ${padded.east} + ${sourceFilterSQL('source')} + LIMIT ${POINT_BUDGET} + `; + } + const data = await db.query(query); + performance.mark('sp-e'); + performance.measure('sp', 'sp-s', 'sp-e'); + const elapsed = performance.getEntriesByName('sp').pop().duration; + + // Stale guard: discard if a newer request was issued + if (myReqId !== requestId) { + console.log(`Discarding stale sample response (req ${myReqId}, current ${requestId})`); + return; + } + + // Cache the padded bounds + data + cachedBounds = padded; + cachedData = Array.from(data); + + renderSamplePoints(cachedData, bounds); + + updateStats('Samples', cachedData.length, cachedData.length, `${(elapsed/1000).toFixed(1)}s`, 'Samples in View', 'Samples in View'); + updatePhaseMsg(`${cachedData.length.toLocaleString()} individual samples. Click one for details.`, 'done'); + console.log(`Point mode: ${cachedData.length} samples in ${elapsed.toFixed(0)}ms`); + + } catch(err) { + if (myReqId !== requestId) return; + console.error("Viewport sample query failed:", err); + updatePhaseMsg('Sample query failed — try again.', 'loading'); + } } - return data; - } catch (error) { - if (statusDiv) { - statusDiv.textContent = `Error: ${error.message}`; - statusDiv.style.background = '#ffebee'; + // --- Render sample points on globe --- + function renderSamplePoints(data, bounds) { + viewer.samplePoints.removeAll(); + const scalar = new Cesium.NearFarScalar(1e2, 8, 2e5, 3); + + for (const row of data) { + const color = SOURCE_COLORS[row.source] || '#666'; + viewer.samplePoints.add({ + id: { + type: 'sample', + pid: row.pid, + label: row.label, + source: row.source, + lat: row.latitude, + lng: row.longitude, + place_name: row.place_name, + result_time: row.result_time + }, + position: Cesium.Cartesian3.fromDegrees(row.longitude, row.latitude, 0), + pixelSize: 6, + color: Cesium.Color.fromCssColorString(color).withAlpha(0.9), + scaleByDistance: scalar, + }); + } } - return []; - } -} -``` -```{ojs} -mutable clickedPointId = null -mutable clickedPointIndex = null -``` + // --- Mode transitions --- + function enterPointMode(pushHistory) { + mode = 'point'; + viewer._globeState.mode = 'point'; + viewer.h3Points.show = false; + viewer.samplePoints.show = true; + if (pushHistory !== false) history.pushState(null, '', buildHash(viewer)); + loadViewportSamples(); + console.log('Entered point mode'); + } -```{ojs} -// Cesium viewer setup -viewer = { - // Wait for Cesium to be available - await new Promise(resolve => { - if (typeof Cesium !== 'undefined') resolve(); - else { - const check = setInterval(() => { - if (typeof Cesium !== 'undefined') { - clearInterval(check); - resolve(); + function exitPointMode(pushHistory) { + mode = 'cluster'; + viewer._globeState.mode = 'cluster'; + viewer.samplePoints.show = false; + viewer.samplePoints.removeAll(); + viewer.h3Points.show = true; + if (pushHistory !== false) history.pushState(null, '', buildHash(viewer)); + cachedBounds = null; + cachedData = null; + + // Restore cluster stats with viewport count + const bounds = getViewportBounds(); + const inView = countInViewport(bounds); + const total = viewer._clusterTotal; + if (total) { + updateStats(`H3 Res${currentRes}`, `${inView.clusters.toLocaleString()} / ${total.clusters.toLocaleString()}`, inView.samples.toLocaleString(), '—', 'Clusters in View / Loaded', 'Samples in View'); + } else { + updateStats(`H3 Res${currentRes}`, viewer.h3Points.length, '—', '—', 'Clusters Loaded', 'Samples Loaded'); } - }, 100); + updatePhaseMsg(`${inView.clusters.toLocaleString()} clusters in view. Zoom closer for individual samples.`, 'done'); + console.log('Exited point mode'); } - }); - - Cesium.Ion.defaultAccessToken = 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJqdGkiOiIwNzk3NjkyMy1iNGI1LTRkN2UtODRiMy04OTYwYWE0N2M3ZTkiLCJpZCI6Njk1MTcsImlhdCI6MTYzMzU0MTQ3N30.e70dpNzOCDRLDGxRguQCC-tRzGzA-23Xgno5lNgCeB4'; - - const container = document.getElementById('cesiumContainer'); - - const v = new Cesium.Viewer(container, { - timeline: false, - animation: false, - baseLayerPicker: false, - fullscreenElement: container, - terrain: Cesium.Terrain.fromWorldTerrain() - }); - - // Point collection for efficient rendering - v.pointCollection = new Cesium.PointPrimitiveCollection(); - v.scene.primitives.add(v.pointCollection); - - // Click handler - const handler = new Cesium.ScreenSpaceEventHandler(v.scene.canvas); - handler.setInputAction((e) => { - const picked = v.scene.pick(e.position); - if (Cesium.defined(picked) && picked.primitive && picked.id !== undefined) { - mutable clickedPointId = picked.id.pid; - mutable clickedPointIndex = picked.id.index; + + // --- Results table refresh (below globe) --- + // Independent of globe mode — always shows top-N samples matching filters. + const RESULTS_TABLE_LIMIT = 200; + let tableReqId = 0; + async function refreshResultsTable() { + const myReq = ++tableReqId; + updateResultsTableMeta('Loading samples matching filters…', true); + try { + const facetActive = hasFacetFilters(); + const facetSQL = facetActive ? facetFilterSQL() : ''; + let query; + if (facetActive) { + query = ` + SELECT l.pid, l.label, l.source, l.latitude, l.longitude, l.place_name + FROM read_parquet('${lite_url}') l + JOIN read_parquet('${facets_url}') f ON l.pid = f.pid + WHERE 1=1 + ${sourceFilterSQL('l.source')} + ${facetSQL} + LIMIT ${RESULTS_TABLE_LIMIT} + `; + } else { + query = ` + SELECT pid, label, source, latitude, longitude, place_name + FROM read_parquet('${lite_url}') + WHERE 1=1 + ${sourceFilterSQL('source')} + LIMIT ${RESULTS_TABLE_LIMIT} + `; + } + const data = await db.query(query); + if (myReq !== tableReqId) return; // stale + renderResultsTable(data); + const note = data.length === RESULTS_TABLE_LIMIT + ? `Showing first ${RESULTS_TABLE_LIMIT} samples matching filters (more exist).` + : `Showing ${data.length.toLocaleString()} samples matching filters.`; + updateResultsTableMeta(note, false); + } catch (err) { + if (myReq !== tableReqId) return; + console.error('Results table query failed:', err); + updateResultsTableMeta('Failed to load results table.', false); + } } - }, Cesium.ScreenSpaceEventType.LEFT_CLICK); - return v; -} -``` + // --- Source filter change handler --- + const resUrls = { 4: h3_res4_url, 6: h3_res6_url, 8: h3_res8_url }; + document.getElementById('sourceFilter').addEventListener('change', async () => { + // Toggle visual state on labels + document.querySelectorAll('#sourceFilter .legend-item').forEach(li => { + const cb = li.querySelector('input'); + li.classList.toggle('disabled', !cb.checked); + }); + // Persist source filter in URL query string for bookmarkable links + // (e.g. ?sources=OPENCONTEXT). Camera state lives in the hash. + const active = getActiveSources(); + const params = new URLSearchParams(location.search); + if (active.length > 0 && active.length < 4) { + params.set('sources', active.join(',')); + } else { + params.delete('sources'); + } + const qs = params.toString(); + const newSearch = qs ? `?${qs}` : ''; + if (location.search !== newSearch) { + history.replaceState(null, '', `${location.pathname}${newSearch}${location.hash}`); + } + if (mode === 'cluster') { + loading = false; // allow loadRes to run (gen counter discards stale results) + await loadRes(currentRes, resUrls[currentRes]); + } else { + cachedBounds = null; // force re-query + await loadViewportSamples(); + } + refreshResultsTable(); + }); -```{ojs} -// Render points on globe -renderPoints = { - if (!viewer || sampleData.length === 0) return null; - - const statusDiv = document.getElementById('loading_status'); - if (statusDiv) { - statusDiv.style.display = 'block'; - statusDiv.textContent = 'Rendering points...'; - } - - // Clear existing points - viewer.pointCollection.removeAll(); - - const scalar = new Cesium.NearFarScalar(1.5e2, 3, 8.0e6, 0.5); - const CHUNK_SIZE = 500; - - for (let i = 0; i < sampleData.length; i += CHUNK_SIZE) { - const chunk = sampleData.slice(i, i + CHUNK_SIZE); - - for (let j = 0; j < chunk.length; j++) { - const row = chunk[j]; - viewer.pointCollection.add({ - id: { pid: row.pid, index: i + j, row: row }, - position: Cesium.Cartesian3.fromDegrees(row.longitude, row.latitude, 0), - pixelSize: 5, - color: getCesiumColor(row.source), - scaleByDistance: scalar, - }); + // --- Material/Context/Specimen filter change handler --- + const facetNote = document.getElementById('facetNote'); + function handleFacetFilterChange() { + const active = hasFacetFilters(); + if (facetNote) facetNote.style.display = (active && mode === 'cluster') ? 'block' : 'none'; + if (mode === 'point') { + cachedBounds = null; + loadViewportSamples(); + } + refreshResultsTable(); } + document.getElementById('materialFilterBody').addEventListener('change', handleFacetFilterChange); + document.getElementById('contextFilterBody').addEventListener('change', handleFacetFilterChange); + document.getElementById('objectTypeFilterBody').addEventListener('change', handleFacetFilterChange); + + // Initial table load + refreshResultsTable(); + + // --- Camera change handler --- + let timer = null; + viewer.camera.changed.addEventListener(() => { + if (timer) clearTimeout(timer); + timer = setTimeout(async () => { + const h = viewer.camera.positionCartographic.height; + + // Determine target mode with hysteresis + const targetMode = h < ENTER_POINT_ALT ? 'point' + : h > EXIT_POINT_ALT ? 'cluster' + : mode; + + if (targetMode === 'point' && mode !== 'point') { + // Make sure we're at res8 clusters before transitioning + if (currentRes !== 8 && !loading) { + await loadRes(8, h3_res8_url); + } + enterPointMode(); + } else if (targetMode === 'cluster' && mode !== 'cluster') { + exitPointMode(); + // Reload appropriate resolution + const target = h > 3000000 ? 4 : h > 300000 ? 6 : 8; + if (target !== currentRes && !loading) { + await loadRes(target, { 4: h3_res4_url, 6: h3_res6_url, 8: h3_res8_url }[target]); + } + } else if (targetMode === 'point') { + // Already in point mode — update viewport samples + loadViewportSamples(); + } else { + // Cluster mode — check if resolution should change + const target = h > 3000000 ? 4 : h > 300000 ? 6 : 8; + if (target !== currentRes && !loading) { + await loadRes(target, { 4: h3_res4_url, 6: h3_res6_url, 8: h3_res8_url }[target]); + } + } + + // Update viewport cluster count (cluster mode only; point mode already shows viewport count) + if (mode === 'cluster' && viewer._clusterData) { + const bounds = getViewportBounds(); + const inView = countInViewport(bounds); + const total = viewer._clusterTotal; + if (total) { + updateStats(`H3 Res${currentRes}`, `${inView.clusters.toLocaleString()} / ${total.clusters.toLocaleString()}`, inView.samples.toLocaleString(), null, 'Clusters in View / Loaded', 'Samples in View'); + } + } + + // Update URL hash (replaceState for continuous movement) + if (!viewer._suppressHashWrite) { + history.replaceState(null, '', buildHash(viewer)); + } + }, 600); + }); + viewer.camera.percentageChanged = 0.1; + + // --- Handle browser back/forward --- + window.addEventListener('hashchange', async () => { + const state = readHash(); + if (state.lat == null || state.lng == null) return; + + viewer._suppressHashWrite = true; + clearTimeout(viewer._suppressTimer); + viewer.camera.cancelFlight(); + viewer.camera.flyTo({ + destination: Cesium.Cartesian3.fromDegrees(state.lng, state.lat, state.alt || 20000000), + orientation: { + heading: Cesium.Math.toRadians(state.heading), + pitch: Cesium.Math.toRadians(state.pitch) + }, + duration: 1.5, + }); + + // After flight settles, force mode and clear suppress flag + viewer._suppressTimer = setTimeout(() => { + viewer._suppressHashWrite = false; + const s = readHash(); + if (s.mode === 'point' && mode !== 'point') enterPointMode(false); + else if (s.mode !== 'point' && mode === 'point') exitPointMode(false); + }, 2000); + + // Handle pid selection + if (state.pid) { + viewer._globeState.selectedPid = state.pid; + try { + const sample = await db.query(` + SELECT pid, label, source, latitude, longitude, place_name, result_time + FROM read_parquet('${lite_url}') + WHERE pid = '${state.pid.replace(/'/g, "''")}' + LIMIT 1 + `); + if (sample && sample.length > 0) { + const s = sample[0]; + updateSampleCard({ + pid: s.pid, label: s.label, source: s.source, + lat: s.latitude, lng: s.longitude, + place_name: s.place_name, result_time: s.result_time + }); + } + } catch(err) { + console.error("Hash pid query failed:", err); + } + } else { + viewer._globeState.selectedPid = null; + updateClusterCard(null); + } + }); - // Update progress - if (statusDiv && i % 5000 === 0) { - const pct = Math.round((i / sampleData.length) * 100); - statusDiv.textContent = `Rendering points... ${pct}%`; + // --- Share button --- + const shareBtn = document.getElementById('shareBtn'); + if (shareBtn) { + shareBtn.addEventListener('click', async () => { + history.replaceState(null, '', buildHash(viewer)); + try { + await navigator.clipboard.writeText(location.href); + const toast = document.getElementById('shareToast'); + if (toast) { + toast.style.opacity = '1'; + setTimeout(() => { toast.style.opacity = '0'; }, 2000); + } + } catch(err) { + prompt('Copy this link:', location.href); + } + }); } - // Yield to browser - if (i + CHUNK_SIZE < sampleData.length) { - await new Promise(resolve => setTimeout(resolve, 0)); + // --- Search handler --- + const searchBtn = document.getElementById('searchBtn'); + const searchInput = document.getElementById('sampleSearch'); + const searchResults = document.getElementById('searchResults'); + + async function doSearch() { + const term = searchInput.value.trim(); + if (!term || term.length < 2) { + searchResults.textContent = 'Type at least 2 characters'; + return; + } + searchResults.textContent = 'Searching...'; + try { + const escaped = term.replace(/'/g, "''"); + const results = await db.query(` + SELECT pid, label, source, latitude, longitude, place_name + FROM read_parquet('${lite_url}') + WHERE label ILIKE '%${escaped}%' + ${sourceFilterSQL('source')} + LIMIT 50 + `); + if (results.length === 0) { + searchResults.textContent = `No results for "${term}"`; + return; + } + searchResults.textContent = `${results.length}${results.length === 50 ? '+' : ''} results for "${term}"`; + + // Show results in the samples panel + const sampEl = document.getElementById('samplesSection'); + if (sampEl) { + let h = `

Search: "${term}" (${results.length})

`; + for (const s of results) { + const color = SOURCE_COLORS[s.source] || '#666'; + const name = SOURCE_NAMES[s.source] || s.source; + const sUrl = sourceUrl(s.pid); + h += `
+
+ ${sUrl ? `${s.label || s.pid}` : `${s.label || s.pid}`} + ${name} +
+
`; + } + sampEl.innerHTML = h; + + // Click search result → fly to it + sampEl.querySelectorAll('.sample-row[data-lat]').forEach(row => { + row.addEventListener('click', (e) => { + if (e.target.tagName === 'A') return; // let links work + const lat = parseFloat(row.dataset.lat); + const lng = parseFloat(row.dataset.lng); + const pid = row.dataset.pid; + if (!isNaN(lat) && !isNaN(lng)) { + viewer.camera.flyTo({ + destination: Cesium.Cartesian3.fromDegrees(lng, lat, 50000), + duration: 1.5 + }); + } + }); + }); + } + + // Fly to the first result + if (results[0].latitude && results[0].longitude) { + viewer.camera.flyTo({ + destination: Cesium.Cartesian3.fromDegrees(results[0].longitude, results[0].latitude, 200000), + duration: 1.5 + }); + } + } catch(err) { + console.error("Search failed:", err); + searchResults.textContent = `Search error: ${err.message}`; + } } - } - if (statusDiv) { - statusDiv.textContent = `Rendered ${sampleData.length.toLocaleString()} points`; - setTimeout(() => { statusDiv.style.display = 'none'; }, 2000); - } + if (searchBtn) searchBtn.addEventListener('click', doSearch); + if (searchInput) searchInput.addEventListener('keydown', (e) => { + if (e.key === 'Enter') doSearch(); + }); - return sampleData.length; -} -``` + // --- Deep-link: restore selection from initial hash --- + const ih = viewer._initialHash; + if (ih.pid) { + viewer._globeState.selectedPid = ih.pid; + try { + const sample = await db.query(` + SELECT pid, label, source, latitude, longitude, place_name, result_time + FROM read_parquet('${lite_url}') + WHERE pid = '${ih.pid.replace(/'/g, "''")}' + LIMIT 1 + `); + if (sample && sample.length > 0) { + const s = sample[0]; + updateSampleCard({ + pid: s.pid, label: s.label, source: s.source, + lat: s.latitude, lng: s.longitude, + place_name: s.place_name, result_time: s.result_time + }); + const detail = await db.query(` + SELECT description FROM read_parquet('${wide_url}') + WHERE pid = '${ih.pid.replace(/'/g, "''")}' + LIMIT 1 + `); + if (detail && detail.length > 0) updateSampleDetail(detail[0]); + else updateSampleDetail({ description: '' }); + } + } catch(err) { + console.error("Deep-link pid query failed:", err); + } + } -## Sample Card + // Enable hash writing now that everything is initialized + viewer._suppressHashWrite = false; -```{ojs} -// Get selected sample data -selectedSample = { - if (clickedPointIndex === null || clickedPointIndex < 0) return null; - if (clickedPointIndex >= sampleData.length) return null; - return sampleData[clickedPointIndex]; + return "active"; } ``` ```{ojs} -// v2: lazy description fetch — only hit the 278 MB wide parquet when a sample -// is actually clicked, rather than pulling description for every row eagerly. -lazyDescription = { - if (explorerVersion !== 'v2') return null; - if (!selectedSample?.pid) return null; - const pid = selectedSample.pid.replace(/'/g, "''"); - try { - const rows = await runQuery(` - SELECT description FROM read_parquet('${wide_url}') - WHERE pid = '${pid}' AND otype = 'MaterialSampleRecord' - LIMIT 1 - `); - return rows[0]?.description || ''; - } catch (e) { - console.warn('Lazy description fetch failed:', e); - return ''; - } -} -``` +//| echo: false +//| output: false + +// === Performance timing panel (opt-in: append ?perf=1 to URL) === +// v0: reads performance.mark/measure entries and renders a small fixed panel. +// Reports navigation→duckdb_init, navigation→viewer_init, phase 1 res4 load, +// and navigation→first paint. Also dumps to console.table for CI / Playwright. +perfPanel = { + if (!phase1) return; // wait for phase 1 to have run + + const params = new URLSearchParams(location.search); + const isOn = params.get('perf') === '1'; + if (!isOn) return; + + // Give first-paint a tick to fire, then collect + await new Promise(r => setTimeout(r, 100)); + + const origin = performance.timeOrigin; + const mark = (name) => { + const e = performance.getEntriesByName(name, 'mark').pop(); + return e ? e.startTime : null; + }; + const measure = (name) => { + const e = performance.getEntriesByName(name, 'measure').pop(); + return e ? e.duration : null; + }; + + // Paint timings from the browser (free, no instrumentation needed) + const paintEntries = performance.getEntriesByType('paint'); + const fcp = paintEntries.find(e => e.name === 'first-contentful-paint')?.startTime; + const fp = paintEntries.find(e => e.name === 'first-paint')?.startTime; + + const rows = [ + ['first-paint (browser)', fp], + ['first-contentful-paint', fcp], + ['duckdb init', measure('duckdb_init')], + ['viewer init', measure('viewer_init')], + ['nav → viewer ready', mark('viewer-init-end')], + ['phase 1 res4 (duration)', measure('p1')], + ['nav → phase 1 complete', mark('p1-end')], + ['nav → first globe frame', mark('first-globe-frame')], + ].filter(([, v]) => v != null); + + // Console table for CI / offline capture + console.table(Object.fromEntries(rows.map(([k, v]) => [k, `${v.toFixed(0)} ms`]))); + + // Render a small floating panel + const fmt = (ms) => ms == null ? '—' : ms >= 1000 ? `${(ms/1000).toFixed(2)} s` : `${ms.toFixed(0)} ms`; + const panel = document.createElement('div'); + panel.id = 'perfPanel'; + panel.style.cssText = ` + position: fixed; bottom: 12px; right: 12px; z-index: 9999; + background: rgba(0,0,0,0.82); color: #e8f5e9; padding: 10px 12px; + border-radius: 6px; font: 11px/1.4 ui-monospace, SFMono-Regular, monospace; + max-width: 320px; box-shadow: 0 2px 12px rgba(0,0,0,0.3); + `; + panel.innerHTML = ` +
+ ⏱ Perf timings + +
+ + ${rows.map(([label, v]) => ` + + + `).join('')} +
${label}${fmt(v)}
+
timeOrigin: ${new Date(origin).toISOString().split('T')[1].slice(0,12)}
+ `; + document.body.appendChild(panel); + panel.querySelector('#perfClose').onclick = () => panel.remove(); -```{ojs} -// Render sample card -sampleCard = { - if (!selectedSample) { - return html`
- Click a point on the globe to see details -
`; - } - - const s = selectedSample; - const sourceColor = SOURCE_COLORS[s.source] || SOURCE_COLORS.default; - - const label = s.label || 'No label'; - // v2: prefer the lazily-fetched description (from wide parquet on click); - // v1: the description is already in sampleData. - const description = (s.description || lazyDescription || '').trim(); - const truncDesc = description.length > 200 ? description.substring(0, 200) + '...' : description; - - let placeStr = ''; - if (s.place_name) { - if (Array.isArray(s.place_name)) { - placeStr = s.place_name.filter(p => p).join(' > '); - } else { - placeStr = String(s.place_name); - } - } - - const placeHtml = placeStr ? `
Place: ${placeStr.substring(0, 100)}
` : ''; - - const pidShort = s.pid ? (s.pid.length > 50 ? s.pid.substring(0, 50) + '...' : s.pid) : ''; - - return html` -
-
- ${s.source} -
-

${label}

-

- ${truncDesc || 'No description'} -

-
-
Location: ${s.latitude.toFixed(4)}, ${s.longitude.toFixed(4)}
- ${placeHtml} -
ID: ${pidShort}
-
-
- `; + return "shown"; } ``` ---- - -## Debug Info - -
-Current State & Query - -```{ojs} -html`
-State:
-  search: "${searchInput || ''}"
-  sources: ${JSON.stringify(Array.from(sourceCheckboxes || []))}
-  maxSamples: ${maxSamples}
-
-WHERE clause:
-${whereClause}
-
-Total matching: ${Number(totalCount).toLocaleString()}
-Loaded: ${sampleData.length.toLocaleString()}
-
` -``` - -
+## How It Works ---- +The globe loads progressively: a 580 KB pre-aggregated H3 res4 file paints the world in clusters within ~1 s, finer-grained res6 / res8 files swap in as you zoom, and only when you zoom past 120 km altitude does the explorer switch to individual sample points (capped at 5K, viewport-bounded). The results table below the globe queries the same 60 MB lite parquet for the first 200 samples that match your filters — independent of the camera. Filters compose: source filtering applies at every zoom level (works on H3 cluster files too), while material / sampled feature / specimen filters apply once you're at sample zoom or in the results table (the H3 tier files are not pre-filtered by those facets). - +| Phase | Data | Size | Points | +|-------|------|------|--------| +| **Instant** | H3 res4 | 580 KB | 38K clusters (continental) | +| **Zoom in** | H3 res6 | 1.6 MB | 112K clusters (city) | +| **Zoom more** | H3 res8 | 2.5 MB | 176K clusters (neighborhood) | +| **Zoom deep** | Map lite | 60 MB (range req.) | Up to 5K individual samples | +| **Click sample** | Full dataset | ~280 MB (range req.) | Full metadata for 1 sample | ---- +All queries run in your browser via DuckDB-WASM with HTTP range requests — only the bytes you need are transferred. No backend. - - - - - +- [Progressive Globe](/tutorials/progressive_globe.html) — Same globe rendering without the faceted filter UI +- [Deep-Dive Analysis](/tutorials/zenodo_isamples_analysis.html) — DuckDB-WASM SQL tutorial +- [Why H3?](/tutorials/why_h3.html) — Why hexagonal hierarchical spatial indexing is the right substrate From 598282cdcc61768c3cf8cd73c91db064daa489d3 Mon Sep 17 00:00:00 2001 From: Raymond Yee Date: Thu, 30 Apr 2026 17:22:04 -0700 Subject: [PATCH 2/4] explorer: retain prior filter UX (counts, prefLabels, slider, clear) Brings the previous explorer's filter UX back on top of the progressive_globe-based rewrite (#153): - Source legend now shows per-source counts loaded from facet_summaries (matches the (12,345)-style badges on the prior live page). - Material / Sampled Feature / Specimen Type checkboxes use SKOS prefLabels via vocab_labels.parquet (~60 KB lookup) with URI-tail fallback. Full URI is preserved as the checkbox value so the SQL filter matches sample_facets_v2's URI-string columns exactly. - Filter semantics flip back to the prior pattern: empty = no filter (show everything); selecting items = include only those. Reduces side-panel noise at startup (was rendering hundreds of pre-checked rows). - Max Samples slider (1K-100K, default 25K) drives both the globe's point-mode budget and the results-table query LIMIT. Live label updates while dragging; query refreshes on release (debounced 300 ms). - "Clear Filters" button next to Share resets to a clean state by reloading to the bare pathname (matches the prior ). - Results table is now scrollable (max-height 600 px) and capped at 200 rendered rows so a 100K-LIMIT query doesn't blow up DOM. Meta line reports `first N of M+` when the LIMIT is hit. Smoke test: 0 JS exceptions, 0 console errors, 0 network failures. Co-Authored-By: Claude Opus 4.7 (1M context) --- tutorials/isamples_explorer.qmd | 196 +++++++++++++++++++++++--------- 1 file changed, 145 insertions(+), 51 deletions(-) diff --git a/tutorials/isamples_explorer.qmd b/tutorials/isamples_explorer.qmd index 736560f..929fbf4 100644 --- a/tutorials/isamples_explorer.qmd +++ b/tutorials/isamples_explorer.qmd @@ -116,6 +116,12 @@ format: .filter-body label:hover { color: #1565c0; } /* Results table below the globe */ #resultsTableWrap { margin-top: 16px; } + #resultsTable { + max-height: 600px; + overflow-y: auto; + border: 1px solid #eee; + border-radius: 6px; + } .results-table { width: 100%; border-collapse: collapse; @@ -179,10 +185,10 @@ Circle size = log(sample count). Color = dominant data source.
- - - - + + + +
@@ -212,8 +218,15 @@ Specimen Type ▾ -
+
+ + +
+
+
@@ -255,6 +268,9 @@ wide_url = `${R2_BASE}/current/wide.parquet` // v2 carries object_type alongside material and context (URI-string columns). facets_url = `${R2_BASE}/isamples_202601_sample_facets_v2.parquet` facet_summaries_url = `${R2_BASE}/isamples_202601_facet_summaries.parquet` +// SKOS prefLabels for Material / Sampled Feature / Specimen Type URIs. +// ~60 KB lookup; falls back to URI tail if a URI isn't covered. +vocab_labels_url = `${R2_BASE}/vocab_labels.parquet` // Canonical palette — see issue #113. Path-relative so this works under // both isamples.org (custom domain at root) and project-pages fork @@ -292,44 +308,31 @@ function getCheckedValues(containerId) { return Array.from(checks).filter(c => c.checked).map(c => c.value); } +// Semantics (matches the prior explorer): empty = no filter, show all. +// Selecting one or more items = include only those. Reduces visual noise +// at startup (you don't see hundreds of pre-checked rows). function hasFacetFilters() { - const mat = getCheckedValues('materialFilterBody'); - const ctx = getCheckedValues('contextFilterBody'); - const ot = getCheckedValues('objectTypeFilterBody'); - const matTotal = document.querySelectorAll('#materialFilterBody input[type="checkbox"]').length; - const ctxTotal = document.querySelectorAll('#contextFilterBody input[type="checkbox"]').length; - const otTotal = document.querySelectorAll('#objectTypeFilterBody input[type="checkbox"]').length; - // Active if some (but not all) are checked, or if none are checked - return (mat.length > 0 && mat.length < matTotal) - || (ctx.length > 0 && ctx.length < ctxTotal) - || (ot.length > 0 && ot.length < otTotal); + return getCheckedValues('materialFilterBody').length > 0 + || getCheckedValues('contextFilterBody').length > 0 + || getCheckedValues('objectTypeFilterBody').length > 0; } function facetFilterSQL() { let sql = ''; const mat = getCheckedValues('materialFilterBody'); - const matTotal = document.querySelectorAll('#materialFilterBody input[type="checkbox"]').length; - if (mat.length > 0 && mat.length < matTotal) { - const list = mat.map(s => `'${s}'`).join(','); + if (mat.length > 0) { + const list = mat.map(s => `'${s.replace(/'/g, "''")}'`).join(','); sql += ` AND f.material IN (${list})`; - } else if (mat.length === 0 && matTotal > 0) { - sql += ' AND 1=0'; } const ctx = getCheckedValues('contextFilterBody'); - const ctxTotal = document.querySelectorAll('#contextFilterBody input[type="checkbox"]').length; - if (ctx.length > 0 && ctx.length < ctxTotal) { - const list = ctx.map(s => `'${s}'`).join(','); + if (ctx.length > 0) { + const list = ctx.map(s => `'${s.replace(/'/g, "''")}'`).join(','); sql += ` AND f.context IN (${list})`; - } else if (ctx.length === 0 && ctxTotal > 0) { - sql += ' AND 1=0'; } const ot = getCheckedValues('objectTypeFilterBody'); - const otTotal = document.querySelectorAll('#objectTypeFilterBody input[type="checkbox"]').length; - if (ot.length > 0 && ot.length < otTotal) { - const list = ot.map(s => `'${s}'`).join(','); + if (ot.length > 0) { + const list = ot.map(s => `'${s.replace(/'/g, "''")}'`).join(','); sql += ` AND f.object_type IN (${list})`; - } else if (ot.length === 0 && otTotal > 0) { - sql += ' AND 1=0'; } return sql; } @@ -782,9 +785,38 @@ phase1 = { //| echo: false //| output: false -// === Load facet summaries and populate filter checkboxes === +// === Load facet summaries + SKOS prefLabels, populate filter checkboxes === +// +// Checkbox semantics: start UNCHECKED (no filter; show everything). User +// checks items to *include only those*. Empty = no filter. This matches the +// prior explorer's UX and keeps the side panel compact at startup. +// +// Checkbox value = full URI (matches the URI strings stored in +// sample_facets_v2.parquet's material / context / object_type columns). +// Display label = SKOS prefLabel (en) when available, URI tail otherwise. facetFilters = { if (!phase1) return; + + // Tiny URI → prefLabel lookup. ~60 KB. Best-effort: fallback to URI tail. + const vocabMap = new Map(); + try { + const vocab = await db.query( + `SELECT uri, pref_label FROM read_parquet('${vocab_labels_url}') WHERE lang = 'en'` + ); + for (const r of vocab) vocabMap.set(r.uri, r.pref_label); + } catch (err) { + console.warn("vocab_labels load failed; falling back to URI tails:", err); + } + const prettyLabel = (uri) => { + if (uri == null) return ""; + const hit = vocabMap.get(uri); + if (hit) return hit; + const s = String(uri); + if (!/^https?:\/\//.test(s)) return s; + const parts = s.replace(/[#?].*$/, "").split("/").filter(Boolean); + return parts.length ? parts[parts.length - 1] : s; + }; + try { const summaries = await db.query(` SELECT facet_type, facet_value, count @@ -792,29 +824,45 @@ facetFilters = { ORDER BY facet_type, count DESC `); - const grouped = { material: [], context: [], object_type: [] }; + const grouped = { source: [], material: [], context: [], object_type: [] }; for (const row of summaries) { if (grouped[row.facet_type]) { - // Extract short label from URI - const shortLabel = row.facet_value.split('/').pop() || row.facet_value; - grouped[row.facet_type].push({ value: shortLabel, fullUri: row.facet_value, count: row.count }); + grouped[row.facet_type].push({ + uri: row.facet_value, + label: prettyLabel(row.facet_value), + count: row.count + }); } } + // Update source count badges in the (hardcoded) legend. + const srcCountMap = new Map(grouped.source.map(s => [s.uri, s.count])); + document.querySelectorAll('#sourceFilter .src-count').forEach(el => { + const key = el.getAttribute('data-source'); + const c = srcCountMap.get(key); + if (c != null) el.textContent = `(${Number(c).toLocaleString()})`; + }); + + const escAttr = (s) => String(s).replace(/&/g, '&').replace(/"/g, '"').replace(/ String(s).replace(/&/g, '&').replace(/ { const body = document.getElementById(bodyId); - if (body && items.length > 0) { - body.innerHTML = items.map(it => - `` - ).join(''); + if (!body) return; + if (items.length === 0) { + body.innerHTML = 'No values'; + return; } + body.innerHTML = items.map(it => + `` + ).join(''); }; renderFilter('materialFilterBody', grouped.material); renderFilter('contextFilterBody', grouped.context); renderFilter('objectTypeFilterBody', grouped.object_type); - console.log(`Facet filters loaded: ${grouped.material.length} materials, ${grouped.context.length} contexts, ${grouped.object_type.length} object types`); + console.log(`Facet filters loaded: ${grouped.material.length} materials, ${grouped.context.length} contexts, ${grouped.object_type.length} object types (vocab labels: ${vocabMap.size})`); } catch(err) { console.warn("Facet summaries failed to load:", err); } @@ -841,7 +889,12 @@ zoomWatcher = { // Hysteresis thresholds to avoid flicker const ENTER_POINT_ALT = 120000; // 120 km → enter point mode const EXIT_POINT_ALT = 180000; // 180 km → exit point mode - const POINT_BUDGET = 5000; + // Sample budget is driven by the Max Samples slider; reads at query time. + const getMaxSamples = () => { + const el = document.getElementById('maxSamples'); + const v = el ? parseInt(el.value, 10) : NaN; + return Number.isFinite(v) && v > 0 ? v : 5000; + }; // Viewport cache: avoid re-querying same area let cachedBounds = null; // { south, north, west, east } @@ -980,7 +1033,7 @@ zoomWatcher = { AND l.longitude BETWEEN ${padded.west} AND ${padded.east} ${sourceFilterSQL('l.source')} ${facetSQL} - LIMIT ${POINT_BUDGET} + LIMIT ${getMaxSamples()} `; } else { query = ` @@ -990,7 +1043,7 @@ zoomWatcher = { WHERE latitude BETWEEN ${padded.south} AND ${padded.north} AND longitude BETWEEN ${padded.west} AND ${padded.east} ${sourceFilterSQL('source')} - LIMIT ${POINT_BUDGET} + LIMIT ${getMaxSamples()} `; } const data = await db.query(query); @@ -1082,11 +1135,16 @@ zoomWatcher = { } // --- Results table refresh (below globe) --- - // Independent of globe mode — always shows top-N samples matching filters. - const RESULTS_TABLE_LIMIT = 200; + // Independent of globe mode — always shows up to maxSamples rows matching + // filters. Cap rendered rows at TABLE_RENDER_CAP to keep DOM tractable + // (the slider can go up to 100K which is fine for the globe but jank for + // a flat HTML table). + const TABLE_RENDER_CAP = 200; let tableReqId = 0; async function refreshResultsTable() { const myReq = ++tableReqId; + const limit = getMaxSamples(); + const renderLimit = Math.min(limit, TABLE_RENDER_CAP); updateResultsTableMeta('Loading samples matching filters…', true); try { const facetActive = hasFacetFilters(); @@ -1100,7 +1158,7 @@ zoomWatcher = { WHERE 1=1 ${sourceFilterSQL('l.source')} ${facetSQL} - LIMIT ${RESULTS_TABLE_LIMIT} + LIMIT ${limit} `; } else { query = ` @@ -1108,15 +1166,21 @@ zoomWatcher = { FROM read_parquet('${lite_url}') WHERE 1=1 ${sourceFilterSQL('source')} - LIMIT ${RESULTS_TABLE_LIMIT} + LIMIT ${limit} `; } const data = await db.query(query); if (myReq !== tableReqId) return; // stale - renderResultsTable(data); - const note = data.length === RESULTS_TABLE_LIMIT - ? `Showing first ${RESULTS_TABLE_LIMIT} samples matching filters (more exist).` - : `Showing ${data.length.toLocaleString()} samples matching filters.`; + const visible = data.slice(0, renderLimit); + renderResultsTable(visible); + let note; + if (data.length === limit) { + note = `Showing first ${visible.length.toLocaleString()} of ${limit.toLocaleString()}+ samples matching filters.`; + } else if (visible.length < data.length) { + note = `Showing first ${visible.length.toLocaleString()} of ${data.length.toLocaleString()} samples matching filters.`; + } else { + note = `Showing ${data.length.toLocaleString()} samples matching filters.`; + } updateResultsTableMeta(note, false); } catch (err) { if (myReq !== tableReqId) return; @@ -1172,6 +1236,36 @@ zoomWatcher = { document.getElementById('contextFilterBody').addEventListener('change', handleFacetFilterChange); document.getElementById('objectTypeFilterBody').addEventListener('change', handleFacetFilterChange); + // --- Max Samples slider --- + const maxSamplesEl = document.getElementById('maxSamples'); + const maxSamplesValueEl = document.getElementById('maxSamplesValue'); + if (maxSamplesEl && maxSamplesValueEl) { + // Live label while dragging, debounced query on release. + let sliderDebounce = null; + maxSamplesEl.addEventListener('input', () => { + const v = parseInt(maxSamplesEl.value, 10); + maxSamplesValueEl.textContent = Number(v).toLocaleString(); + clearTimeout(sliderDebounce); + sliderDebounce = setTimeout(() => { + if (mode === 'point') { + cachedBounds = null; + loadViewportSamples(); + } + refreshResultsTable(); + }, 300); + }); + } + + // --- Clear Filters button --- + const clearBtn = document.getElementById('clearFiltersBtn'); + if (clearBtn) { + clearBtn.addEventListener('click', () => { + // Strip query string and hash, reload to a clean state. Simplest + // and matches the prior explorer's "" behavior. + location.href = location.pathname; + }); + } + // Initial table load refreshResultsTable(); From a9407e3c4dcf2fe8780f33df2ffef443dcfb49fe Mon Sep 17 00:00:00 2001 From: Raymond Yee Date: Thu, 30 Apr 2026 17:42:58 -0700 Subject: [PATCH 3/4] =?UTF-8?q?explorer:=20address=20Codex=20review=20?= =?UTF-8?q?=E2=80=94=20search=20composes=20facets;=20cluster=20docs=20hone?= =?UTF-8?q?st?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit P2 fixes from Codex review of #153: 1. doSearch() now composes facet filters (Material / Sampled Feature / Specimen Type) via JOIN sample_facets_v2 on pid, mirroring the pattern used by loadViewportSamples and refreshResultsTable. Was only honoring text + source filter, so search could surface and fly to samples outside the active facet selection — inconsistent with the rest of the explorer. 2. "How It Works" prose no longer claims source filtering "applies at every zoom level." Now states explicitly that at cluster zoom the H3 tier files carry one row per (h3_cell, dominant_source), so source filtering keeps cells whose *dominant* source matches; mixed cells where the unchecked source isn't dominant are still drawn (and contribute their full sample_count to the legend). Per-source filtering kicks in fully at sample zoom and in the table/search. Per-source-per-cell H3 builds would fix this properly but require a different parquet build — out of scope. 3. Updated the data-tier table to reflect Max Samples slider (default 25K, max 100K) instead of the stale 5K hardcoded value. Smoke test still PASS (0 exceptions, 0 errors, 0 network fails). Co-Authored-By: Claude Opus 4.7 (1M context) --- tutorials/isamples_explorer.qmd | 23 +++++++++++++++++++---- 1 file changed, 19 insertions(+), 4 deletions(-) diff --git a/tutorials/isamples_explorer.qmd b/tutorials/isamples_explorer.qmd index 929fbf4..382752d 100644 --- a/tutorials/isamples_explorer.qmd +++ b/tutorials/isamples_explorer.qmd @@ -1407,13 +1407,28 @@ zoomWatcher = { searchResults.textContent = 'Searching...'; try { const escaped = term.replace(/'/g, "''"); - const results = await db.query(` + // Compose with facet filters so search honors the same Material / + // Sampled Feature / Specimen Type selections that the table and + // point-mode globe use. Without this, search would surface (and + // fly to) samples outside the active filters. + const facetActive = hasFacetFilters(); + const facetSQL = facetActive ? facetFilterSQL() : ''; + const query = facetActive ? ` + SELECT l.pid, l.label, l.source, l.latitude, l.longitude, l.place_name + FROM read_parquet('${lite_url}') l + JOIN read_parquet('${facets_url}') f ON l.pid = f.pid + WHERE l.label ILIKE '%${escaped}%' + ${sourceFilterSQL('l.source')} + ${facetSQL} + LIMIT 50 + ` : ` SELECT pid, label, source, latitude, longitude, place_name FROM read_parquet('${lite_url}') WHERE label ILIKE '%${escaped}%' ${sourceFilterSQL('source')} LIMIT 50 - `); + `; + const results = await db.query(query); if (results.length === 0) { searchResults.textContent = `No results for "${term}"`; return; @@ -1589,14 +1604,14 @@ perfPanel = { ## How It Works -The globe loads progressively: a 580 KB pre-aggregated H3 res4 file paints the world in clusters within ~1 s, finer-grained res6 / res8 files swap in as you zoom, and only when you zoom past 120 km altitude does the explorer switch to individual sample points (capped at 5K, viewport-bounded). The results table below the globe queries the same 60 MB lite parquet for the first 200 samples that match your filters — independent of the camera. Filters compose: source filtering applies at every zoom level (works on H3 cluster files too), while material / sampled feature / specimen filters apply once you're at sample zoom or in the results table (the H3 tier files are not pre-filtered by those facets). +The globe loads progressively: a 580 KB pre-aggregated H3 res4 file paints the world in clusters within ~1 s, finer-grained res6 / res8 files swap in as you zoom, and only when you zoom past 120 km altitude does the explorer switch to individual sample points (capped at the Max Samples slider, viewport-bounded). The results table below the globe queries the 60 MB lite parquet for samples matching your filters — independent of the camera. Filters compose at sample zoom (and in the results table and search): source + material + sampled feature + specimen type all apply via a join on the facets parquet. **At cluster zoom**, the H3 tier files only carry one row per `(h3_cell, dominant_source)`, so source filtering keeps cells whose *dominant* source matches — a cell where OpenContext is present but isn't the majority will be hidden when you uncheck OpenContext. Material / sampled feature / specimen filters do not apply at cluster zoom at all (the tier files aren't pre-filtered by those facets). | Phase | Data | Size | Points | |-------|------|------|--------| | **Instant** | H3 res4 | 580 KB | 38K clusters (continental) | | **Zoom in** | H3 res6 | 1.6 MB | 112K clusters (city) | | **Zoom more** | H3 res8 | 2.5 MB | 176K clusters (neighborhood) | -| **Zoom deep** | Map lite | 60 MB (range req.) | Up to 5K individual samples | +| **Zoom deep** | Map lite | 60 MB (range req.) | Up to Max Samples slider value (default 25K, max 100K) individual samples in viewport | | **Click sample** | Full dataset | ~280 MB (range req.) | Full metadata for 1 sample | All queries run in your browser via DuckDB-WASM with HTTP range requests — only the bytes you need are transferred. No backend. From 09f33a626ac93c0baa6abd04041ad48b1dd3ab47 Mon Sep 17 00:00:00 2001 From: Raymond Yee Date: Thu, 30 Apr 2026 17:52:40 -0700 Subject: [PATCH 4/4] explorer: correct OpenContext example in cluster-filter caveat MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The cluster H3 filter is `WHERE dominant_source IN (checked)`, so "OpenContext present but not dominant" does not keep a cell visible when OpenContext is unchecked — the cell's dominant source is what gets matched. Reworked the prose to spell out both consequences: (1) cell whose dominant is X disappears entirely when X unchecked, even if it contains samples from still-checked sources; (2) cell whose dominant is X stays with full sample_count when X checked, even if some samples in it are from unchecked sources. Codex P3. Co-Authored-By: Claude Opus 4.7 (1M context) --- tutorials/isamples_explorer.qmd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tutorials/isamples_explorer.qmd b/tutorials/isamples_explorer.qmd index 382752d..a9912c1 100644 --- a/tutorials/isamples_explorer.qmd +++ b/tutorials/isamples_explorer.qmd @@ -1604,7 +1604,7 @@ perfPanel = { ## How It Works -The globe loads progressively: a 580 KB pre-aggregated H3 res4 file paints the world in clusters within ~1 s, finer-grained res6 / res8 files swap in as you zoom, and only when you zoom past 120 km altitude does the explorer switch to individual sample points (capped at the Max Samples slider, viewport-bounded). The results table below the globe queries the 60 MB lite parquet for samples matching your filters — independent of the camera. Filters compose at sample zoom (and in the results table and search): source + material + sampled feature + specimen type all apply via a join on the facets parquet. **At cluster zoom**, the H3 tier files only carry one row per `(h3_cell, dominant_source)`, so source filtering keeps cells whose *dominant* source matches — a cell where OpenContext is present but isn't the majority will be hidden when you uncheck OpenContext. Material / sampled feature / specimen filters do not apply at cluster zoom at all (the tier files aren't pre-filtered by those facets). +The globe loads progressively: a 580 KB pre-aggregated H3 res4 file paints the world in clusters within ~1 s, finer-grained res6 / res8 files swap in as you zoom, and only when you zoom past 120 km altitude does the explorer switch to individual sample points (capped at the Max Samples slider, viewport-bounded). The results table below the globe queries the 60 MB lite parquet for samples matching your filters — independent of the camera. Filters compose at sample zoom (and in the results table and search): source + material + sampled feature + specimen type all apply via a join on the facets parquet. **At cluster zoom**, the H3 tier files only carry one row per `(h3_cell, dominant_source)`, so source filtering matches on each cell's *dominant* source — two consequences worth knowing: (1) a cell whose dominant source is OpenContext disappears entirely when you uncheck OpenContext, even if some of its samples are from sources that remain checked; (2) a cell whose dominant source is OpenContext stays visible with its full `sample_count` when OpenContext is checked, even though some samples in it may be from sources you unchecked. Material / sampled feature / specimen filters don't apply at cluster zoom at all (the tier files aren't pre-filtered by those facets) — they kick in once you zoom into sample mode, in the results table, and in search. | Phase | Data | Size | Points | |-------|------|------|--------|