viz smart: DataTables data viewer drawer with Explore/Preview link (#4283) - #4284
Merged
Conversation
…4283) Add an interactive data viewer to `viz smart` HTML dashboards: next to the rowcount in the metadata frontmatter, an "(Explore)" link (all rows embedded) or "(Preview)" link (first N rows) opens the underlying table in a bottom drawer, mirroring the Data Dictionary drawer mechanics. - New `--preview-threshold <n>` flag (default 50,000; 0 disables): at or under the threshold every row embeds and the link reads "(Explore)"; above it only the first <n> rows embed and the link reads "(Preview)". - Vendored DataTables 3.0.0 + DateTime 2.0.0 + SearchBuilder 2.0.0 (vanilla, no jQuery; MIT) in src/cmd/assets/, embedded gzip+base64 by default and inflated lazily on first drawer open (zero page-load cost); plain inline under QSV_VIZ_NO_COMPRESS; version-pinned cdn.datatables.net tags with Subresource Integrity under QSV_VIZ_CDN (unit test recomputes sha384 of the vendored files against the pinned hashes, like plotly's). - DataTable has global search, a per-column filter input header row, and a SearchBuilder whose per-column conditions derive from the stats cache: Integer/Float -> num, Date/DateTime -> date, else string - so numeric columns get numeric conditions, dates get Before/After/Between, and low-cardinality categoricals get auto-populated value dropdowns without client-side type sniffing on a partial preview. - Rows embed as a JSON array-of-arrays payload (gzip+base64 when >= 4 KiB and compression is on, else plain with &/</> \u-escaped so no cell can smuggle a </script>); cells render via DataTable.render.text() so cell values are data, not markup. - Bottom drawer coexists with the dictionary side drawer (different axes, z-index 1090 vs 1100), pushes the fixed logo/theme widgets up, closes on Esc, and dispatches a resize so plotly re-fits. DataTables' native dark palette follows the qsv theme toggle by mirroring body.qsv-dark onto html.dark. - HTML-only: image exports never pay the extra data pass. - gen_gallery.py caps gallery smart runs at --preview-threshold 500 (its payloads are plain under QSV_VIZ_NO_COMPRESS; the 50k default would have added ~6MB to smart_nyc311.html alone - now +317KB, ~757KB across the 13 regenerated pages). - 2 unit + 9 integration tests; 3 existing panel-scoped tests now pin --preview-threshold 0 (the viewer legitimately embeds all raw columns, including panel-excluded ones), and the metadata test tracks the new Rows cell markup. Full viz suite: 363 passed. Closes #4283 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Up to standards ✅🟢 Issues
|
qsv --update-mcp-skills picks up the new flag in qsv-viz.json, plus previously-unregenerated accurate drift from the funnel/pipeline work (#4281/#4282): the funnel subcommand in the viz enum and the pipeline relationship docs in the viz + describegpt skills. Skills CHANGELOG Unreleased note added; skill count unchanged at 55. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Six UX tweaks to the DataTables data viewer drawer: - Stacking: the data drawer now paints ABOVE the dictionary drawer when both are open (z-index 1120 vs 1100). - SearchBuilder rides in a Buttons popover: a "Filter (n)" button on the same controls row as the page-length selector and the global search box (language.searchBuilder.button + layout.topStart), replacing the permanent top pane and its whitespace. Esc with the popover open closes only the popover; a bare Esc closes the drawer. - Focus management: opening the drawer moves focus into it (tabindex="-1"); closing returns focus to the Explore/Preview link in the metadata table at the top of the dashboard. - Responsive extension replaces scrollX: columns that do not fit collapse into an expandable child-row control per row (nyc311's 41 columns show 7 + expander); the per-column filter input row tracks Responsive's column visibility via the responsive-resize event. - Resizable drawer: a grip bar on the top edge drags the height between 20vh and 90vh via the --qsv-data-h CSS var, so the body margin and the fixed logo/theme widgets follow; releasing re-fits plotly and re-balances the table columns. - Fixed header/footer: only the table's own layout row scrolls (flex column + overflow:auto) with a sticky two-row thead (title row at top:0, filter row offset by the measured title-row height), keeping the controls row above and the info/paging bar below always visible. The FixedHeader extension was deliberately NOT used - it tracks window scroll only and is inert inside the fixed-position drawer; native sticky delivers the same UX and stays correct under drawer resize. Vendored bundle regenerated with Buttons 4.0.0 + Responsive 4.0.0 added (combo dt-3.0.0/b-4.0.0/date-2.0.0/r-4.0.0/sb-2.0.0; new SRI hashes; unit-test code map extended). Gallery regenerated. Full viz suite: 363 passed. Browser-verified: popover + condition count, responsive collapse + filter-row sync, sticky scrolling, grip resize, focus round-trip, both drawers stacked, light/dark, console clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…port (#4283) roborev #3864 (Low): the grip persists --qsv-data-h as absolute px, so a later window resize or device rotation could leave the drawer outside the intended 20vh-90vh bounds. All consumers (drawer height, body margin, fixed logo/theme offsets) now read one body-scoped `--qsv-data-h-eff: clamp(20vh, var(--qsv-data-h, min(55vh, 560px)), 90vh)` so the effective height tracks the CURRENT viewport and the offsets can never disagree. Browser-verified both bounds; gallery regenerated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…4283) In the gallery, smart dashboards are embedded in auto-sized iframes that never scroll themselves (the parent page does), so position:fixed pins the data drawer to the bottom of the FULL iframe height - often far below the visible viewport. Clicking Explore/Preview appeared to do nothing, and scrollIntoView cannot fix it: it is a no-op on fixed-position elements. New revealBottom()/revealTop() helpers scroll the PARENT window directly via window.frameElement geometry: opening aligns the iframe's bottom edge (where the drawer lives) with the parent viewport (re-aligned once more after the open transition + the gallery's iframe auto-grow settle); closing scrolls back to this dashboard's top where focus lands on the Explore/Preview link. Same-origin only - frameElement access throws in cross-origin/file:// embeddings and we quietly skip, which is the prior behavior. Standalone pages never scroll (self === top guard). Browser-verified in the served gallery: open scrolls the drawer fully into view, close returns to the dashboard top; standalone pages unchanged. Gallery regenerated; full viz suite 363 passed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…#4283) The frameElement-based reveal only worked for same-origin embeddings; a file://-opened gallery has opaque frames where every direct parent-window access throws, so clicking Explore/Preview still appeared to do nothing — the same constraint that made the gallery's iframe auto-sizing use postMessage in the first place. The reveal now posts {qsvVizReveal:"bottom"|"top"} to the parent (works cross-origin), with the direct frameElement scroll kept as a fallback for same-origin embedders without the listener. Gallery side: the resize listener honors the reveal messages (matching the sender iframe by e.source, like height reports); the jump stabilizer treats a reveal as reader takeover so it can no longer snap the view back and undo the reveal after a TOC jump; and pointerdown joins the takeover-forwarding events, since a click inside a dashboard is as much reader engagement as a scroll. Browser-verified end-to-end in the served gallery, including the adversarial flow (hash-jump arms the stabilizer, then a real click on Explore): open reveals the drawer, close returns to the dashboard top. Message path exercised in isolation (posted from the iframe context). Gallery regenerated; full viz suite 363 passed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
) roborev #3868 (Medium): the 450ms settle re-align in show() fired even if the drawer was closed within the window — over the postMessage path this scrolled the parent back to the iframe bottom right after qsvCloseData() had revealed the dashboard top. The delayed callback now re-checks that the drawer still carries the `open` class first. Gallery regenerated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The gallery's height reporter posted documentElement.scrollHeight, which is max(content, viewport) — once the drawer's reserved body margin had grown an iframe, scrollHeight could never report smaller than the iframe's own height, so closing the drawer left the reserved space as permanent whitespace (a one-way ratchet). The reporter now posts a content-derived height (body border-box + body vertical margins, which is exactly what the drawer's reserved space rides on; fixed-position chrome deliberately excluded), so the iframe grows on open AND shrinks back on close, and the report can never echo the viewport back into itself. Also: the drawer's post-open re-align is now a short series (450ms / 1.2s / 2.2s, still gated on the drawer being open) — on a still-settling gallery, other iframes growing above the target cancel an in-flight smooth scroll (the same churn the jump stabilizer fights), and a single re-align could land before the churn ended, leaving the drawer opened but out of view. Once the drawer region is visible the parent's reveal handler is a no-op, so the series converges and never fights the reader. Browser-verified on a fresh gallery load: click Explore -> iframe grows 2931->3663 and the drawer lands exactly at the viewport bottom; close -> iframe shrinks back to 2931 with the dashboard top in view. Full viz suite 363 passed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ant (#4283) roborev #3870 (Medium): the 450ms/1.2s/2.2s reveal re-align series only checked that the drawer was still open, so a reader who scrolled back up to inspect the dashboard (drawer left open) was yanked back down by a pending re-align. A revealSeq counter now invalidates the series on any reader input (wheel/touchstart/keydown/pointerdown, capture+passive); each delayed callback requires the drawer open AND the sequence unchanged. The arming click's own pointerdown fires before show() captures its sequence number, so it never cancels the series it starts. Verifying this surfaced the root cause of the reveal flakiness seen across sessions: the gallery page sets scroll-behavior:smooth, and on an iframe-heavy load a smooth scrollTo animation can be canceled at its first frame by concurrent layout work and never arrive — programmatic scrolls appeared to no-op while user scrolling worked. The jump stabilizer's snap() documents the identical lesson ("must be instant"). All reveal scrolls (viz.rs same-origin fallbacks + the gallery reveal listener) now use behavior:"instant"; "auto" would inherit the page's smooth CSS. Browser-verified: fresh-load click reveals deterministically in <800ms; wheel-up during the reveal window is respected (no yank-back for 3s+); close still shrinks the iframe and returns to the dashboard top. Full viz suite 363 passed; gallery regenerated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
jqnatividad
added a commit
that referenced
this pull request
Jul 27, 2026
Follow-up to #4284: the 4 LLM-gated PREGENERATED gallery pages received the same refreshed resize reporter (content-derived height + pointerdown takeover forwarding) during the gallery regens but were not staged with the rest. One-line swap per page; all 17 gallery dashboards now carry the identical reporter. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Jul 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements #4283: browse the underlying table from a
viz smartdashboard.What
Next to the rowcount in the metadata frontmatter, a link opens the underlying data in a bottom drawer (same drawer mechanics as the Data Dictionary side drawer) holding a DataTables 3 table:
--preview-thresholdrows (default 50,000) and ALL rows are embedded; "(Preview)" above it, embedding only the first N rows (drawer bar says "first N of M rows (preview)").Integer/Float→num,Date/DateTime→date, elsestring— so numeric columns get numeric conditions, dates get Before/After/Between, and low-cardinality categoricals get auto-populated value dropdowns. Whole-dataset stats beat client-side type sniffing on a partial preview.--preview-threshold 0disables the viewer entirely (no link, no payload, no data pass). HTML-only: image exports are untouched.How
src/cmd/assets/, riding the existing embed infrastructure:__qsvGunzipmachinery — zero page-load cost, and the library never executes unless the drawer is used;QSV_VIZ_NO_COMPRESS=1: plain inline (fully readable output, pre-DecompressionStream browsers);QSV_VIZ_CDN=1: version-pinnedcdn.datatables.nettags with Subresource Integrity — a unit test recomputes sha384 of the vendored files against the pinned hashes (the CDN serves the identical bytes), mirroring the plotly SRI discipline.&/</>are\u-escaped so no cell can smuggle a</script>, and cells render throughDataTable.render.text()so cell values are data, not markup (caught in-browser: the default HTML rendering executed markup found in CSV cells).resizeso plotly re-fits. DataTables' native dark palette follows the qsv theme toggle by mirroringbody.qsv-darkontohtml.dark(nothing else keys on it).Testing
--no-headerslabels, CDN SRI tags).--preview-threshold 0since the viewer legitimately embeds all raw columns; the metadata test tracks the new Rows cell markup.deferRender, SearchBuilder value dropdowns populate on categorical columns, both drawers open together on the nyc311 gallery page, no console errors.Gallery
Regenerated.
gen_gallery.pycaps gallery smart runs at--preview-threshold 500— gallery payloads are plain (QSV_VIZ_NO_COMPRESSfor fig-scraping), so the 50k default would have added ~6MB tosmart_nyc311.htmlalone; the cap keeps it to +317KB (~757KB across 13 pages) while still demoing both Explore and Preview modes. The 4 LLM-gatedPREGENERATEDpages pick up the viewer on their nextQSV_VIZ_REGEN_LLM=1refresh.Closes #4283
🤖 Generated with Claude Code
https://claude.ai/code/session_01TYN1JzoqK3wMQjgnigXeBn