Skip to content

feat: streaming Export — stream full query result to disk, uncapped (#87)#103

Merged
BorisTyshkevich merged 1 commit into
mainfrom
feat/export-stream-87
Jul 1, 2026
Merged

feat: streaming Export — stream full query result to disk, uncapped (#87)#103
BorisTyshkevich merged 1 commit into
mainfrom
feat/export-stream-87

Conversation

@BorisTyshkevich

Copy link
Copy Markdown
Collaborator

What & why

Adds an Export button to the editor toolbar (next to Share) that runs the
current editor query uncapped and streams the response straight to a
user-chosen file via the File System Access API, bypassing the result grid
entirely — memory stays flat regardless of result size. Replaces the old
result-panel Export (a buffered CSV/TSV download of the already-loaded grid).

  • Format follows the query: an explicit trailing FORMAT <name> (in
    either order relative to a SETTINGS clause — ClickHouse allows both)
    streams verbatim with a matching extension; otherwise defaults to
    TabSeparatedWithNames.
  • Mid-stream ClickHouse errors (after the response has already started,
    so the HTTP status can't change) are detected via the
    X-ClickHouse-Exception-Tag header + trailing __exception__ frame and
    excised with a 32 KiB hold-back write buffer, so the error text is never
    committed to the file — reported as "Export incomplete" instead. Verified
    the tag header's presence against a real ClickHouse 26.3.10 build (see
    Verification below).
  • Its own query_id/AbortController + progress banner — Cancel aborts
    the stream and issues its own KILL QUERY, entirely independent of the
    grid run's cancel state; re-entrance is guarded synchronously so a second
    click while the save dialog is open can't start a second export.
  • Session-aware: attaches the tab's session_id (sessionParamsFor) so
    an export depending on an earlier CREATE TEMPORARY TABLE/SET in the
    same tab works like Run does.
  • Multi-statement scripts are blocked with the same friendly message
    EXPLAIN uses, rather than sending an invalid combined request.
  • Chromium + secure-context only (showSaveFilePicker); the button stays
    visible but aria-disabled with a tooltip elsewhere.

Closes #87.

Verification

Beyond the unit suite, manually verified against the real antalya
ClickHouse 26.3.10 demo cluster (npm run local), driving the actual UI with
Playwright + a stubbed showSaveFilePicker so the real code path runs
unmodified:

  • Happy-path export produces byte-correct TSV, correct query_id tagging,
    correct toast.
  • X-ClickHouse-Exception-Tag is present on every response from this build.
  • Pre-header / fully-buffered failures are handled correctly.
  • ⚠️ Once a genuine mid-stream flush occurs on this specific cluster (~1.2 MB+
    of incompressible row data before the failure), the connection fails with a
    browser-level "network error" before ClickHouse's documented trailer frame
    ever arrives — reproduced identically through the real UI and via a raw
    fetch probe, with and without compression, so it isn't a harness quirk.
    The app's generic failure path (abort the writable, report "Export
    failed: network error", leave the partial file) handles this safely, but I
    could not observe the graceful "excise the trailer, report Export
    incomplete" path firing against a real server — only in unit tests
    against the documented wire format. Flagging for a maintainer call: chase
    this further (try another/newer real cluster) or accept the safe fallback.

Checklist

  • npm test passes (the per-file coverage gate is non-negotiable) — 1183 tests, core/net files 100%, app.js within its documented floor
  • Tests added/updated in the same change as the code
  • npm run build succeeds (single-file dist/sql.html)
  • Layers kept honest: pure logic in src/core/, network in src/net/ (injected fetch), DOM in src/ui/
  • No new runtime dependency
  • README / CHANGELOG.md ([Unreleased]) updated
  • Reconciled affected tracked work (roadmap Roadmap to 1.0.0 #68 checked off in a follow-up comment once merged)

Co-Authored-By: Claude Sonnet 5 noreply@anthropic.com

https://claude.ai/code/session_01EfEGDNjwJcudck7Dzm82Pu

)

Adds an Export button to the editor toolbar that runs the current query
uncapped and streams the response straight to a user-chosen file via the
File System Access API, bypassing the result grid so memory stays flat
regardless of result size. Format follows the query's own FORMAT clause
(default TabSeparatedWithNames); a mid-stream ClickHouse exception is
detected via the X-ClickHouse-Exception-Tag header + trailing frame and
excised with a hold-back buffer so it never reaches the file. Replaces
the old buffered CSV/TSV result-panel Export.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfEGDNjwJcudck7Dzm82Pu
@BorisTyshkevich BorisTyshkevich mentioned this pull request Jul 1, 2026
24 tasks
@BorisTyshkevich BorisTyshkevich merged commit 1f006c0 into main Jul 1, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Export button: stream full query result to disk as TSV (File System Access API, uncapped, bypasses the grid)

1 participant