Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,32 @@ auto-generated per-PR notes; this file is the curated, human-readable history.

## [Unreleased]

### Added
- **Click a closed database row to draw its schema graph** (#124): expanding a
collapsed db in the tree now also draws its lineage in the bottom drawer, the
same as dragging it — collapsing again doesn't re-fetch or re-draw.
Drag-to-drawer is unchanged. On a schema with 50+ view/MV objects needing
`EXPLAIN AST`, the inline graph now draws **progressively**: the free edges
(dependencies/target/engine-arg/dictionary — no extra round trip) paint
immediately, then a single second layout merges in the view/MV source edges
once `EXPLAIN AST` settles, with a "resolving N/M…" toolbar readout. Below
that threshold the fetch is fast enough that a visible first paint would just
be flicker, so it still draws in one step. The loading placeholder / toolbar
now has a working **Cancel**: it aborts the in-flight fetch and either keeps
the already-drawn free-edges graph (marked partial) or falls back to the
empty-results placeholder, whichever has something to show.

### Fixed
- The inline schema-lineage graph had a stale-write race (same class as #97):
running or Explaining a query — or dragging/clicking a second db/table —
while a lineage fetch was still in flight could let the stale fetch's
resolution land on the tab's *new* result once it finally settled, silently
showing an old graph instead of the actual query output. A request-identity
guard now drops any write from a superseded fetch. Separately, an abort
during the best-effort `system.dictionaries` read inside the lineage fetch is
now correctly propagated as a cancellation instead of silently degrading to
"no dictionaries, continue".

## [0.2.0] - 2026-07-01

### Added
Expand Down
74 changes: 57 additions & 17 deletions src/net/ch-client.js
Original file line number Diff line number Diff line change
Expand Up @@ -77,9 +77,9 @@ export async function authedFetch(ctx, url, sql, signal) {
}
}

/** Run a query and return parsed JSON (FORMAT JSON). Throws on CH error. */
export async function queryJson(ctx, sql) {
const resp = await authedFetch(ctx, chUrl(ctx.origin, { format: 'JSON' }), sql);
/** Run a query and return parsed JSON (FORMAT JSON). Throws on CH error. `signal` (optional) aborts the request. */
export async function queryJson(ctx, sql, signal) {
const resp = await authedFetch(ctx, chUrl(ctx.origin, { format: 'JSON' }), sql, signal);
if (!resp.ok) throw new Error(parseExceptionText(await resp.text()));
return resp.json();
}
Expand Down Expand Up @@ -140,34 +140,65 @@ export async function loadSchema(ctx) {
return [...byDb.entries()].map(([db, v]) => ({ db, comment: v.comment, expanded: false, tables: v.tables }));
}

// Below this many view/MV objects needing `EXPLAIN AST`, a visible free-edges-
// first paint is just flicker — the fan-out settles fast enough on a small
// schema that nobody perceives two draws, only a redraw. `loadSchemaLineage`
// skips `onBase`/`onProgress` entirely below the threshold so the caller does
// one single, final draw instead (matching the pre-progressive-draw behavior).
export const AST_PROGRESSIVE_THRESHOLD = 50;

/**
* Load object-lineage rows for a database: the `system.tables` columns the graph
* builder needs + `system.dictionaries` sources, and (for views/MVs) the
* `EXPLAIN AST` source tables attached as `row.astTables`. `target_database`/
* `target_table` are intentionally not selected — they're a ClickHouse-Cloud-only
* column (absent on OSS/Altinity builds), so the MV target is parsed from
* `create_table_query` in `buildSchemaGraph`. Returns `{ tables, dictionaries }`.
*
* `opts.signal` cancels every underlying request (including the best-effort
* `system.dictionaries` read — an abort there propagates as a rejection of the
* whole call, not a silent "no dictionaries"; see `tryQueryData`).
* `opts.onBase({tables, dictionaries})` fires as soon as the free data (no
* `EXPLAIN AST` needed) is known — the caller can draw a first-pass graph from
* it (issue #124's progressive draw) before the per-view/MV source resolution
* below even starts. `opts.onProgress(done, total)` fires as each `EXPLAIN AST`
* settles (success or best-effort failure), for a "resolving N/M…" indicator.
* Both are skipped when fewer than `opts.progressiveThreshold` (default
* `AST_PROGRESSIVE_THRESHOLD`) objects need `EXPLAIN AST` — see the constant's
* comment.
*/
export async function loadSchemaLineage(ctx, focus) {
export async function loadSchemaLineage(ctx, focus, opts = {}) {
const { signal, onBase, onProgress, progressiveThreshold = AST_PROGRESSIVE_THRESHOLD } = opts;
const db = (focus && focus.db) || '';
const cols = 'database, name, engine, engine_full, create_table_query, as_select, '
+ 'toString(uuid) AS uuid, dependencies_database, dependencies_table, '
+ 'loading_dependencies_database, loading_dependencies_table, comment, '
// Card metadata (ignored by the inline graph; used by the rich fullscreen cards).
+ 'toUInt64(ifNull(total_rows, 0)) AS total_rows, toUInt64(ifNull(total_bytes, 0)) AS total_bytes, '
+ 'partition_key, sorting_key, primary_key, sampling_key';
const tablesJson = await queryJson(ctx, `SELECT ${cols} FROM system.tables WHERE database = ${sqlString(db)} ORDER BY startsWith(name, '_'), name`);
const tablesJson = await queryJson(ctx, `SELECT ${cols} FROM system.tables WHERE database = ${sqlString(db)} ORDER BY startsWith(name, '_'), name`, signal);
const tables = tablesJson.data || [];
// Best-effort: a denied/missing system.dictionaries (low-priv users lack
// SELECT on it) must degrade to no dictionary edges, never abort the graph.
const dictionaries = await tryQueryData(ctx, `SELECT database, name, source FROM system.dictionaries WHERE database = ${sqlString(db)}`) || [];
// SELECT on it) must degrade to no dictionary edges, never abort the graph —
// but a genuine cancellation must still propagate (tryQueryData rethrows it).
const dictionaries = await tryQueryData(ctx, `SELECT database, name, source FROM system.dictionaries WHERE database = ${sqlString(db)}`, signal) || [];
// Robust source extraction for views/MVs: let ClickHouse parse the SELECT.
await Promise.all(tables.map(async (t) => {
if (!t.as_select || (t.engine !== 'View' && t.engine !== 'MaterializedView')) return;
const astTargets = tables.filter((t) => t.as_select && (t.engine === 'View' || t.engine === 'MaterializedView'));
const total = astTargets.length;
const progressive = total >= progressiveThreshold;
if (progressive && onBase) onBase({ tables, dictionaries });
let done = 0;
await Promise.all(astTargets.map(async (t) => {
try {
const ast = await queryJson(ctx, 'EXPLAIN AST ' + t.as_select);
const ast = await queryJson(ctx, 'EXPLAIN AST ' + t.as_select, signal);
t.astTables = parseAstTables((ast.data || []).map((r) => r.explain).join('\n'));
} catch { /* best-effort — leave astTables undefined */ }
} catch (e) {
if (signal && signal.aborted && e && e.name === 'AbortError') throw e;
/* best-effort — leave astTables undefined */
} finally {
done++;
if (progressive && onProgress) onProgress(done, total);
}
}));
return { tables, dictionaries };
}
Expand Down Expand Up @@ -288,14 +319,23 @@ export async function loadTableDetail(ctx, db, table) {
};
}

// Run a query for its `data` rows, returning null on ANY error. Editor
// reference data is best-effort: a missing system table on older ClickHouse (or
// a denied SELECT) must degrade gracefully, never surface as a query error.
async function tryQueryData(ctx, sql) {
// Run a query for its `data` rows, returning null on ANY error EXCEPT a
// cancellation of a caller-supplied signal. Editor reference data / schema-
// lineage best-effort reads are meant to degrade gracefully on a missing
// system table or a denied SELECT — but when the caller passed a `signal` and
// aborted it, that means the caller's whole operation was cancelled, not that
// this particular sub-query failed, so it must propagate rather than be
// swallowed into "no data, continue" (#124). Gated on `signal.aborted`
// (not just the error's name) so a caller that never passed a signal — every
// site except `loadSchemaLineage` — keeps today's unconditional swallow, even
// if the underlying fetch happens to throw an AbortError-shaped error for some
// unrelated reason.
async function tryQueryData(ctx, sql, signal) {
try {
const json = await queryJson(ctx, sql);
const json = await queryJson(ctx, sql, signal);
return json.data || [];
} catch {
} catch (e) {
if (signal && signal.aborted && e && e.name === 'AbortError') throw e;
return null;
}
}
Expand Down
5 changes: 5 additions & 0 deletions src/state.js
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,11 @@ export function createState(read = { loadJSON, loadStr }) {
// effects; `resultView` is the active Table/JSON/Chart tab. Via `.value`.
running: signal(false),
abortController: null,
// In-flight schema-lineage fetch (issue #124's inline drawer graph) — its own
// AbortController, separate from `abortController` (run/script) and the
// export controllers, since a graph fetch isn't gated by `running` and a
// second click/drag must be able to supersede an in-flight one.
schemaGraphAbortController: null,
resultView: signal('table'),
// True while a streaming Export (issue #87) is in flight — separate from
// `running` (the grid run) so an export and a grid run never clobber each
Expand Down
80 changes: 73 additions & 7 deletions src/ui/app.js
Original file line number Diff line number Diff line change
Expand Up @@ -510,6 +510,7 @@ export function createApp(env = {}) {
if (!srcSql.trim()) return;
await ensureConfig();
if (!(await getToken())) { chCtx.onSignedOut(); return; }
cancelSchemaGraph(); // a Run/Explain takes over the result — don't leave a lineage fetch running

// EXPLAIN-view bookkeeping: the Explain button (opts.explain) forces any query
// into EXPLAIN-view mode; a normal Run clears that; switching an EXPLAIN tab
Expand Down Expand Up @@ -632,6 +633,7 @@ export function createApp(env = {}) {
if (app.state.running.value) return;
await ensureConfig();
if (!(await getToken())) { chCtx.onSignedOut(); return; }
cancelSchemaGraph(); // a script run takes over the result — don't leave a lineage fetch running
app.state.forceExplain = false;
const tab = app.activeTab();
const t0 = now();
Expand Down Expand Up @@ -833,26 +835,89 @@ export function createApp(env = {}) {
}
}

// Render the ClickHouse object-lineage graph for a dropped database/table into
// the data pane (queries system.* + EXPLAIN AST; the editor SQL is untouched).
// Abort any in-flight schema-lineage fetch. Called both as a manual Cancel
// (clearResult: true — the user asked to stop) and automatically whenever a
// new operation takes over the drawer (a fresh graph request, or Run/Explain
// replacing the tab's result outright) — in the automatic case the caller
// overwrites tab.result itself right after, so aborting the network request
// is all that's needed there (the identity guard in showSchemaGraph makes
// this belt-and-suspenders, not load-bearing, for correctness).
//
// With clearResult, the visible result depends on how far the fetch got: if
// Phase A (the free-edges graph) had already drawn, keep it on screen marked
// `partial` (its view/MV source edges may be incomplete); otherwise there's
// nothing worth keeping, so drop back to the normal empty-results placeholder.
function cancelSchemaGraph({ clearResult = false } = {}) {
if (app.state.schemaGraphAbortController) app.state.schemaGraphAbortController.abort();
app.state.schemaGraphAbortController = null;
if (!clearResult) return;
const tab = app.activeTab();
const sg = tab.result && tab.result.schemaGraph;
if (!sg || !sg.loading) return;
if (sg.nodes && sg.nodes.length) {
sg.loading = false;
sg.partial = true;
} else {
tab.result = null;
}
renderResults(app);
}

// Render the ClickHouse object-lineage graph for a dropped/clicked
// database/table into the data pane (queries system.* + EXPLAIN AST; the
// editor SQL is untouched). Two-phase on a large schema (#124): draws as soon
// as the free edges (dependencies/target/engine-arg/dictionary) are known,
// then a single second layout merges in view/MV source edges once EXPLAIN AST
// settles — so the pane isn't blank for the whole round trip. Below
// AST_PROGRESSIVE_THRESHOLD view/MV objects, loadSchemaLineage skips straight
// to one draw instead (onBase/onProgress never fire) — a visible first paint
// is just flicker when the whole fetch settles almost as fast anyway.
async function showSchemaGraph(focus) {
if (!focus || !focus.db) return;
await ensureConfig();
if (!(await getToken())) { chCtx.onSignedOut(); return; }
cancelSchemaGraph(); // a new click/drag replaces whatever graph was in flight
const tab = app.activeTab();
// Show a loading placeholder first — the lineage queries (system.* + an
// EXPLAIN AST per view/MV) can take a moment on a large database.
// Show a loading placeholder first — even Phase A (system.tables +
// system.dictionaries) is a network round trip.
tab.result = newResult('Table');
tab.result.schemaGraph = { focus, loading: true, nodes: [], edges: [] };
// `result` is the stale-write guard (mirrors #97's identity-guard shape):
// captured once, checked before every later write, so a Run/Explain or a
// second graph request that replaces tab.result mid-fetch can never have
// this call's (Phase A or Phase B) result land on the new tab.result.
const result = tab.result;
renderResults(app);
const controller = new AbortController();
app.state.schemaGraphAbortController = controller;
try {
const rows = await ch.loadSchemaLineage(chCtx, focus);
const g = buildSchemaGraph(rows, focus);
const lineage = await ch.loadSchemaLineage(chCtx, focus, {
signal: controller.signal,
onBase: (base) => {
if (tab.result !== result) return; // superseded before Phase A even landed
const g = buildSchemaGraph(base, focus);
result.schemaGraph = { focus, nodes: g.nodes, edges: g.edges, tableCount: (base.tables || []).length, loading: true };
renderResults(app);
},
onProgress: (done, total) => {
if (tab.result !== result || !result.schemaGraph || !result.schemaGraph.loading) return;
result.schemaGraph.progress = { done, total };
renderResults(app);
},
});
if (tab.result !== result) return; // superseded while Phase B was resolving
const g = buildSchemaGraph(lineage, focus);
// tableCount lets the renderer explain an empty result ("N tables, none linked").
tab.result.schemaGraph = { focus, nodes: g.nodes, edges: g.edges, tableCount: (rows.tables || []).length };
result.schemaGraph = { focus, nodes: g.nodes, edges: g.edges, tableCount: (lineage.tables || []).length };
} catch (e) {
// AbortError means cancelSchemaGraph() already left the pane in a clean
// state (partial graph or the empty placeholder) — nothing more to do.
if (e.name === 'AbortError') return;
if (tab.result !== result) return;
tab.result = newResult('Table');
tab.result.error = String((e && e.message) || e);
} finally {
if (app.state.schemaGraphAbortController === controller) app.state.schemaGraphAbortController = null;
}
renderResults(app);
}
Expand Down Expand Up @@ -1460,6 +1525,7 @@ export function createApp(env = {}) {
setExplainView,
setResultRowLimit,
showSchemaGraph,
cancelSchemaGraph,
expandSchemaGraph,
openNodeDetail,
insertCreate,
Expand Down
9 changes: 7 additions & 2 deletions src/ui/placeholder.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,13 @@
import { h } from './dom.js';
import { Icon } from './icons.js';

export function loadingPlaceholder(msg) {
// `onCancel`, when given, adds a Cancel button (mirrors the `.exp-cancel`
// button in results.js's export progress banner) — used by the schema-graph
// drawer's pre-Phase-A loading state (#124), where there's nothing on screen
// yet to keep the graph's own toolbar Cancel visible instead.
export function loadingPlaceholder(msg, onCancel) {
return h('div', { class: 'placeholder starting' },
h('span', { class: 'spin' }, Icon.spinner()),
h('div', null, msg));
h('div', null, msg),
onCancel ? h('button', { class: 'exp-cancel', title: 'Cancel', onclick: onCancel }, Icon.close(), h('span', null, 'Cancel')) : null);
}
36 changes: 28 additions & 8 deletions src/ui/results.js
Original file line number Diff line number Diff line change
Expand Up @@ -158,9 +158,13 @@ export function renderResults(app) {
} else if (r.error) {
inner.appendChild(h('div', { class: 'results-error' }, r.error));
} else if (r.schemaGraph) {
inner.appendChild(r.schemaGraph.loading
? loadingPlaceholder('Loading data flow…')
: renderSchemaGraph(app, r));
// Progressive draw (#124): once Phase A resolves (tableCount known) the
// real graph draws even while Phase B (per-view/MV EXPLAIN AST) is still
// loading — only the pre-Phase-A window (nothing known yet, always still
// loading by construction) shows the cancellable placeholder.
inner.appendChild(r.schemaGraph.tableCount != null
? renderSchemaGraph(app, r)
: loadingPlaceholder('Loading data flow…', () => app.actions.cancelSchemaGraph({ clearResult: true })));
} else if (r.explainView) {
inner.appendChild(renderExplainView(app, r));
} else if (r.rawText != null) {
Expand Down Expand Up @@ -441,16 +445,32 @@ function buildToolbar(app, r) {
}
if (r && r.schemaGraph) {
// Schema-lineage view: a title + Expand (fullscreen); no view-switcher / stats.
const f = r.schemaGraph.focus || {};
const sg = r.schemaGraph;
const f = sg.focus || {};
const title = f.kind === 'table' ? f.db + '.' + f.table : f.db;
toolbar.appendChild(h('div', { class: 'result-view-tabs' }, h('span', { class: 'res-graph-title' }, 'Schema · ' + title)));
if (sg.partial) toolbar.appendChild(h('span', { class: 'cancelled-badge' }, 'Cancelled · view/MV sources may be incomplete'));
toolbar.appendChild(h('div', { style: { flex: '1' } }));
// Expand is meaningless until the graph has loaded, or when there's nothing
// to draw (no connected objects → the pane shows a message, not a graph).
if (!r.schemaGraph.loading && r.schemaGraph.nodes.length) {
if (sg.loading && sg.tableCount != null) {
// Phase A has already drawn the graph into the body; Phase B (per-view/MV
// EXPLAIN AST) is still resolving — a live progress readout + Cancel, same
// shape as the run-in-progress stat/cancel block below. Pre-Phase-A (no
// graph in the body yet) the loading placeholder carries its own Cancel
// instead, so this doesn't duplicate it.
if (sg.progress) {
toolbar.appendChild(h('div', { class: 'stat live' }, h('span', { class: 'ic spin' }, Icon.spinner()),
h('span', { class: 'v' }, 'resolving ' + sg.progress.done + '/' + sg.progress.total + ' view sources…')));
}
toolbar.appendChild(h('button', {
class: 'res-act cancel-act', title: 'Cancel schema graph',
onclick: () => app.actions.cancelSchemaGraph({ clearResult: true }),
}, Icon.close(), h('span', null, 'Cancel')));
} else if (!sg.loading && sg.nodes.length) {
// Expand is meaningless when there's nothing to draw (no connected
// objects → the pane shows a message, not a graph).
toolbar.appendChild(h('button', {
class: 'res-act', title: 'Open the graph fullscreen with rich cards (pan & zoom)',
onclick: () => app.actions.expandSchemaGraph(r.schemaGraph.focus),
onclick: () => app.actions.expandSchemaGraph(sg.focus),
}, Icon.expand(), h('span', null, 'Expand')));
}
return toolbar;
Expand Down
Loading