Architecture

Architecture

Blinkql is a static, client-only application. There is no application server: the host serves HTML, JavaScript, CSS and WebAssembly files, and everything else happens in the browser.

System overview

flowchart TB
  subgraph Browser["Browser tab"]
    subgraph Main["Main thread (UI only)"]
      Editor["SQL editor<br/>CodeMirror 6"]
      Catalog["Catalog &<br/>schema browser"]
      Grid["Results grid<br/>virtualized"]
      Charts["Charts<br/>ECharts / uPlot"]
      UIStore["UI state<br/>Zustand"]
      ResultStore["Result store<br/>Arrow tables, outside React"]
    end
    subgraph Worker["Engine worker"]
      Duck["DuckDB-wasm"]
    end
    OPFS[("OPFS<br/>data files")]
    IDB[("IndexedDB<br/>queries, history, settings")]
  end
  Files["Local files<br/>drag & drop / file picker"] -->|registered by handle| Duck
  Remote["Remote Parquet / CSV<br/>HTTPS + CORS"] <-->|range requests| Duck
  Editor -->|SQL| Duck
  Duck -->|Arrow record batches| ResultStore
  ResultStore -->|visible row window| Grid
  ResultStore -->|aggregated series| Charts
  Catalog <-->|schema queries| Duck
  Duck <--> OPFS
  UIStore <--> IDB

Loading

Query lifecycle

sequenceDiagram
  actor U as User
  participant UI as UI (main thread)
  participant DB as DuckDB-wasm (worker)
  U->>UI: Run query (Ctrl/Cmd + Enter)
  UI->>DB: cancel any in-flight query
  UI->>DB: send(sql)
  DB-->>UI: Arrow record batches (streamed)
  UI->>UI: append batches to result store (outside React)
  UI->>U: render first visible rows immediately
  U->>UI: scroll
  UI->>UI: read only the rows in view from Arrow vectors

Loading

Core design rules

1. The main thread is for the UI only. Query execution happens in the engine worker; any other heavy work (formatting large exports, file conversion) moves to a worker too once profiling shows it's needed.
2. Large data never enters React state. Results live in a result store as Arrow tables. Components subscribe to small derived values: row count, column metadata, the visible window. See ADR-004.
3. Read columns, don't build rows. Cell values are read lazily from Arrow vectors at render time instead of converting results into arrays of row objects.
4. Push work down to the engine. Sorting, filtering, aggregation and chart downsampling are expressed as SQL, so DuckDB's vectorized engine does them instead of JavaScript loops.
5. Stream, then refine. Show the first rows as soon as they arrive and fill in the rest progressively.
6. Fail gracefully. Engine errors, out-of-memory conditions and worker crashes produce a clear message and a recovery path (restart the engine, suggest LIMIT or aggregation), never a blank screen.

Data ingestion

• Local files are registered with the engine by file handle, so DuckDB-wasm reads them directly instead of copying the whole file into JavaScript memory first. Drop them anywhere on the workbench or use Import files.
• One view per file, named after the file: Machine Events.csv becomes machine_events. Names that clash get a numeric suffix (_2), and names that would start with a digit get a t_ prefix. Creating the view reads the schema, so an unreadable file fails at import rather than at the first query.
• Formats: CSV and TSV, Parquet, JSON (an array, or one object per line) and JSON Lines (.jsonl, .ndjson). Parquet and JSON support are DuckDB extensions, served from Blinkql's own origin (ADR-008).
• Format detection: DuckDB's CSV sniffer detects delimiter, header and column types. Overriding its choices is planned.
• Remote files are queried by URL. For Parquet, only the footer and the byte ranges needed by the query are fetched.
• Persistence: each imported local file is copied to OPFS in the background and imported again under the same name on the next visit (ADR-011). A CSV table can be converted to Parquet, which replaces its kept copy (P-09).

Rendering results

• Virtualized grid: only rows (and later, columns) inside the viewport become DOM nodes.
• Canvas grid (planned): cells are drawn on a <canvas>, removing per-cell DOM cost entirely. A lightweight DOM layer remains for selection, copy and screen-reader access.
• Charts receive data that SQL has already aggregated or bucketed (for example with time_bucket), so a chart never receives millions of points.

Planned code organization

src/
  app/              # shell, layout, providers
  engine/           # DuckDB-wasm bootstrap, connection, query API, cancellation
  features/
    editor/         # CodeMirror setup, SQL language, autocomplete
    catalog/        # file import, schema browser, profiling
    results/        # result store, virtualized grid
    charts/         # chart builder and renderers
    templates/      # operations analytics pack
  lib/              # shared utilities
bench/
  datasets/         # deterministic dataset generator
  scenarios/        # benchmark scenarios
  results/          # committed benchmark results (JSON)
e2e/                # Playwright tests

Deployment topology

flowchart LR
  Dev["Pull request"] --> CI["GitHub Actions<br/>typecheck · lint · test · build · e2e · bench"]
  CI --> Preview["Vercel preview URL"]
  Main["Merge to main"] --> Prod["blinkql.vercel.app<br/>static files + headers"]

Loading

No servers, no databases, nothing to scale. Every visitor brings their own compute.