[absorbed into #630 Phase 3] extract progress-stream and late-exception protocol primitives

[BorisTyshkevich]

Part of #630.

Depends on: #632

Goal

Move reusable ClickHouse streaming-protocol behavior from SQL Browser into @altinity/clickhouse-http without moving SQL Browser result/view state.

This unit owns three generic protocol concerns:

1. incremental decoding of ClickHouse progress-bearing JSON-lines formats;
2. standard ClickHouse HTTP exception-text extraction;
3. post-header / late exception-frame detection using X-ClickHouse-Exception-Tag and the __exception__ trailer.

Ownership decision

Move protocol mechanics, not SQL Browser's accumulator.

Package-owned

• the streamed line shape (meta, row, progress, exception);
• one TextDecoder with streaming decode across byte chunks;
• newline buffering and final remainder handling;
• malformed-line skip behavior exactly as currently shipped;
• per-network-chunk callback semantics;
• parseExceptionText() behavior;
• tagged late-exception framing;
• legacy untagged tail fallback;
• byte-accurate clean-data boundary reporting.

SQL Browser-owned

Keep these application/result concerns in src/core/stream.ts or a narrower successor:

• StreamResult;
• newResult();
• applyStreamLine() / result accumulation;
• row caps and capped state;
• UI progress percentage;
• rawText, rawFormat, cancelled;
• editor caret extraction (parseErrorPos);
• auth-expiry recognition and the SQL Browser login-denial message.

Package API

Expose a callback-driven streaming primitive because SQL Browser relies on both parsed-line callbacks and exact network-chunk repaint boundaries:

export interface ClickHouseProgressLine {
  meta?: Array<{ name: string; type: string; [key: string]: unknown }>;
  row?: Record<string, unknown>;
  progress?: {
    total_rows_to_read?: unknown;
    read_rows?: unknown;
    read_bytes?: unknown;
    elapsed_ns?: unknown;
    [key: string]: unknown;
  };
  exception?: string;
  [key: string]: unknown;
}

export interface ProgressStreamCallbacks {
  onLine?: (line: ClickHouseProgressLine) => void;
  onChunk?: () => void;
}

export function readProgressStream(
  body: ReadableStream<Uint8Array>,
  callbacks?: ProgressStreamCallbacks,
): Promise<void>;

Exact naming may follow package conventions, but semantics are fixed:

• one reader;
• one TextDecoder for the entire body;
• decoder.decode(value, { stream: true }) semantics across chunks;
• split only on newline boundaries;
• empty complete lines ignored;
• malformed complete JSON lines skipped as today;
• final non-empty remainder parsed once if complete JSON;
• reader/body errors, including AbortError, propagate unchanged;
• onChunk fires once per successfully read network chunk after complete lines from that chunk have been delivered, matching current behavior.

Do not add a second higher-level result accumulator in the package.

HTTP exception parsing

Move the current parseExceptionText() logic into the package. Preserve its current contract:

• detect ClickHouse's {"exception": ...} line form;
• return the extracted exception string when parseable;
• fall back to the raw response text when not recognized/parseable.

SQL Browser callers may use a temporary compatibility re-export while they are migrated in later units, but there must be one implementation.

Late exception-frame parsing

Move the protocol knowledge currently in findExceptionFrame() into the package.

The package-facing API must accept bytes rather than requiring SQL Browser callers to pre-convert bytes into a Latin-1 surrogate string:

export interface ClickHouseExceptionFrame {
  message: string;
  cleanBytes: number;
}

export function findExceptionFrame(
  tailBytes: Uint8Array,
  tag?: string | null,
): ClickHouseExceptionFrame | null;

Required behavior:

Tagged servers

Recognize the trailer framed by the server-provided tag:

\r\n__exception__\r\n<tag>\r\n<message>\n<len> <tag>\r\n__exception__\r\n

• decode exception message as UTF-8;
• report the exact count of clean bytes before the exception frame;
• do not match a marker without the response's server-chosen tag;
• do not interpret legitimate export bytes containing marker-like text as an exception unless the full framing matches.

Legacy fallback

When no tag exists, preserve the current anchored legacy detection of a final Code: <n>. DB::Exception: tail.

The fallback must remain end-anchored so exception-like text in real query/export data does not become a false positive when valid data follows.

SQL Browser integration

• src/net/clickhouse-http-transport.ts's temporary streamLines compatibility method delegates to the package stream reader.
• src/core/stream.ts imports or temporarily re-exports package protocol functions/types as needed, but must not retain duplicate implementations.
• Existing StreamResult/applyStreamLine tests remain SQL Browser tests.
• ExportService may continue using a compatibility wrapper until [absorbed into #630 Phase 7] migrate query execution and export, then delete generic client mechanics #637; this unit moves the implementation, not every consumer.

Tests

Progress reader

Cover at minimum:

• one line per chunk;
• several lines in one chunk;
• one line split across chunks;
• a multibyte UTF-8 character split across chunks;
• empty lines;
• malformed complete line skipped;
• trailing complete JSON without newline;
• trailing partial JSON ignored as today;
• meta, row, progress, and exception shapes passed unchanged;
• exact onChunk count/order;
• reader rejection propagated by object identity;
• AbortError propagated, not converted/swallowed.

Exception text

Port the full existing behavior tests and add malformed/unknown-body fallback coverage if missing.

Exception frame

Cover:

• tagged frame after clean data;
• UTF-8 multibyte exception text;
• correct cleanBytes at byte boundaries;
• marker-like content in clean data;
• wrong tag does not match;
• legacy untagged final exception;
• legacy exception-like text followed by more valid data does not match;
• no exception returns null;
• clean bytes containing invalid UTF-8 are not decoded/altered while locating the boundary.

Run the full repository gate. The existing browser cancellation suite from #631 must remain green; this unit must not change signal ownership.

npm run check:types
npm run check:arch
npm run check:schemas
npm run check:examples
npm test
npm run build

Run the targeted client browser suite if stream integration changes its raw-ESM harness/import map.

Acceptance criteria

• Progress JSON-lines decoding has one implementation in @altinity/clickhouse-http.
• ClickHouseProgressLine (or equivalent) is package-owned and has no dependency on SQL Browser types.
• SQL Browser's StreamResult and accumulation/view policy remain outside the package.
• parseExceptionText has one package implementation.
• Late exception-frame detection has one package implementation that accepts raw bytes.
• Tagged and legacy exception framing retain current false-positive protections.
• src/net/clickhouse-http-transport.ts delegates stream reading rather than implementing it.
• src/core/stream.ts no longer owns duplicate generic protocol parsers.
• Stream/body errors propagate unchanged.
• Full gate passes and [absorbed into #630 Phase 1] freeze native Fetch, Response, and cancellation semantics #631 cancellation semantics remain intact.

Non-goals

• Adding high-level queryJson/queryText/queryProgress request methods; [absorbed into #630 Phase 4] add consuming query APIs, ClickHouse errors, and stateless KILL QUERY #634 owns them.
• Moving StreamResult, row-cap trimming, UI percentages, or cancellation presentation state.
• Changing SQL Browser output-mode mappings.
• Changing auth-expiry/login classification.
• Migrating ExportService's full workflow; [absorbed into #630 Phase 7] migrate query execution and export, then delete generic client mechanics #637 owns that application integration.
• Adding compression/decompression behavior.

Agent execution notes

Before planning, read #630–#632, src/core/stream.ts, src/net/clickhouse-http-transport.ts, ExportService's current hold-back logic and tests, and all current stream/exception tests. Treat current byte-level framing and callback ordering as compatibility contracts; move them, do not redesign them.