feat(sea): type async-execute napi surface + pecotesting e2e#394
feat(sea): type async-execute napi surface + pecotesting e2e#394msrathore-db wants to merge 3 commits into
Conversation
1056dfa to
4e673fd
Compare
2538c19 to
e314073
Compare
4e673fd to
7d07de3
Compare
e314073 to
e21a4bb
Compare
7d07de3 to
8c5618c
Compare
e21a4bb to
ca04a30
Compare
Mirrors the new napi `submitStatement` / `AsyncStatement` /
`AsyncResultHandle` shapes from kernel branch
`msrathore/krn-async-execute` (commit `1957878`) on the JS side
and adds an end-to-end test against pecotesting.
Plumbing:
- `SeaNativeLoader.ts` — new typed interfaces
`SeaNativeStatementStatus` (string union mirroring the kernel
`StatementStatus` variant names), `SeaNativeAsyncStatement`,
`SeaNativeAsyncResultHandle`. `SeaNativeConnection` gains the
`submitStatement(sql, options?)` method.
- `native/sea/index.d.ts` — adds `AsyncStatement` /
`AsyncResultHandle` class declarations and the
`Connection.submitStatement(...)` method. Mirrors the napi-rs
codegen exactly.
- `tests/e2e/sea/async-execute-e2e.test.ts` — three cases:
1. submit → awaitResult → drain 100 rows; assert byte-decoded
Arrow row count matches the range(0, 100) query.
2. status() returns a string variant from the kernel enum
(Pending/Running/Succeeded/Closed depending on warehouse
responsiveness).
3. cancel() against a still-pending statement (range(0,
100_000_000)) completes within a 2s wire-latency budget.
Deferred to a follow-on PR: a full `SeaAsyncOperationBackend`
implementing the facade's `IOperationBackend` interface so
`DBSQLOperation` async-mode (matching Thrift's
`IDBSQLOperation` polling-mode surface) flows transparently.
The kernel + napi surface is ready; the JS adapter is the next
increment.
Cross-PR dependency: requires kernel branch
`msrathore/krn-async-execute` (commit `1957878`). Stacks on
`msrathore/sea-statement-options` (F3 + F4).
Co-authored-by: Isaac
Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
Same skip-gate fix as F2/F4 (DA round 1 H1): - `before()` verifies `native/sea/index.linux-x64-gnu.node` exists, skips with a clear "run yarn build:native" message if absent. - `loadBinding()` uses `createRequire(import.meta.url)` so the require works under both CJS and the ESM-reparse path mocha 11+ may use (MODULE_TYPELESS_PACKAGE_JSON reparse-as-ESM). Pre-emptive fix even though F3b wasn't on the DA fixup batch list — the same defect pattern would have been flagged on the next round. Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
Same ESM-reparse fix as F2/F4. Verified live e2e passes (submit → awaitResult → drain, status() variant, cancel-while-pending). Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
ca04a30 to
de55b96
Compare
|
Re-review requested — F1-F4 stack underneath this PR is now DA-approved at round-3; this PR (F3b NodeJS async-execute) is the next adversarial-review target. Round-3 final tip: This PR was parked behind the F1-F4 quality bar per team-lead's hard-stop directive. With F1-F4 + matrix now in APPROVED FOR MERGE state, F3b adversarial review starts on this PR + the kernel companion (#76). Findings will be addressed per the same protocol that closed F1-F4 round-1 → round-2 → round-3:
DA preview notes on F3b (Q1 IPC encoder lift to Co-authored-by: Isaac |
Summary
JS-side typing + live e2e test for the new napi async-execute surface delivered by kernel branch
msrathore/krn-async-execute(commit1957878, PR https://github.com/databricks/databricks-sql-kernel/pull/76).Plumbing
SeaNativeLoader.ts— new typed interfaces:SeaNativeStatementStatus— string union mirroring the kernelStatementStatusvariant names ('Pending' | 'Running' | 'Succeeded' | 'Failed' | 'Cancelled' | 'Closed' | 'Unknown')SeaNativeAsyncStatement—statementId,status(),awaitResult(),cancel(),close()SeaNativeAsyncResultHandle—statementId,fetchNextBatch(),schema()(no cancel/close — parent owns lifecycle)SeaNativeConnection.submitStatement(sql, options?)methodnative/sea/index.d.ts— mirrors the napi-rs codegen: newAsyncStatement/AsyncResultHandleclass declarations +Connection.submitStatement(...)method.tests/e2e/sea/async-execute-e2e.test.ts— three live-warehouse cases:range(0, 100); byte-decode the Arrow IPC and assert row countrange(0, 100_000_000)) completes within 2s wire-latency budgetDeferred (follow-on PR)
A full
SeaAsyncOperationBackendimplementingIOperationBackendsoDBSQLOperationasync-mode flows through the same facade callers use for Thrift. The kernel + napi surface is ready; the facade-level integration is the next increment. This PR ships the kernel→napi→JS shape and locks the behaviour with a live e2e.ULTRATHINK on async-cancel safety
Kernel's
AwaitResultCancelGuard(src/statement/executed_async.rs:305-356) installs an RAII drop hook inawait_result()that fires a fire-and-forgetcancel_statementif the future is dropped mid-poll. This covers:Promise.racelosersAbortController.abort()-driven cancellationPromise.race([awaitResult(), timer]))The napi
util::guardedcatch_unwindcovers V8 panic-across-boundary on top. TheArc<Mutex<Option<…>>>lock shape in the napiAsyncStatementwrapper matches the existingConnectionandStatementso cancel/close queue behind any in-flightawaitResult()until it returns naturally — the kernel handles the drop-the-future case independently.Cross-repo dependency
Requires kernel branch
msrathore/krn-async-execute(PR https://github.com/databricks/databricks-sql-kernel/pull/76). Stacks onmsrathore/sea-statement-options(F3 + F4, PR #393).Test plan