Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/local-dev.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,8 +55,8 @@ DS_LOCAL_DATA_ROOT=/tmp/my-ds-local bun run src/local/cli.ts start --name defaul

## Programmatic API

The published `@prisma/streams-local` package surface is built for both Bun and
Node consumers.
The published `@prisma/streams-local` package surface is built for Bun
`>=1.2.0` and Node `>=22` consumers.

```ts
import { startLocalDurableStreamsServer } from "@prisma/streams-local";
Expand Down
8 changes: 4 additions & 4 deletions docs/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,8 +70,8 @@ See [security.md](./security.md) and [auth.md](./auth.md).

## Prerequisites

- Bun `>=1.3.11`
- Node.js `>=22` for local embedding and Node-based consumers
- Bun `>=1.3.11` for the full self-hosted server and repository workflows
- Bun `>=1.2.0` or Node.js `>=22` for the published `@prisma/streams-local` package

## Quick Start

Expand Down Expand Up @@ -116,8 +116,8 @@ console.log(server.exports.sqlite.path);
await server.close();
```

The published `@prisma/streams-local` surface is built to run on both Bun and
Node. The full self-hosted server remains Bun-only.
The published `@prisma/streams-local` surface is built to run on Bun `>=1.2.0`
and Node `>=22`. The full self-hosted server remains Bun-only.

The local embedded runtime always applies the built-in `1024 MB` auto-tune
preset. That keeps the local package on the current control-plane and endpoint
Expand Down
18 changes: 14 additions & 4 deletions docs/releasing.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,14 @@ and other trusted local workflows.

## Release Checklist

Release branch policy:

- Always cut and publish releases from `main` only.
- Do not run `release.yml` against feature branches or temporary release
branches.
- Merge the release changes into `main` first, then dispatch the release
workflow from `main`.

0. Ensure npm trusted publishing is configured for both packages:

- `@prisma/streams-local`
Expand Down Expand Up @@ -46,7 +54,7 @@ These tests build the generated package directories, pack them, install them
into temporary consumers, and verify:

- Node end-to-end usage of `@prisma/streams-local`
- Bun end-to-end usage of `@prisma/streams-local`, including the live `/touch/*` path
- Bun end-to-end usage of `@prisma/streams-local` on Bun `1.2.x` and newer, including the live `/touch/*` path
- stateful local-runtime reopen flows that must read `/_schema` and skip
duplicate first-schema installs when the registry already matches
- local package exposure of `GET /v1/server/_details` and `GET /v1/stream/{name}/_routing_keys`
Expand Down Expand Up @@ -81,10 +89,10 @@ npm publish --access public ./dist/npm/streams-local
npm publish --access public ./dist/npm/streams-server
```

Or use the repository release workflow after pushing to `main`:
Or use the repository release workflow from `main` only:

```bash
gh workflow run release.yml
gh workflow run release.yml --ref main
```

The GitHub workflow builds, validates, and publishes both packages with npm
Expand Down Expand Up @@ -113,6 +121,8 @@ For `@prisma/streams-local`, the build intentionally:
each embed their own copy of the runtime
- keeps the local runtime Bun-compatible even though the generated bundle
targets the Node module surface
- publishes a local-package Bun engine floor of `>=1.2.0` while keeping the
full server on the repository Bun floor
- pins the embedded local runtime to the built-in `1024 MB` auto-tune preset so
Prisma CLI gets a predictable cache and concurrency budget
- keeps npm dependencies external instead of rebundling them into the local
Expand All @@ -133,7 +143,7 @@ The split gives you:

## Current Packaging Contract

- `@prisma/streams-local` supports Bun and Node
- `@prisma/streams-local` supports Bun `>=1.2.0` and Node `>=22`
- `@prisma/streams-local/internal/daemon` is intentionally internal
- `@prisma/streams-server` is Bun-only
- the root repository package is still private and is not the publish target
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "prisma-streams",
"version": "0.1.3",
"version": "0.1.4",
"private": true,
"description": "Prisma Streams is a Bun + TypeScript implementation of the Durable Streams HTTP protocol, with a local development server for Prisma CLI integration.",
"repository": {
Expand Down
14 changes: 10 additions & 4 deletions scripts/build-local-node.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { cpSync, mkdirSync, readdirSync, readFileSync, rmSync, writeFileSync } f
import { dirname, join, resolve } from "node:path";
import { fileURLToPath } from "node:url";
import { spawnSync } from "node:child_process";
import { localPackageBunEngine } from "./package-contract.mjs";

const __dirname = dirname(fileURLToPath(import.meta.url));
const repoRoot = resolve(__dirname, "..");
Expand Down Expand Up @@ -36,8 +37,8 @@ function replaceInBuiltLocalFiles(replacements) {
function writeDistReadme() {
const readme = `# Prisma Streams Local Build

This directory contains the generated Node-compatible package artifacts for the
published \`@prisma/streams-local\` runtime.
This directory contains the generated Node/Bun-compatible package artifacts for
the published \`@prisma/streams-local\` runtime.

## What Local Streams Is

Expand All @@ -52,13 +53,18 @@ The embedded local runtime always applies the built-in \`1024 MB\` auto-tune
preset, so Prisma CLI gets a predictable cache and concurrency budget and the
same current HTTP surface, including \`GET /v1/server/_details\`.

Published runtime floor:

- Bun \`${localPackageBunEngine}\`
- Node.js \`>=22\`

## Supported Package Surface

- \`@prisma/streams-local\`
- \`@prisma/streams-local/internal/daemon\` (internal Prisma CLI plumbing)

The full self-hosted server remains Bun-only and is not part of this Node build
surface.
The full self-hosted server remains Bun-only and is not part of this local
build surface.

## Integrating It

Expand Down
3 changes: 2 additions & 1 deletion scripts/build-npm-packages.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { chmodSync, cpSync, mkdirSync, readFileSync, rmSync, writeFileSync } fro
import { dirname, join, resolve } from "node:path";
import { fileURLToPath } from "node:url";
import { spawnSync } from "node:child_process";
import { localPackageBunEngine } from "./package-contract.mjs";

const __dirname = dirname(fileURLToPath(import.meta.url));
const repoRoot = resolve(__dirname, "..");
Expand Down Expand Up @@ -104,7 +105,7 @@ function buildLocalPackage() {
license: rootPackage.license,
type: "module",
engines: {
bun: rootPackage.engines.bun,
bun: localPackageBunEngine,
node: rootPackage.engines.node,
},
publishConfig: {
Expand Down
1 change: 1 addition & 0 deletions scripts/package-contract.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export const localPackageBunEngine = ">=1.2.0";
4 changes: 4 additions & 0 deletions scripts/test-bun-local-package.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import { fileURLToPath } from "node:url";
import { basename, dirname, join, resolve } from "node:path";
import { tmpdir } from "node:os";
import { spawnSync } from "node:child_process";
import { localPackageBunEngine } from "./package-contract.mjs";

const repoRoot = resolve(dirname(fileURLToPath(import.meta.url)), "..");
const TABLE_KEY_POSTS = "8c646d3dd6bc68f4";
Expand Down Expand Up @@ -38,6 +39,9 @@ try {
if ("@durable-streams/client" in (localPackageManifest.dependencies ?? {})) {
throw new Error("@prisma/streams-local should not publish @durable-streams/client");
}
if (localPackageManifest.engines?.bun !== localPackageBunEngine) {
throw new Error(`@prisma/streams-local should publish bun ${localPackageBunEngine}, got ${localPackageManifest.engines?.bun}`);
}
const packOutput = run("npm", ["pack", "--pack-destination", packDir], localPackageDir);
const tarballName = packOutput.split(/\r?\n/).filter(Boolean).at(-1);
if (!tarballName) throw new Error("npm pack did not produce a tarball name");
Expand Down
4 changes: 4 additions & 0 deletions scripts/test-node-local-package.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import { fileURLToPath } from "node:url";
import { basename, dirname, join, resolve } from "node:path";
import { tmpdir } from "node:os";
import { spawnSync } from "node:child_process";
import { localPackageBunEngine } from "./package-contract.mjs";

const repoRoot = resolve(dirname(fileURLToPath(import.meta.url)), "..");
const TABLE_KEY_POSTS = "8c646d3dd6bc68f4";
Expand Down Expand Up @@ -38,6 +39,9 @@ try {
if ("@durable-streams/client" in (localPackageManifest.dependencies ?? {})) {
throw new Error("@prisma/streams-local should not publish @durable-streams/client");
}
if (localPackageManifest.engines?.bun !== localPackageBunEngine) {
throw new Error(`@prisma/streams-local should publish bun ${localPackageBunEngine}, got ${localPackageManifest.engines?.bun}`);
}
const packOutput = run("npm", ["pack", "--pack-destination", packDir], localPackageDir);
const tarballName = packOutput.split(/\r?\n/).filter(Boolean).at(-1);
if (!tarballName) throw new Error("npm pack did not produce a tarball name");
Expand Down
2 changes: 1 addition & 1 deletion src/bootstrap.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
import { mkdirSync, rmSync } from "node:fs";
import { dirname } from "node:path";
import { zstdDecompressSync } from "node:zlib";
import type { Config } from "./config";
import { SqliteDurableStore } from "./db/db";
import type { ObjectStore } from "./objectstore/interface";
import { zstdDecompressSync } from "./util/zstd";
import { localSegmentPath, schemaObjectKey, segmentObjectKey, streamHash16Hex } from "./util/stream_paths";
import { retry } from "./util/retry";
import { dsError } from "./util/ds_error.ts";
Expand Down
2 changes: 1 addition & 1 deletion src/manifest.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
import { zstdCompressSync } from "node:zlib";
import { Result } from "better-result";
import type {
IndexRunRow,
Expand All @@ -13,6 +12,7 @@ import type {
StreamRow,
} from "./db/db";
import { encodeOffsetResult } from "./offset";
import { zstdCompressSync } from "./util/zstd";
import { dsError } from "./util/ds_error.ts";

function b64(bytes: Uint8Array): string {
Expand Down
2 changes: 1 addition & 1 deletion src/profiles/metrics/block_format.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { Result } from "better-result";
import { zstdCompressSync, zstdDecompressSync } from "node:zlib";
import { BinaryCursor, BinaryPayloadError, BinaryWriter, readI64 } from "../../search/binary/codec";
import { zstdCompressSync, zstdDecompressSync } from "../../util/zstd";
import type { MetricsBlockRecord } from "./normalize";

export type MetricsBlockSectionInput = {
Expand Down
2 changes: 1 addition & 1 deletion src/search/agg_format.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
import { Result } from "better-result";
import { zstdCompressSync, zstdDecompressSync } from "node:zlib";
import { BinaryCursor, BinaryPayloadError, BinaryWriter, concatBytes, readF64, readI64, readU16, readU32 } from "./binary/codec";
import { RestartStringTableView, encodeRestartStringTable } from "./binary/restart_strings";
import { readUVarint, readZigZagVarint, writeUVarint, writeZigZagVarint } from "./binary/varint";
import { zstdCompressSync, zstdDecompressSync } from "../util/zstd";
import type { SearchCompanionPlan, SearchCompanionPlanRollup } from "./companion_plan";

export type AggSummaryState = {
Expand Down
2 changes: 1 addition & 1 deletion src/segment/format.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
import { Result } from "better-result";
import { zstdCompressSync, zstdDecompressSync } from "node:zlib";
import { Bloom256 } from "../util/bloom256";
import { crc32c } from "../util/crc32c";
import { concatBytes, readU32BE, readU64BE, writeU32BE, writeU64BE } from "../util/endian";
import { dsError } from "../util/ds_error.ts";
import { zstdCompressSync, zstdDecompressSync } from "../util/zstd";

export type SegmentRecord = {
appendNs: bigint;
Expand Down
24 changes: 24 additions & 0 deletions src/util/zstd.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
import * as zlib from "node:zlib";
import { dsError } from "./ds_error.ts";

type ZstdOperation = (bytes: Uint8Array) => Uint8Array | Buffer;

function getZstdOperation(name: "zstdCompressSync" | "zstdDecompressSync") {
const operation = (zlib as Record<string, unknown>)[name];

if (typeof operation !== "function") {
throw dsError(
`${name} is not available in this runtime. Prisma Streams local requires node:zlib zstd support.`,
);
}

return operation as ZstdOperation;
}

export function zstdCompressSync(bytes: Uint8Array) {
return getZstdOperation("zstdCompressSync")(bytes);
}

export function zstdDecompressSync(bytes: Uint8Array) {
return getZstdOperation("zstdDecompressSync")(bytes);
}
4 changes: 2 additions & 2 deletions test/live_stream2_read_perf.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ describe("live stream-2 read perf fixtures", () => {
expect(bodyBytes).toBeGreaterThan(600_000);
expect(elapsedMs).toBeLessThan(100);
});
});
}, 15_000);

test("wal-tail read stays below 100ms on warm live fixture data", async () => {
await withFixtureApp(async (app) => {
Expand All @@ -96,5 +96,5 @@ describe("live stream-2 read perf fixtures", () => {
expect(bodyBytes).toBeGreaterThan(600_000);
expect(elapsedMs).toBeLessThan(100);
});
});
}, 15_000);
});
Loading