chore: add real session bundle measurement harness (#1336)

docs/architecture/session-bundle-size-and-cold-start-decision.md

@@ -0,0 +1,144 @@
+---
+doc_id: architecture.session-bundle-size-and-cold-start
+title: "Session Bundle Size, Cold Start, and Storage Policy"
+language: en
+source_language: en
+document_status: decision-record
+status: proposed
+date: 2026-07-23
+issue: 1336
+---
+
+# Session Bundle Size, Cold Start, and Storage Policy
+
+This is the timeboxed measurement record for [#1336](https://github.com/maka-agent/maka-agent/issues/1336). It defines the measurement contract and provisional implementation constraints. It does not turn synthetic fixtures or a developer workstation into production telemetry.
+
+## Decision Status
+
+The following are provisional planning constraints until the command is run against a sufficiently large set of sanitized real session exports:
+
+1. Keep a per-session inline bundle below **32 MiB compressed**. Put the manifest and normal session state inline; offload large workspace and artifact blobs by content address before the bundle reaches the cap.
+2. Treat `node_modules` as an image/cache concern and `.git` as a workspace materialization concern. Neither belongs in Session Bundle state.
+3. Use an S3-with-CAS-class repository: immutable content-addressed blobs plus a small mutable manifest pointer committed with conditional write semantics.
+4. Use Node 24's native Zstandard implementation for the `tar.zst` codec. Do not spawn an external compressor on the activation path.
+5. Keep hydrate plus local runtime bootstrap separate from provider first-token latency. The command reports a planning estimate only when an explicit provider TTFB input is supplied.
+
+These are candidate budgets, not measured SLOs. A report is decision-ready only when it is produced from sanitized real session exports.
+
+## Measurement Method
+
+The checked-in command is:
+
+```bash
+npm run measure:session-bundle -- \
+  --workspace /path/to/session-1/checkout \
+  --workspace /path/to/session-2/checkout \
+  --session-export /path/to/sanitized/state-1 \
+  --session-export /path/to/sanitized/state-2 \
+  --iterations 2 \
+  --provider-ttfb-ms 250 \
+  --runtime-build-id release-image-2026-07-23
+```
+
+`--session-export` is repeatable and each path must contain exactly one real exported state tree with `sessions/<id>/session.jsonl`. Each export is paired by position with the corresponding repeated `--workspace` path. A single workspace may be supplied for a smoke run or for comparison exports, but the report marks that evidence `workspacePairing: shared` and it can never be decision-ready. One-to-one workspace/export pairing is required for decision-ready evidence; canonical paths and session IDs must both be unique, so copying one export to another directory cannot manufacture percentile evidence. JSON and JSONL files receive a defense-in-depth redaction pass before measurement; exports must already be sanitized and must never contain credentials. `--provider-ttfb-ms` is optional; when omitted, the report records that no provider estimate was supplied and omits the first-token planning estimate.
+
+For each sample the command:
+
+- walks each selected checkout without following symlinks and recursively excludes every path segment named `.git` or `node_modules` from that sample's portable workspace stream;
+- excludes common workspace secret files such as `.env`, credentials/secrets files, private keys, certificates, and logs from the portable workspace stream;
+- materializes the selected state export and builds a real POSIX `manifest.json + state/** + workspace/**` tar stream;
+- compresses that exact tar stream with gzip, Brotli, and Node native Zstandard for codec comparison;
+- extracts and validates the archive, including byte counts and SHA-256 digests;
+- starts a fresh Node process, extracts the archive, opens the real session/runtime stores, constructs the Harbor cell runtime, and completes one FakeBackend turn;
+- reports provider TTFB only as the explicit input to a planning estimate. It is not a network measurement.
+
+When no `--session-export` is supplied, the command creates one real Maka FakeBackend run as a smoke fixture. That path validates the archive and bootstrap implementation but is explicitly marked `fake-bootstrap-smoke-only` and `decisionReady: false`.
+
+## Evidence Contract
+
+The JSON report records:
+
+- `evidence.kind`, the independent `bundleSizeDecisionReady` and `bootstrapLatencyDecisionReady` gates, and the combined `evidence.decisionReady`;
+- the bootstrap runtime identity (`node`, `platform`, `arch`, and the supplied non-empty build ID);
+- the exact archive format and layout;
+- raw tar and compressed byte distributions;
+- per-sample state, archive, hydrate, and fresh-process bootstrap measurements;
+- per-sample workspace roots, byte categories, and recursively filtered portable byte counts (with an aggregate only when multiple workspaces are supplied);
+- provider input provenance and the resulting planning estimate.
+
+Only a report with `evidence.kind: sanitized-real-session-exports`, at least 100 measured samples, one-to-one workspace/export pairing, and a controlled bootstrap runtime (Node 24.x on Linux x64 or arm64 with a non-empty `--runtime-build-id`) is marked `evidence.decisionReady: true` and may be used to update the budgets below. A smaller, shared-workspace, smoke, or uncontrolled-runtime report is suitable for regression tests and implementation debugging only.
+
+## Bundle and Repository Policy
+
+Use an S3-with-CAS-class repository, not a process-local or etcd-class value store:
+
+```text
+maka/session/{sessionId}/manifest       mutable pointer, conditional update
+maka/blob/{sha256}                     immutable state/workspace/artifact blob
+```
+
+The manifest is small and contains `schemaVersion`, `sessionId`, revision, activation identity, and ordered file/blob references. The control plane commits a new revision with an `If-Match`-class condition on the previous manifest revision. Blobs are immutable and may be uploaded before the manifest; an unreferenced blob is garbage-collectable.
+
+v1 limits:
+
+- 32 MiB maximum for an inline compressed session value;
+- 256 KiB maximum manifest size;
+- offload any individual file over 1 MiB, or offload the complete bundle when it would exceed 24 MiB before the hard 32 MiB rejection;
+- keep secrets, provider connections, device identity, activation input, and logs out of both the manifest and blobs.
+
+The 24 MiB soft threshold leaves room for manifest growth, tar framing, metadata, and measurement variance. The 32 MiB hard limit gives the repository a deterministic failure mode instead of allowing an activation to create an unexpectedly slow value.
+
+The repository interface should remain small:
+
+```ts
+interface SessionRepository {
+  checkout(sessionId: string): Promise<{ revision: string; manifest: Uint8Array | null }>;
+  putBlob(digest: string, bytes: Uint8Array): Promise<void>;
+  commitManifest(
+    sessionId: string,
+    expectedRevision: string,
+    manifest: Uint8Array,
+  ): Promise<boolean>;
+}
+```
+
+## Cold-Start Accounting
+
+The command separates these phases:
+
+1. archive hydrate: read, Zstandard decompress, tar extraction, digest validation;
+2. fresh-process Maka bootstrap: open stores, materialize the paired workspace, rebase the restored session paths, create the Harbor cell, and signal local runtime/session readiness from Runtime's `onRunStarted` hook after `run.begin()` and before provider execution; the child still completes and validates the turn after the timing point;
+3. provider first-token latency: an explicit external input, not measured by this command.
+
+The operational budget is therefore expressed as two independent budgets:
+
+- hydrate plus local bootstrap: a p99 target measured on the versioned image and a warm regional object-store path;
+- provider first token: a provider-specific target measured in the activation environment.
+
+No end-to-end first-token claim should be made by adding an invented provider number to a synthetic local benchmark. The report keeps the input visible so this distinction remains auditable.
+
+## Workspace Dependency Policy
+
+| Entry | v1 policy | Reason |
+| --- | --- | --- |
+| `node_modules/` | Exclude from the bundle. Bake Maka runtime dependencies into the versioned image. Materialize project dependencies from the lockfile through a cache keyed by lockfile and platform. | Reinstalling on every activation makes cold start dependent on a package registry. |
+| `.git/` | Exclude from the bundle. Materialize the repository at a commit/ref and persist uncommitted work as a patch or content-addressed workspace overlay. | Repository object growth is unrelated to session state growth. |
+| source workspace | Include small, session-owned changes inline when the cap allows. Offload large files by digest. | Workspace material must not make KV hydration unbounded. |
+
+## Follow-Up Measurements
+
+- Capture at least 100 sanitized real coding sessions before converting the 32 MiB cap or any latency target into a production SLO.
+- Measure remote object-store throughput, TLS, and regional p99 separately from local filesystem measurements.
+- Add provider-specific activation benchmarks before promising an end-to-end first-token target.
+- Keep tar traversal, path safety, quota enforcement, and manifest-integrity tests alongside the import/export implementation.
+- Re-run the command against a clean release checkout; a developer worktree is not a representative release baseline.
+
+## Reproducibility
+
+The script is exposed as:
+
+```bash
+npm run measure:session-bundle -- --workspace /path/to/checkout
+```
+
+It prints JSON so CI can archive the raw report and compare drift without making the decision record depend on hand-copied numbers.

package.json

@@ -28,7 +28,7 @@
     "test": "npm run clean && npm run build:test && npm run test:scripts && node scripts/run-workspace-tests-parallel.mjs",
     "test:dist": "npm run test:scripts && node scripts/run-workspace-tests-parallel.mjs --serial",
     "test:fast": "npm run build:test && npm run test:scripts && node scripts/run-workspace-tests-parallel.mjs",
-    "test:scripts": "node --test scripts/storybook-visual-smoke.test.mjs scripts/run-headless-tests.test.mjs scripts/run-workspace-tests-parallel.test.mjs scripts/sync-model-metadata.test.mjs scripts/cua-driver-provenance.test.mjs scripts/cu-e2e-fixture.test.mjs scripts/cu-e2e-scenarios.test.mjs scripts/cu-provider-matrix.test.mjs scripts/cu-report-sanitize.test.mjs scripts/cu-real-model-launcher.test.mjs scripts/cu-real-ax-model-e2e-contract.test.mjs scripts/cu-synthetic-model-scenario-contract.test.mjs scripts/cu-real-function-model-e2e-contract.test.mjs scripts/cu-real-anthropic-model-e2e-contract.test.mjs scripts/cu-real-runtime-model-e2e-contract.test.mjs scripts/license-contract.test.mjs scripts/macos-arm64-release.test.mjs",
+    "test:scripts": "node --test scripts/storybook-visual-smoke.test.mjs scripts/run-headless-tests.test.mjs scripts/run-workspace-tests-parallel.test.mjs scripts/sync-model-metadata.test.mjs scripts/cua-driver-provenance.test.mjs scripts/cu-e2e-fixture.test.mjs scripts/cu-e2e-scenarios.test.mjs scripts/cu-provider-matrix.test.mjs scripts/cu-report-sanitize.test.mjs scripts/cu-real-model-launcher.test.mjs scripts/cu-real-ax-model-e2e-contract.test.mjs scripts/cu-synthetic-model-scenario-contract.test.mjs scripts/cu-real-function-model-e2e-contract.test.mjs scripts/cu-real-anthropic-model-e2e-contract.test.mjs scripts/cu-real-runtime-model-e2e-contract.test.mjs scripts/license-contract.test.mjs scripts/macos-arm64-release.test.mjs scripts/measure-session-bundle.test.mjs",
     "dev": "npm --workspace @maka/desktop run dev:hmr --",
     "dev:full": "npm run build && npm --workspace @maka/desktop run start",
     "build": "npm --workspace @maka/core run build && npm --workspace @maka/storage run build && npm --workspace @maka/mcp run build && npm --workspace @maka/runtime run build && npm --workspace @maka/runtime-host run build && npm --workspace @maka/computer-use run build && npm --workspace @maka/headless run build && npm --workspace maka-agent run build && npm --workspace @maka/ui run build && npm --workspace @maka/desktop run build",
@@ -41,6 +41,7 @@
     "check:release": "npm run check:stale && npm run check:officecli-bundle && node scripts/check-dead-css.mjs --check",
     "package:macos-arm64": "node scripts/package-macos-arm64.mjs",
     "verify:macos-arm64": "node scripts/verify-macos-arm64-dmg.mjs",
+    "measure:session-bundle": "node scripts/measure-session-bundle.mjs",
     "check:chat-visual": "electron scripts/check-chat-marker-computed-style.mjs",
     "sync:model-metadata": "node scripts/sync-model-metadata.mjs",
     "cost:deepseek-baseline": "node scripts/deepseek-live-cost-baseline.mjs",

packages/headless/src/__tests__/harbor-cell.test.ts

@@ -181,6 +181,35 @@ class CellChildAdmissionProbeBackend implements AgentBackend {
   async dispose(): Promise<void> {}
 }
 
+class RunStartOrderingProbeBackend implements AgentBackend {
+  readonly kind: BackendKind = 'fake';
+  readonly sessionId: string;
+
+  constructor(
+    sessionId: string,
+    private readonly events: string[],
+    private readonly delayMs: number,
+  ) {
+    this.sessionId = sessionId;
+  }
+
+  async *send(input: BackendSendInput): AsyncIterable<SessionEvent> {
+    this.events.push('backend-send');
+    await new Promise((resolve) => setTimeout(resolve, this.delayMs));
+    yield {
+      type: 'complete',
+      id: 'run-start-ordering-complete',
+      turnId: input.turnId,
+      ts: Date.now(),
+      stopReason: 'end_turn',
+    };
+  }
+
+  async stop(): Promise<void> {}
+  async respondToPermission(_decision: PermissionDecision): Promise<void> {}
+  async dispose(): Promise<void> {}
+}
+
 const registerCellBackend = (registry: BackendRegistry): void => {
   registry.register(
     'fake',
@@ -1006,6 +1035,54 @@ describe('runHarborCell', () => {
     });
   });
 
+  test('fires onRunStarted after run.begin and before provider execution', async () => {
+    await withDirs(async ({ workspaceDir, outputDir, storageRoot }) => {
+      const events: string[] = [];
+      await runHarborCell({
+        config,
+        instruction: 'measure run start ordering',
+        cwd: workspaceDir,
+        outputDir,
+        storageRoot,
+        onRunStarted: () => {
+          events.push('run-started');
+        },
+        registerBackends: (registry) => {
+          registry.register(
+            'fake',
+            (ctx) => new RunStartOrderingProbeBackend(ctx.sessionId, events, 50),
+          );
+        },
+      });
+
+      assert.deepEqual(events.slice(0, 2), ['run-started', 'backend-send']);
+    });
+  });
+
+  test('rejects resume input that disagrees with the stored execution config', async () => {
+    await withDirs(async ({ workspaceDir, outputDir, storageRoot }) => {
+      const first = await runHarborCell({
+        config,
+        instruction: 'create resumable session',
+        cwd: workspaceDir,
+        outputDir,
+        storageRoot,
+      });
+
+      await assert.rejects(
+        runHarborCell({
+          config: { ...config, model: 'different-model' },
+          instruction: 'resume with conflicting config',
+          cwd: workspaceDir,
+          outputDir: join(outputDir, 'resume'),
+          storageRoot,
+          resumeSessionId: first.invocation.sessionId,
+        }),
+        /resume session model.*different-model.*fake-model/i,
+      );
+    });
+  });
+
   test('settles the active session before its hard deadline and writes final usage', async () => {
     await withDirs(async ({ workspaceDir, outputDir, storageRoot }) => {
       const deadline = { settleAfterMs: 1_000 };

packages/headless/src/harbor-cell.ts

@@ -27,6 +27,7 @@ import {
   type SynthesisCacheArtifactStore,
   type SynthesisCacheLoader,
   type SynthesisCacheWriter,
+  type TurnStartOptions,
 } from '@maka/runtime';
 import {
   createAttachmentByteReader,
@@ -138,6 +139,10 @@ export interface RunHarborCellInput {
   settleAfterMs?: number;
   now?: () => number;
   newId?: () => string;
+  /** Resume one already-materialized session instead of creating a fresh session. */
+  resumeSessionId?: string;
+  /** Optional measurement hook passed through Runtime's run-start boundary. */
+  onRunStarted?: TurnStartOptions['onRunStarted'];
 }
 
 export interface HarborCellContinuationPolicy {
@@ -292,6 +297,9 @@ export async function runHarborCellWithStorage(
   const sessionStore = storage.executionStores.sessionStore;
   const agentRunStore = storage.executionStores.agentRunStore;
   const runtimeEventStore = storage.executionStores.runtimeEventStore;
+  const resumedSession = input.resumeSessionId
+    ? await sessionStore.readHeaderSnapshot(input.resumeSessionId)
+    : undefined;
   const backends = new BackendRegistry();
   const sessionCapabilities = createHeadlessSessionCapabilityBridge();
   const task: Task = {
@@ -303,6 +311,23 @@ export async function runHarborCellWithStorage(
   const economyTaskMode = resolveEconomyTaskMode(input.config, task);
   const prompt = resolveHeadlessSystemPrompt(input.config, { heavyTaskMode, economyTaskMode });
   const config = { ...input.config, systemPrompt: prompt.systemPrompt };
+  if (resumedSession) {
+    const executionFacts = [
+      ['cwd', input.cwd, resumedSession.cwd],
+      ['backend', input.config.backend, resumedSession.backend],
+      ['llmConnectionSlug', config.llmConnectionSlug, resumedSession.llmConnectionSlug],
+      ['model', config.model, resumedSession.model],
+      ['thinkingLevel', config.thinkingLevel, resumedSession.thinkingLevel],
+      ['permissionMode', 'execute', resumedSession.permissionMode],
+    ] as const;
+    for (const [name, expected, observed] of executionFacts) {
+      if (expected !== observed) {
+        throw new Error(
+          `Harbor resume session ${name} expected ${String(expected)}, observed ${String(observed)}`,
+        );
+      }
+    }
+  }
   const registerBackends =
     input.registerBackends ?? ((registry: BackendRegistry) => registerFakeBackend(registry));
   await registerBackends(backends, {
@@ -354,16 +379,17 @@ export async function runHarborCellWithStorage(
   });
   sessionCapabilities.bind(manager);
 
-  const session = await manager.createSession({
-    cwd: input.cwd,
-    backend: input.config.backend,
-    llmConnectionSlug: config.llmConnectionSlug,
-    model: config.model,
-    ...(config.thinkingLevel ? { thinkingLevel: config.thinkingLevel } : {}),
-    permissionMode: 'execute',
-    name: `harbor-cell:${input.config.id}`,
-  });
-
+  const session =
+    resumedSession ??
+    (await manager.createSession({
+      cwd: input.cwd,
+      backend: input.config.backend,
+      llmConnectionSlug: config.llmConnectionSlug,
+      model: config.model,
+      ...(config.thinkingLevel ? { thinkingLevel: config.thinkingLevel } : {}),
+      permissionMode: 'execute',
+      name: `harbor-cell:${input.config.id}`,
+    }));
   let deadlineReached = false;
   let settlementError: unknown;
   let settlementAttempt: Promise<void> | undefined;
@@ -405,7 +431,10 @@ export async function runHarborCellWithStorage(
       for await (const event of manager.sendMessage(
         session.id,
         { turnId, text: nextText },
-        { runId },
+        {
+          runId,
+          ...(input.onRunStarted ? { onRunStarted: input.onRunStarted } : {}),
+        },
       )) {
         if ((event as { type?: string }).type === 'permission_request') {
           const { requestId } = event as { requestId: string };