feat: Complete Phase M — Technical Debt Resolution (M.1 + M.2 + M.3) by Copilot · Pull Request #252 · objectstack-ai/objectos

Copilot · 2026-02-12T04:21:31Z

Implements all 12 tasks across Phase M, resolving all 8 identified technical debt items for production readiness.

M.1 — Critical Security

Rate limiting (api/middleware/rate-limit.ts) — Sliding-window counter per IP, 100 req/min general, 10 req/min on /api/v1/auth/*, standard X-RateLimit-* headers, periodic GC of expired entries
Input sanitization — HTML entity encoding (sanitize.ts), 1MB body limit (body-limit.ts), JSON content-type guard (content-type-guard.ts), Zod validation factory (validate.ts). All registered on /api/v1/* in api/index.ts
WebSocket auth (packages/realtime/src/plugin.ts) — Token extraction from better-auth.session_token cookie, Sec-WebSocket-Protocol header, or ?token= query param. Auth required by default; 5-minute heartbeat re-validation. +5 tests
Mock data tree-shaking — Relocated mock data to __mocks__/, added DevDataProvider with dynamic import() behind import.meta.env.DEV guard, VITE_USE_MOCK_DATA env flag

M.2 — Infrastructure

Persistent job storage (packages/jobs/src/persistent-storage.ts) — PersistentJobStorage implementing JobStorage backed by StorageBackend KV
Dead Letter Queue — moveToDeadLetter(), replayDeadLetter(), purgeDeadLetters(). Auto-DLQ on max retries in queue.ts
Schema migration engine — SchemaDiffer (column diff detection), MigrationRunnerImpl (apply/rollback with checksum tracking), MigrationGenerator (diff → executable migration). +22 tests
Migration CLI (migration-cli.ts) — MigrationCLI facade with up(), down(), status() for @objectstack/cli integration
Browser sync E2E — 5 Playwright specs: service worker registration, online sync, offline mutation queue, conflict resolution, selective sync

M.3 — Platform Hardening

Worker Thread host (worker-plugin-host.ts) — Level 1 isolation via worker_threads, MessagePort RPC, V8 resourceLimits
Child Process host (process-plugin-host.ts) — Level 2 isolation via child_process.fork(), IPC channel
Plugin watchdog (plugin-watchdog.ts) — Periodic heartbeat pings, exponential backoff restarts (2^n), configurable maxRestarts

// Plugin isolation usage
const host = new WorkerThreadPluginHost({
  pluginPath: '/path/to/plugin.js',
  isolation: 'worker',
  resourceLimits: { maxOldGenerationSizeMb: 128 },
}, { workerEntry: new URL('./worker-entry.js', import.meta.url).pathname });

const watchdog = new PluginWatchdog({ maxRestarts: 5, backoffMs: 1000 });
watchdog.watch(host);

Tests: 18 suites passing (244 web, 44 realtime, 86 jobs, 65 storage). CodeQL: 0 alerts.

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

vercel · 2026-02-12T04:21:35Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
objectos	Canceled		Feb 12, 2026 7:35am
objectos-demo	Error		Feb 12, 2026 7:35am

M.1.1: Rate limiting middleware with sliding-window counter M.1.2: Input sanitization (body-limit, XSS strip, content-type guard, Zod validate) M.1.3: WebSocket auth enforcement with token extraction and session heartbeat M.1.4: Mock data tree-shaking with DevDataProvider and __mocks__/ relocation Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>

- Add 5 WebSocket auth enforcement tests (TD-5) - Add 2 DevDataProvider export tests (TD-8) - Fix test to use auth: { required: false } for non-auth tests - Update ROADMAP.md and technical-debt-resolution.md — M.1 ✅ complete Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>

… M.2.2) - Add PersistentJobStorage backed by StorageBackend KV interface - Add Dead Letter Queue with moveToDeadLetter, getDeadLetters, replayDeadLetter, purgeDeadLetters - Add PersistenceBackend and DeadLetterEntry types - Wire persistent storage into JobsPlugin via persistence config option - Move failed jobs to DLQ automatically when using PersistentJobStorage - All 86 existing tests continue to pass Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>

…-backed Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>

Add SchemaDiffer, MigrationRunnerImpl, and MigrationGenerator classes with full type definitions and test coverage. Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>

…ons array Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>

M.2.4: MigrationCLI with up/down/status commands M.2.5: 5 Playwright E2E sync test specs + sync helpers Update ROADMAP.md — M.2 ✅ complete Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>

Add tiered plugin isolation infrastructure: - M.3.1: WorkerThreadPluginHost (worker_threads with MessagePort RPC) - M.3.2: ChildProcessPluginHost (child_process.fork with IPC) - M.3.3: PluginWatchdog (heartbeat monitoring + exponential backoff restart) - Types: PluginIsolationLevel, PluginHostConfig, PluginHostStatus, WatchdogConfig, PluginHost - Entry scripts: worker-entry.ts, process-entry.ts Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>

Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>

M.3.1: WorkerThreadPluginHost (Level 1 isolation) M.3.2: ChildProcessPluginHost (Level 2 isolation) M.3.3: PluginWatchdog with auto-restart and exponential backoff All Phase M (M.1 + M.2 + M.3) complete — 8/8 technical debt items resolved Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>

…egex - Fix WorkerThreadPluginHost and ChildProcessPluginHost to not use __dirname - Improve sanitize.ts with multi-pass loop for robust HTML/script removal - Remove unused regex constants Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>

…anitize middleware HTML entity encoding (<, >, &, ", ') is more robust than regex-based tag stripping and eliminates CodeQL js/bad-tag-filter and js/incomplete-multi-character-sanitization alerts. Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>

Copilot

Pull request overview

Completes Phase M technical-debt items across the platform kernel (HTTP hardening, realtime WS auth, infra persistence/migrations, plugin isolation) plus supporting tests, docs, and dev tooling for mock data and sync E2E.

Changes:

Add storage schema migration engine (differ/generator/runner/CLI) + tests.
Add plugin isolation hosts (worker thread + child process) and a watchdog.
Add API hardening middleware (rate limit/body limit/content-type guard/sanitize), realtime WS auth enforcement tests, persistent job storage + DLQ, and sync Playwright specs.

Reviewed changes

Copilot reviewed 40 out of 40 changed files in this pull request and generated 16 comments.

Show a summary per file

File	Description
packages/storage/test/schema-migration.test.ts	New tests for SchemaDiffer/MigrationRunnerImpl/MigrationGenerator behavior.
packages/storage/src/worker-plugin-host.ts	Worker-thread plugin host implementing RPC + lifecycle.
packages/storage/src/worker-entry.ts	Worker entrypoint that loads plugin module and handles RPC.
packages/storage/src/process-plugin-host.ts	Child-process plugin host implementing IPC RPC + lifecycle.
packages/storage/src/process-entry.ts	Child process entrypoint that loads plugin module and handles IPC RPC.
packages/storage/src/plugin-watchdog.ts	Watchdog to heartbeat/restart isolated plugin hosts with backoff.
packages/storage/src/types.ts	Adds plugin isolation and schema migration types.
packages/storage/src/schema-differ.ts	Implements schema diffing for columns.
packages/storage/src/migration-runner.ts	Applies/rolls back migrations and records applied versions.
packages/storage/src/migration-generator.ts	Generates up/down migration functions from schema diffs.
packages/storage/src/migration-cli.ts	CLI facade for up/down/status over the migration runner.
packages/storage/src/index.ts	Exports new migration + plugin isolation modules.
packages/realtime/src/plugin.ts	Adds WS auth handshake + periodic session revalidation logic.
packages/realtime/test/plugin.test.ts	Updates existing tests for default auth requirement; adds auth enforcement tests.
packages/jobs/src/types.ts	Adds persistence config + DLQ types.
packages/jobs/src/queue.ts	Moves permanently-failed jobs to DLQ when using persistent storage.
packages/jobs/src/plugin.ts	Adds persistence-based storage upgrade logic and rebuild helper.
packages/jobs/src/persistent-storage.ts	KV-backed persistent JobStorage with DLQ APIs.
packages/jobs/src/index.ts	Exports persistent storage and new types.
api/middleware/rate-limit.ts	Sliding-window in-memory rate limiter middleware.
api/middleware/body-limit.ts	Middleware to reject large requests based on Content-Length.
api/middleware/content-type-guard.ts	Middleware to enforce JSON Content-Type for mutations.
api/middleware/sanitize.ts	Middleware to HTML-entity encode JSON string values and store sanitized body.
api/middleware/validate.ts	Zod validation middleware storing validated body on context.
api/index.ts	Registers new API hardening middleware on `/api/v1/*`.
apps/web/src/providers/dev-data-provider.tsx	Dev-only mock data provider using dynamic imports for tree-shaking.
apps/web/src/lib/mock-data.ts	Re-exports mock data from `__mocks__` location.
apps/web/src/lib/mock-workflow-data.ts	Re-exports workflow mock data from `__mocks__` location.
apps/web/src/lib/mocks/mock-data.ts	New tree-shakeable mock metadata/records module.
apps/web/src/lib/mocks/mock-workflow-data.ts	New tree-shakeable workflow/automation mock module.
apps/web/src/tests/providers/dev-data-provider.test.ts	Basic export tests for DevDataProvider and hook.
apps/web/.env.development	Enables mock data by default in dev via VITE_USE_MOCK_DATA.
e2e/fixtures/sync-helpers.ts	Playwright helpers for offline/online and sync UI waits.
e2e/sync-registration.spec.ts	E2E: service worker registration checks.
e2e/sync-online.spec.ts	E2E: online shell load and optional rate-limit header check.
e2e/sync-offline.spec.ts	E2E: offline mode and storage APIs availability checks.
e2e/sync-conflict.spec.ts	E2E: basic conflict-resolution UI bundle presence checks.
e2e/sync-selective.spec.ts	E2E: selective sync settings page + browser storage APIs checks.
docs/guide/technical-debt-resolution.md	Marks Phase M tasks as complete in guide.
ROADMAP.md	Bumps roadmap version/status and marks Phase M items complete.

Copilot · 2026-02-12T05:52:22Z

packages/storage/src/worker-plugin-host.ts

+        // Worker entry must be provided explicitly or resolved by the caller.
+        // In ESM, __dirname is not available — callers should use
+        // `new URL('./worker-entry.js', import.meta.url).pathname` to resolve.
+        this.workerEntry = options?.workerEntry ?? 'worker-entry.js';
+    }


The comment says the worker entry must be provided explicitly/resolved by the caller, but the implementation defaults workerEntry to 'worker-entry.js'. That default is unlikely to resolve correctly (it’s relative to process.cwd() when passed to new Worker(...)) and contradicts the intended API. Consider making workerEntry required (no default), or resolve it deterministically (e.g., via new URL(..., import.meta.url) inside the package).

Copilot · 2026-02-12T05:52:22Z

packages/storage/src/worker-plugin-host.ts

+            this.worker.on('error', (err: Error) => {
+                this.alive = false;
+                this.rejectAllPending(err);
+            });


If the worker emits an error during startup (before the ready message), start() never calls rejectStart, so the returned promise can hang indefinitely. Consider tracking a “starting” state and rejecting start() on error/exit until ready has been received.

Copilot · 2026-02-12T05:52:22Z

packages/storage/src/process-entry.ts

+    // Validate plugin path — must be absolute to prevent path traversal
+    if (!require('node:path').isAbsolute(pluginPath)) {
+        throw new Error('pluginPath must be an absolute path');
+    }


process-entry.ts uses require('node:path') to call isAbsolute(). In an ESM module this will throw because require is not defined, preventing the child process host from starting. Use an ESM import from node:path instead.

Copilot · 2026-02-12T05:52:23Z

api/middleware/body-limit.ts

+    const contentLength = c.req.header('content-length');
+    if (contentLength && parseInt(contentLength, 10) > maxSize) {
+      return c.json(
+        { error: 'Payload too large', maxSize },
+        413,
+      );
+    }


bodyLimit only enforces limits when Content-Length is present. Requests using chunked transfer encoding (or incorrect/missing Content-Length) can bypass this check. Consider enforcing the max size based on bytes actually read (or using a body parser limit that doesn’t rely on headers).

Suggested change

const contentLength = c.req.header('content-length');

if (contentLength && parseInt(contentLength, 10) > maxSize) {

return c.json(

{ error: 'Payload too large', maxSize },

413,

);

}

const rawReq = c.req.raw;

// Fast-path: honor Content-Length when present

const contentLength = rawReq.headers.get('content-length');

if (contentLength && parseInt(contentLength, 10) > maxSize) {

return c.json(

{ error: 'Payload too large', maxSize },

413,

);

}

// If there's no body, nothing to limit.

if (!rawReq.body) {

await next();

return;

}

// Enforce limit based on actual bytes read, regardless of transfer encoding.

const [limitStream, forwardStream] = rawReq.body.tee();

const reader = limitStream.getReader();

let totalBytes = 0;

// Read until stream ends or we exceed maxSize.

while (true) {

const { done, value } = await reader.read();

if (done) {

break;

}

if (value) {

totalBytes += value.byteLength;

if (totalBytes > maxSize) {

// Exceeded limit: stop reading and reject.

reader.releaseLock();

return c.json(

{ error: 'Payload too large', maxSize },

413,

);

}

}

}

// Within limit: create a new Request with the untouched tee branch

// so downstream middleware/handlers can still consume the body.

const limitedReq = new Request(rawReq, { body: forwardStream });

// Hono's Context.req wraps the underlying Request; update its raw field.

// @ts-expect-error: accessing framework-internal property

(c.req as any).raw = limitedReq;

Copilot · 2026-02-12T05:52:23Z

packages/storage/src/process-plugin-host.ts

+        // Process entry must be provided explicitly or resolved by the caller.
+        // In ESM, __dirname is not available — callers should use
+        // `new URL('./process-entry.js', import.meta.url).pathname` to resolve.
+        this.processEntry = options?.processEntry ?? 'process-entry.js';
+    }


The comment says the process entry must be provided explicitly/resolved by the caller, but the implementation defaults processEntry to 'process-entry.js', which is relative to process.cwd() for fork() and is likely to break depending on where the process is launched from. Consider making processEntry required or resolving it deterministically within the package.

Copilot · 2026-02-12T05:52:25Z

packages/storage/src/process-plugin-host.ts

+                this.alive = false;
+                if (code !== 0) {
+                    this.rejectAllPending(new Error(`Child process exited with code ${code}`));
+                }
+            });


If the child process exits with a non-zero code before sending ready, start() doesn’t reject the startup promise, so callers may wait indefinitely. The exit handler should reject start() when startup hasn’t completed yet.

Copilot · 2026-02-12T05:52:25Z

packages/storage/src/migration-runner.ts

+            appliedAt: new Date().toISOString(),
+            checksum: simpleHash(`${migration.version}:${migration.name}`),
+        };


MigrationRecord.checksum is documented as change-detection, but it’s computed from only ${version}:${name}. That won’t detect edits to the migration’s up/down logic. Consider hashing a stable representation of the migration contents (e.g., up.toString()/down.toString() or an explicit checksum exported alongside the migration).

Copilot · 2026-02-12T05:52:25Z

api/index.ts

 import { secureHeaders } from 'hono/secure-headers';
+import { rateLimit } from './middleware/rate-limit.js';
+import { bodyLimit } from './middleware/body-limit.js';
+import { sanitize } from './middleware/sanitize.js';
+import { contentTypeGuard } from './middleware/content-type-guard.js';


The PR description mentions the Zod validation middleware being registered on /api/v1/*, but api/index.ts doesn’t import or apply validate(). If validation is intended as part of the global middleware stack, add it here (or update the PR description/docs to reflect that validation is route-specific).

Copilot · 2026-02-12T05:52:25Z

packages/storage/src/worker-entry.ts

+        throw new Error('worker-entry requires a valid pluginPath in workerData');
+    }
+    if (!require('node:path').isAbsolute(pluginPath)) {
+        throw new Error('pluginPath must be an absolute path');
+    }


worker-entry.ts uses require('node:path') to call isAbsolute(). This file is authored as ESM (uses import/dynamic import()), so require will be undefined at runtime and the worker will fail to start. Replace this with an ESM import (e.g., import { isAbsolute } from 'node:path') or another ESM-safe check.

Copilot · 2026-02-12T05:52:26Z

packages/storage/src/worker-plugin-host.ts

+import { Worker, type ResourceLimits } from 'node:worker_threads';
+import { resolve } from 'node:path';
+import type { PluginHost, PluginHostConfig } from './types.js';


Unused import: resolve from node:path is never referenced in this file. Please remove it to avoid lint/build failures in strict setups.

Initial plan

061f279

Copilot AI assigned Copilot and hotlong Feb 12, 2026

Copilot started work on behalf of hotlong February 12, 2026 04:21 View session

Copilot AI and others added 12 commits February 12, 2026 04:28

docs: fix persistent-storage JSDoc to say KV-backed instead of SQLite…

f2dd04e

…-backed Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>

feat(storage): add schema migration engine (Phase M.2.3)

291e527

Add SchemaDiffer, MigrationRunnerImpl, and MigrationGenerator classes with full type definitions and test coverage. Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>

fix(storage): address code review - stable checksum, document operati…

6602577

…ons array Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>

feat(M.2): complete Phase M.2 — Infrastructure

19d785a

M.2.4: MigrationCLI with up/down/status commands M.2.5: 5 Playwright E2E sync test specs + sync helpers Update ROADMAP.md — M.2 ✅ complete Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>

fix(storage): validate plugin paths are absolute in entry scripts

f130ff3

Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Complete all development as per roadmap~~ feat: Complete Phase M — Technical Debt Resolution (M.1 + M.2 + M.3) Feb 12, 2026

Copilot AI requested a review from hotlong February 12, 2026 05:13

Copilot finished work on behalf of hotlong February 12, 2026 05:13

hotlong marked this pull request as ready for review February 12, 2026 05:45

Copilot AI review requested due to automatic review settings February 12, 2026 05:45

hotlong merged commit 46526d9 into main Feb 12, 2026
3 of 5 checks passed

Copilot started reviewing on behalf of hotlong February 12, 2026 05:46 View session

Copilot AI reviewed Feb 12, 2026

View reviewed changes

vercel bot had a problem deploying to Preview – objectos-demo February 12, 2026 07:21 Failure

vercel bot deployed to Preview – objectos February 12, 2026 07:35 View deployment

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Complete Phase M — Technical Debt Resolution (M.1 + M.2 + M.3)#252

feat: Complete Phase M — Technical Debt Resolution (M.1 + M.2 + M.3)#252
hotlong merged 13 commits intomainfrom
copilot/complete-development-roadmap-again

Copilot AI commented Feb 12, 2026 •

edited

Loading

Uh oh!

vercel bot commented Feb 12, 2026 •

edited

Loading

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Feb 12, 2026

Uh oh!

Copilot AI Feb 12, 2026

Uh oh!

Copilot AI Feb 12, 2026

Uh oh!

Copilot AI Feb 12, 2026

Uh oh!

Copilot AI Feb 12, 2026

Uh oh!

Copilot AI Feb 12, 2026

Uh oh!

Copilot AI Feb 12, 2026

Uh oh!

Copilot AI Feb 12, 2026

Uh oh!

Copilot AI Feb 12, 2026

Uh oh!

Copilot AI Feb 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

-    const contentLength = c.req.header('content-length');
-    if (contentLength && parseInt(contentLength, 10) > maxSize) {
-      return c.json(
-        { error: 'Payload too large', maxSize },
-,
-      );
-    }
+    const rawReq = c.req.raw;
+    // Fast-path: honor Content-Length when present
+   const contentLength = rawReq.headers.get('content-length');
+    if (contentLength && parseInt(contentLength, 10) > maxSize) {
+      return c.json(
+        { error: 'Payload too large', maxSize },
+,
+      );
+    }
+    // If there's no body, nothing to limit.
+    if (!rawReq.body) {
+      await next();
+      return;
+    }
+    // Enforce limit based on actual bytes read, regardless of transfer encoding.
+    const [limitStream, forwardStream] = rawReq.body.tee();
+    const reader = limitStream.getReader();
+    let totalBytes = 0;
+    // Read until stream ends or we exceed maxSize.
+    while (true) {
+      const { done, value } = await reader.read();
+      if (done) {
+        break;
+      }
+      if (value) {
+        totalBytes += value.byteLength;
+        if (totalBytes > maxSize) {
+          // Exceeded limit: stop reading and reject.
+          reader.releaseLock();
+          return c.json(
+            { error: 'Payload too large', maxSize },
+,
+          );
+        }
+      }
+    }
+    // Within limit: create a new Request with the untouched tee branch
+    // so downstream middleware/handlers can still consume the body.
+    const limitedReq = new Request(rawReq, { body: forwardStream });
+    // Hono's Context.req wraps the underlying Request; update its raw field.
+    // @ts-expect-error: accessing framework-internal property
+    (c.req as any).raw = limitedReq;

Conversation

Copilot AI commented Feb 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

M.1 — Critical Security

M.2 — Infrastructure

M.3 — Platform Hardening

Uh oh!

vercel bot commented Feb 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Feb 12, 2026 •

edited

Loading

vercel bot commented Feb 12, 2026 •

edited

Loading