Skip to content

Split tests into modular files#6

Merged
PatrickJS merged 8 commits into
mainfrom
refactor-nested-folders-helpers
May 14, 2026
Merged

Split tests into modular files#6
PatrickJS merged 8 commits into
mainfrom
refactor-nested-folders-helpers

Conversation

@PatrickJS
Copy link
Copy Markdown
Owner

@PatrickJS PatrickJS commented May 14, 2026

Restructure the test suite by removing the large monolithic test/jsondb.test.js and splitting its contents into focused test files. Added new tests under test/cli, test/config, test/doctor, test/package, test/runtime, test/schema, and test/sync, and moved examples.test.js into test/examples/examples.test.js (updated import path). This improves maintainability and clarity by grouping related tests and reducing a single massive test file.

Summary by CodeRabbit

  • New Features

    • Custom source readers for extensible fixture format support
    • Doctor tool for data quality diagnostics (duplicate IDs, inconsistent types, relation suggestions)
    • REST response format customization per file extension
    • Schema manifest generation with resource/field customization hooks
    • Resource naming strategies for nested fixture folders
    • Runtime event streaming for live data changes
    • Multiple runtime adapter types (JSON, memory, source, static)
  • Documentation

    • Expanded README with source reader, REST format, and nested fixture documentation
    • New specification for source reader interface and behavior

PatrickJS added 8 commits May 13, 2026 23:31
Restructure the test suite by removing the large monolithic test/jsondb.test.js and splitting its contents into focused test files. Added new tests under test/cli, test/config, test/doctor, test/package, test/runtime, test/schema, and test/sync, and moved examples.test.js into test/examples/examples.test.js (updated import path). This improves maintainability and clarity by grouping related tests and reducing a single massive test file.
Allow fixtures to be organized in nested folders and propagate context for schema manifest generation.

- Discover source files recursively (listSourceFilesInDirectory) and return sorted relative paths.
- Derive resource names from fixture basenames (path.basename) so nested paths map to a single resource key.
- Enable recursive watching for sourceDir in both the CLI and server watchers so changes in nested folders are picked up.
- Pass resource/config/diagnostic/fieldPath context into itemManifest and use renderFieldMap for object items so nested array/object fields are rendered correctly in the schema manifest.
- Update README with docs and examples for nested fixture folders, schemaManifest.customizeField usage, sample generated manifest, and related notes (includes mention of schemaOutFile).

These changes make it possible to group fixtures by domain (e.g. db/cms/pages.json) while keeping simple resource routes and richer UI metadata in the generated schema manifest.
Introduce REST response formats and fixture-based resource naming strategies. Add config-public utilities (mergeManifest, parseFixturePath, resourceNameFromPath, resource naming strategies), expose them from index, and wire format parsing/rendering into the REST handler (support .json/.md/.html and custom formats, with default json fallback). Add resources.naming and resources.customizeResource handling when resolving source resource names, plus duplicate-resource diagnostics for nested fixtures. Update schema manifest to support customizeResource and mergeManifest-based field/resource customization with non-serializable diagnostics. Update README and types, and add tests for REST formats, mergeManifest, resource naming/customization, and duplicate resource detection.
Introduce a reader pipeline to support custom source formats. Adds config.sources.readers default and example, new types for reader context/results, and built-in readers for JSON/JSONC/CSV/.schema.mjs/.schema.json(.c). Refactor schema loading to use readSourceFile, normalize multi-source results, compute source hashes, and surface structured diagnostics for reader errors and invalid results. Update parseFixturePath to preserve generic extensions. Include README and SPEC docs for Source Readers and extensive tests covering reader behavior, ordering, multi-source returns, and error conditions.
Move schema-related functionality into modular files under src/features/schema (fields, generated, project, relations, resource, sources, validation). Adds robust source reading (CSV/JSON/JSONC/MJS), field inference/normalization, resource building, relation validation, seed normalization/id generation, and schema validation/constraints. Refactors src/schema.js to delegate to these new modules and centralizes generation of REST/GraphQL metadata and diagnostics to improve maintainability and testability.
Split the monolithic CLI and several core modules into smaller, focused files and feature namespaces. The top-level src/cli.js now delegates to src/cli/index.js (runCli) and command handlers under src/cli/commands (create, doctor, generate, schema, serve, sync, types), plus args and output helpers. Many implementation files were moved/rewired into features/ and shared/ (runtime, config, schema, sync, csv, db, doctor, generate/hono, integrations, etc.) with re-exports to preserve original public API surface. This refactor improves modularity, testability and maintainability while preserving backward-compatible imports.
Add a pluggable runtime system and event hub, replacing direct JSON state handling with runtime adapters. Implements createRuntime and built-in adapters (json, memory, static) plus atomic file writes and in-memory behavior; introduces createRuntimeEventHub for live event streaming. Refactor JsonDbCollection/JsonDbDocument/JsonFixtureDb to use the runtime adapters and emit runtime events on successful writes. Move JSON I/O and atomic write logic into src/features/storage/json.js and add memory/static adapters and tests. Add HTTP feature registry (including SSE runtime log and graphql handler) and wire it into the server with a /__jsondb/log endpoint. Add generator registry for CLI generate command and modularized schema metadata generation. Update default config to include runtime options and TypeScript definitions to expose runtime APIs and events. Add tests covering atomic write, runtimes, static read-only behavior, and live event emission.
Introduce a new 'source' runtime adapter that reads/writes plain JSON fixture files and exposes source-file persistence and live events. The adapter enforces writable JSON-backed resources, reuses json runtime state helpers, and populates source metadata for sync. Integrate the adapter into builtin runtimes, update syncJsonFixtureDb to hydrate the runtime and write source metadata, and simplify openJsonFixtureDb hydration logic. Also: update types to include 'source', make server and source discovery ignore dot-folders inside db, add tests for source writes and dot-folder ignoring, and add related utility functions. Removed legacy allResourcesUseDefaultJsonRuntime logic.
@PatrickJS PatrickJS merged commit 80ecfff into main May 14, 2026
3 of 4 checks passed
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 14, 2026

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 029d678e-48f6-4bb9-aeff-0c3e01f45ea6

📥 Commits

Reviewing files that changed from the base of the PR and between e1ab356 and 0dc4349.

📒 Files selected for processing (108)
  • README.md
  • SPEC.md
  • jsondb.config.example.mjs
  • src/cli.js
  • src/cli/args.js
  • src/cli/commands/create.js
  • src/cli/commands/doctor.js
  • src/cli/commands/generate.js
  • src/cli/commands/schema.js
  • src/cli/commands/serve.js
  • src/cli/commands/sync.js
  • src/cli/commands/types.js
  • src/cli/index.js
  • src/cli/output.js
  • src/config-public.js
  • src/config.d.ts
  • src/config.js
  • src/csv.js
  • src/db.js
  • src/doctor.js
  • src/errors.js
  • src/features/config/defaults.js
  • src/features/config/forks.js
  • src/features/config/load.js
  • src/features/config/public.js
  • src/features/doctor/duplicate-ids.js
  • src/features/doctor/field-consistency.js
  • src/features/doctor/index.js
  • src/features/doctor/relations.js
  • src/features/generate/registry.js
  • src/features/http/registry.js
  • src/features/runtime/collection.js
  • src/features/runtime/db.js
  • src/features/runtime/document.js
  • src/features/runtime/json-pointer.js
  • src/features/runtime/state.js
  • src/features/schema/builders.js
  • src/features/schema/fields.js
  • src/features/schema/generated.js
  • src/features/schema/manifest.js
  • src/features/schema/metadata.js
  • src/features/schema/project.js
  • src/features/schema/relations.js
  • src/features/schema/resource.js
  • src/features/schema/sources.js
  • src/features/schema/validation.js
  • src/features/storage/events.js
  • src/features/storage/json.js
  • src/features/storage/json.test.js
  • src/features/storage/memory.js
  • src/features/storage/runtime.js
  • src/features/storage/source.js
  • src/features/storage/static.js
  • src/features/sync/defaults.js
  • src/features/sync/index.js
  • src/features/sync/mirror-state.js
  • src/features/sync/runtime-dirs.js
  • src/features/sync/source-writes.js
  • src/features/sync/synthetic-seed.js
  • src/fs-utils.js
  • src/generate/hono.js
  • src/generate/hono/app.js
  • src/generate/hono/graphql.js
  • src/generate/hono/index.js
  • src/generate/hono/package.js
  • src/generate/hono/readme.js
  • src/generate/hono/repository.js
  • src/generate/hono/rest.js
  • src/generate/hono/schema.js
  • src/generate/hono/sqlite.js
  • src/generate/hono/validators.js
  • src/hono.js
  • src/index.d.ts
  • src/index.js
  • src/integrations/hono.js
  • src/integrations/sqlite.js
  • src/integrations/vite.js
  • src/jsonc.js
  • src/mock.js
  • src/names.js
  • src/rest/handler.js
  • src/rest/handler.test.js
  • src/schema-builders.js
  • src/schema-manifest.js
  • src/schema.js
  • src/server.js
  • src/server.test.js
  • src/shared/csv.js
  • src/shared/errors.js
  • src/shared/fs-utils.js
  • src/shared/jsonc.js
  • src/shared/mock.js
  • src/shared/names.js
  • src/sqlite.js
  • src/state.js
  • src/sync.js
  • src/vite.js
  • test/cli/cli.test.js
  • test/config/config.test.js
  • test/doctor/doctor.test.js
  • test/examples/examples.test.js
  • test/jsondb.test.js
  • test/package/exports.test.js
  • test/runtime/package-api.test.js
  • test/schema/manifest.test.js
  • test/schema/sources.test.js
  • test/schema/validation.test.js
  • test/sync/sync.test.js

📝 Walkthrough

Walkthrough

This PR undergoes a major architectural refactoring, extracting monolithic modules into a feature-organized structure under src/features/ and src/shared/, while simultaneously introducing new capabilities: configurable source readers for custom schema/data formats, REST response format negotiation per file extension, a "doctor" diagnostic system for fixture validation, nested fixture folder support with resource naming strategies, and enhanced schema manifest generation with customization hooks. The CLI is refactored from inline command handling to a modular command registry. Type definitions, tests, and integrations (Hono, SQLite, Vite) are reorganized accordingly.

Changes

Core Architectural Reorganization & Feature Addition

Layer / File(s) Summary
Configuration & Public API Refactoring
src/config.js, src/config.d.ts, src/config-public.js, src/features/config/*, jsondb.config.example.mjs, SPEC.md
Config loading moved to src/features/config/load.js with deep-merge helpers; public utilities (defineConfig, mergeManifest, resourceNameFromPath, parseFixturePath) added to src/features/config/public.js and re-exported. Config shape extended with sources.readers, resources, and runtime options; defaults centralized in src/features/config/defaults.js.
Source Reader Pipeline & Schema Loading
src/features/schema/sources.js, src/features/schema/project.js, SPEC.md, README.md
New module sources.js implements custom source reader support (user readers run before built-ins), source discovery/validation, and resource naming with hooks. Schema loading in project.js aggregates sources, validates relations, and produces diagnostics; new docs explain reader context/interfaces and nested fixture folder discovery.
Resource Schema Normalization & Validation
src/features/schema/resource.js, src/features/schema/fields.js, src/features/schema/validation.js
resource.js builds normalized resources from data/schema with CSV coercion and collection ID generation; fields.js normalizes field definitions and infers schemas from sample data; validation.js validates records/seeds against schema constraints (types, uniqueness, patterns, ranges).
Schema Manifest & Metadata Generation
src/features/schema/manifest.js, src/features/schema/metadata.js, src/features/schema/generated.js, src/schema-manifest.js
Manifest generation supports customizeResource/customizeField hooks with error handling and JSON-serializability validation; infers UI components/labels from field metadata; REST/GraphQL metadata composed via pluggable contributors. Generated schema includes version, resources, relations, and metadata.
Doctor & Schema Relations Validation
src/features/doctor/*, src/doctor.js
New doctor feature validates resource consistency (duplicate/mixed-type IDs, field typing), suggests relations, validates fork configurations, and emits structured findings; validateProjectRelations checks relation target existence/field types and seed value coverage.
CLI Reorganization
src/cli.js, src/cli/index.js, src/cli/args.js, src/cli/commands/*, src/cli/output.js
CLI refactored from monolithic to modular command pattern: src/cli/index.js routes command dispatch; individual command modules for sync, types (with --watch support), schema, doctor/check, create, serve, generate; output helpers centralized in output.js.
Runtime Database Interface
src/features/runtime/db.js, src/features/runtime/collection.js, src/features/runtime/document.js, src/features/runtime/json-pointer.js, src/db.js
JSON fixture DB implementation moved to modular runtime: db.js provides openJsonFixtureDb/JsonFixtureDb factory; collection.js implements CRUD with defaults/validation/unique constraints; document.js supports full-document/JSON-pointer operations; json-pointer.js provides RFC 6901 traversal.
Storage Adapter Registry & Implementations
src/features/storage/runtime.js, src/features/storage/json.js, src/features/storage/memory.js, src/features/storage/source.js, src/features/storage/static.js, src/features/storage/events.js
New pluggable runtime adapter registry selecting per-resource storage strategy (json/memory/source/static); JSON adapter persists state files with atomic writes and serialized write queues; memory adapter uses in-memory cloning; source adapter mirrors back to fixture files; static adapter is read-only. Event hub emits versioned change events.
Fixture Sync & Seeding
src/features/sync/*, src/sync.js
Sync orchestration now delegates to src/features/sync/index.js; new modules handle default application (safe migrations), synthetic seed generation from schema, runtime directory creation, and writing generated IDs back to fixture sources.
REST Format Negotiation
src/rest/handler.js, src/rest/handler.test.js, README.md
REST handler now parses format extension (.json, .md, .html) from resource paths and delegates to pluggable format renderers via config.rest.formats; supports string aliases, custom functions, and structured 404 errors for unknown formats. Tests cover format selection and rendering.
Hono Integration & Code Generation
src/integrations/hono.js, src/generate/hono/*, src/hono.js, src/generate/hono.js
Hono integration moved to src/integrations/hono.js providing createJsonDbHonoApp with REST/GraphQL route registration and error handling. New src/generate/hono/ modules generate Hono starter projects: SQLite adapter (with migrations and seeding), REST/GraphQL route handlers, validators, schema/seed modules, repository types, app entry, server entry, package.json, tsconfig, and README.
SQLite Integration
src/integrations/sqlite.js, src/sqlite.js
SQLite fixture DB moved to src/integrations/sqlite.js; provides SqliteJsonDb with collection/document repositories, CRUD operations with serialization/deserialization, atomic writes, and structured SQL table/type generation from resource definitions.
Vite Integration
src/integrations/vite.js, src/vite.js
Vite plugin moved to src/integrations/vite.js providing dev server integration, virtual module support for client code, fork validation, and dynamic routing.
Shared Utilities
src/shared/*, src/errors.js, src/csv.js, src/jsonc.js, src/fs-utils.js, src/names.js, src/mock.js
Common utilities extracted to src/shared/: error types/serialization (errors.js), CSV parsing (csv.js), JSONC parsing/stripping (jsonc.js), filesystem operations (fs-utils.js), naming/pluralization helpers (names.js), and mock delay/error behavior (mock.js). Entry points in src/ re-export from shared modules.
Type Definitions Expansion
src/index.d.ts, src/config.d.ts, src/index.js
TypeScript declarations expanded with types for source readers (JsonDbSourceReader, JsonDbSourceReaderContext, JsonDbSourceReaderResult), runtime capabilities/adapters, per-resource runtime options, runtime events, REST format renderers, and resource naming strategies. New helper function declarations for mergeManifest, resourceNameFromPath, parseFixturePath.
HTTP Feature Registry
src/features/http/registry.js, src/server.js, src/server.test.js
New pluggable HTTP feature registry (defaultHttpFeatureRegistry) for request handling in preMock and postMock phases; built-in features include runtime-log (SSE streaming via /__jsondb/log) and GraphQL. Server integrates registry-driven request handling and ignores dot-prefixed source directories.
Comprehensive Test Coverage
test/cli/cli.test.js, test/config/config.test.js, test/doctor/doctor.test.js, test/package/exports.test.js, test/runtime/package-api.test.js, test/schema/*.test.js, test/sync/sync.test.js
New integration test suites covering CLI argument handling, config loading/merging, doctor findings and fork validation, runtime CRUD/concurrency/events, schema manifest customization, source reader pipelines, field validation/constraints, and fixture sync with nested folders. Old monolithic test/jsondb.test.js removed.

Sequence Diagram(s)

sequenceDiagram
    participant CLI
    participant ConfigLoader
    participant SourceReaders
    participant SchemaBuilder
    participant RuntimeAdapter
    participant HTTPServer
    
    CLI->>ConfigLoader: loadConfig(options)
    ConfigLoader->>ConfigLoader: Parse --cwd, --config
    ConfigLoader->>ConfigLoader: Merge defaults + user config
    
    CLI->>SourceReaders: Discover & read sources from db/
    SourceReaders->>SourceReaders: Custom readers run first
    SourceReaders->>SourceReaders: Fall back to built-in readers
    SourceReaders->>SchemaBuilder: Return schema/data sources
    
    SchemaBuilder->>SchemaBuilder: Build resources from sources
    SchemaBuilder->>SchemaBuilder: Validate relations & seeds
    SchemaBuilder->>SchemaBuilder: Generate schema metadata
    
    CLI->>RuntimeAdapter: Create runtime (json/memory/source/static)
    RuntimeAdapter->>RuntimeAdapter: Hydrate resources
    RuntimeAdapter->>RuntimeAdapter: Emit versioned events on writes
    
    HTTPServer->>HTTPServer: Register HTTP feature registry
    HTTPServer->>HTTPServer: preMock: runtime-log feature
    HTTPServer->>HTTPServer: postMock: GraphQL feature
    HTTPServer->>HTTPServer: Handle REST with format negotiation
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

  • PatrickJS/jsondb#4: Adds schema manifest generation with customizeField and customizeResource hooks, now integrated into src/features/schema/manifest.js and CLI schema manifest command.
  • PatrickJS/jsondb#5: Extends schema manifest features and CLI schema validation, building on the manifest infrastructure reorganized in this PR.
  • PatrickJS/jsondb#3: Implements doctor/check functionality for fixture validation, now refactored into src/features/doctor/* and wired into the CLI with exit-code control.

Poem

🐰 Modules spring forth in organized rows,
From monoliths scattered to gardens of code,
Readers and formats and doctors now grow,
With source and with schema along the bright road.
A storage adapter for every tale,
And CLI commands that will never fail! 🌿

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch refactor-nested-folders-helpers

@PatrickJS PatrickJS deleted the refactor-nested-folders-helpers branch May 14, 2026 08:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant