Sovereign Core — Modular Architecture & File-System Based Routing

[kasunben]

Goal

Refactor and formalize the Sovereign Core architecture into a modular, scalable, and discoverable codebase.

Introduce file-system based routing for both API and web routes, with consistent conventions for middlewares, error handling, and dependency injection—allowing independent feature modules to evolve cleanly while preserving privacy and maintainability.

Outcomes / Deliverables

• Well-defined core module structure (core/, routes/, middleware/, services/, lib/).
• File-system based router that auto-registers routes under /routes/* with predictable URL mapping. #116
• Unified context loader (res.locals, req.context) exposing app config, user session, roles/capabilities.
• Consistent middleware chain for auth, permissions, logging, validation, and error handling.
• Separation between core framework code and feature modules (Auth, Projects, Boards, etc.).
• Comprehensive documentation and bootstrap script for new modules.

Scope

• Backend app structure (src/) re-organized into modular domains.

• Router autoloader (Express + dynamic imports) scanning /routes hierarchy.

• Convention mapping:

  /routes/api/users/get.mjs        → GET /api/users
  /routes/api/users/post.mjs       → POST /api/users
  /routes/web/projects/index.mjs   → GET /projects
  /routes/web/projects/blog/get.mjs → GET /projects/blog
  /routes/auth/login.mjs           → GET/POST /auth/login
  

• Middleware registry (global, per-route, per-scope).

• Built-in lifecycle hooks: beforeLoad(), afterLoad(), onError().

• Plug-and-play service injection (db, cache, mailer, etc.).

• Type-safe route definitions (auto-generated typing with Zod or TypeScript inference).

• Hot-reload in dev via Vite/ESBuild watcher.

User Stories

• As a developer, I can add a new route by simply creating a file in /routes without touching central router code.
• As a maintainer, I can disable or version a module by toggling a directory flag/config.
• As a security engineer, I can see exactly which middlewares protect which routes.
• As a contributor, I can build and test a new feature module without breaking core logic.

Architecture & Conventions

Core Directory Layout

src/
 ├── core/               # framework: loader, logger, error handler, DI container
 ├── routes/
 │    ├── api/
 │    │    ├── users/
 │    │    │    ├── get.mjs
 │    │    │    └── post.mjs
 │    │    └── projects/
 │    │         └── index.mjs
 │    ├── auth/
 │    │    ├── login.mjs
 │    │    └── register.mjs
 │    └── web/
 │         └── about.mjs
 ├── middleware/
 │    ├── auth.mjs
 │    ├── rateLimit.mjs
 │    ├── errorHandler.mjs
 │    └── ...
 ├── services/
 │    ├── db.mjs
 │    ├── mailer.mjs
 │    ├── cache.mjs
 │    └── ...
 ├── lib/
 │    ├── utils.mjs
 │    └── constants.mjs
 └── index.mjs

Routing Logic

// Example loader sketch
import { Router } from 'express';
import fs from 'node:fs/promises';
import path from 'node:path';

export async function loadRoutes(baseDir, prefix = '') {
  const router = Router();
  const entries = await fs.readdir(baseDir, { withFileTypes: true });

  for (const entry of entries) {
    const fullPath = path.join(baseDir, entry.name);
    if (entry.isDirectory()) {
      router.use(`/${entry.name}`, await loadRoutes(fullPath, `${prefix}/${entry.name}`));
    } else if (entry.isFile() && entry.name.endsWith('.mjs')) {
      const routeModule = await import(fullPath);
      const [method] = entry.name.split('.');
      if (typeof router[method] === 'function') {
        router[method === 'index' ? 'get' : method.toLowerCase()](
          method === 'index' ? '/' : '',
          ...(routeModule.middlewares || []),
          routeModule.default || routeModule.handler
        );
      }
    }
  }
  return router;
}

Tasks

• ADR-CORE-001: Define modular architecture principles (directory layout, naming, DI strategy).
• Bootstrap Script: CLI to scaffold new route/module with stub (yarn new:route api/users/get).
• Router Loader:
  • Recursively load /routes files into Express.
  • Support per-scope prefixing (/api, /auth, /web).
  • Register route methods from filenames (get.mjs, post.mjs, etc.).
  • Integrate middlewares defined in module exports.
• Middleware Registry:
  • Global: logging, security headers, rate limiting, error handler.
  • Scoped: api/* gets auth, JSON parser; web/* gets session + template engine.
• Dependency Injection: simple container (core/container.mjs) for shared services (db, cache, mailer).
• Error Handling: unified AppError class + centralized handler returning JSON or HTML.
• Type Safety: route definitions auto-typed; Zod schema validation integrated.
• Hot Reload (Dev): watcher re-imports changed route files.
• Tests: integration tests ensuring all route files load successfully, return correct responses.
• Docs:
  • "Creating a Route" guide.
  • Architecture overview diagram.
  • Coding standards (naming, imports, export shape).
  • Versioning strategy for modules.

Acceptance Criteria

• All routes autoload correctly from /routes with no manual registration.
• Unit/integration tests pass for API + web routes.
• Adding, modifying, or removing a file updates routes automatically in dev.
• Middleware composition works per scope (auth, api, web).
• Core remains independent of domain logic.
• Documentation complete; contributors can create new modules without mentoring.

Risks & Mitigations

Risk	Mitigation
Route loading performance on startup	Cache directory tree; lazy-load heavy modules.
Ambiguous filenames or conflicts	Enforce naming rules via linting; CI validation.
Middleware order confusion	Define fixed load order (global → scoped → local).
Security gaps in dynamic imports	Whitelist /routes root; reject symlinked or hidden files.
Breaking changes in structure	Maintain versioned loader; changelog per release.

Dependencies

• Observability Epic: structured logs for route load times and errors.
• Security Hardening Epic: middleware integration (headers, rate limiting).
• CI Gates Epic: ensures route autoloader test passes and no missing exports.

Follow-Ups (Post-MVP)

• Code-splitting and lazy import for inactive feature modules.
• Route metadata registry for docs and capability-based access checks.
• CLI to visualize module dependency graph.
• Versioned API support (/api/v1, /api/v2) using directory namespaces.

[kasunben]

Instance ID / Tenant ID

Each running instance of Sovereign should have a unique instance-id, as a future-proof configuration for federated network. Also, each instance has a default tenant (tenant-0) or more tenants if multi-tenancy enabled. Each User and Project object should have a tenant-id field and tenant-id value consist of two parts.

{
   tenantId: ${instance-id}/${tenant-id}
}

[kasunben]

• DI flow
• entry-point flow (project launcher, sidebar)
• build config entries (plugging, project types
• Add responsive support
• Fix: Check SPA styles
• Fix PaperTrail iPad Control
• Plugin Type: API
• Tenant Specific: Custom start page for tenant users
• Plugins: CRM
• Plugins: Multi-channel Inbox
• SPA Plugin: API Layer (REST, GraphQL, WS)
• Plugin Type: RSS aggregator
• Delete guest projects and data
• Plugin: Mailing list
• Subscription model, plugin enrol page per plugins
• Custom Domain Integration
• Plugin: Splitwise
• Plugin: Billing & Invoicing
• Plugin: Travel/itinerary Builder
• Add timestamps and instance-id to manifest.json