P1: Technical hardening for DB-backed configuration, coding standards, and modularity

[Joncallim]

Summary

P1 technical hardening issue to improve Forge's code readability, modularity, scalability, and default coding standards before the next Workforce execution slices.

The main architectural direction is: move operator/domain configuration out of hardcoded TypeScript and into DB-backed editable records wherever the value changes over time. Keep TypeScript constants only for true protocol invariants, security allowlists, and language-level type safety.

This issue should be implemented as small PRs, but tracked as one maintainability initiative.

Scope

This should cover:

• DB-backed editable provider presets, model recommendations, capability taxonomy, MCP capability data, prompt templates, and Forge coding standards.
• Built-in coding standards so Forge-generated plans, handoffs, reviews, and documentation consistently recommend readable, modular, scalable code.
• Removal or reduction of hardcoded model/provider/role/status/capability/UI values where DB-backed records are a better source of truth.
• Refactoring large route/page modules, including web/app/dashboard/tasks/[id]/page.tsx, not just provider setup files.
• Language-level improvements such as enums, discriminated unions, satisfies, Zod validation, adapter maps, and a TOML parser dependency if it improves correctness and maintainability.

Guiding source-of-truth rule

Use the right source of truth for each kind of value:

• DB-backed editable records for runtime/operator configuration:
  • provider presets
  • model recommendations
  • capability taxonomy
  • MCP capability catalog
  • prompt templates
  • agent/workforce defaults
  • Forge coding/documentation standards
  • UI-visible labels/descriptions that should evolve without code edits
• TypeScript constants and type-level constructs for true invariants:
  • protocol fence names
  • security allowlists
  • adapter function names
  • internal event names where changing them would be a breaking code change
• Database enums/check constraints for persisted lifecycle/status values.
• Migrations and seed scripts for initial/default records.

Hardcoded values are acceptable when they are genuine invariants. Hardcoded defaults, catalogs, recommendations, prompt instructions, provider metadata, capability lists, and generated-document standards should generally become DB-backed records.

Findings and recommendations

1. Move provider presets and model recommendations into DB-backed editable records

web/lib/recommendations.ts currently holds shipped presets, role-to-model recommendations, model IDs, estimated costs, notes, and provider assignments in TypeScript.

Recommendation:

• Add DB tables for provider presets and recommendation records, for example:
  • provider_presets
  • provider_preset_agents
  • model_recommendations
• Seed current presets/recommendations through migrations or seed scripts.
• Preserve current UI behavior, but read from the DB instead of static imports.
• Keep validation with Zod or equivalent schema validation.
• Record recommendation provenance/version fields so defaults can be upgraded safely.

Acceptance target:

• A provider/model recommendation can be added, archived, reordered, or edited without changing dashboard TypeScript.
• Existing presets continue to work after migration.

2. Consolidate provider metadata and isolate provider runtime behavior

Provider type arrays, labels, base URL requirements, categories, placeholders, default URLs, setup links, and runtime factory behavior are split across provider type/catalog/registry code.

Recommendation:

• Move editable provider metadata to DB-backed records.
• Keep runtime-specific adapter behavior in code.
• Introduce a provider adapter registry rather than growing switches across files.

Example shape:

type ProviderAdapter = {
  type: ProviderType
  createFactory(config: ProviderConfig): ProviderFactory
  normalizeBaseUrl?: (url: string | null | undefined) => string | undefined
  supportsChatCompletions?: boolean
  supportsModelListing?: boolean
}

const PROVIDER_ADAPTERS = {
  anthropic: anthropicAdapter,
  openai: openAiAdapter,
  ollama: ollamaAdapter,
  lmstudio: lmStudioAdapter,
} satisfies Record<ProviderType, ProviderAdapter>

Acceptance target:

• Metadata-only provider changes do not require touching runtime factory code.
• Behavior-specific providers require one adapter plus seeded metadata.

3. Add DB-backed Forge coding and documentation standards

Forge should carry its own coding standards so generated plans, handoffs, reviews, and documentation consistently meet the same quality bar.

Recommendation:

• Add DB-backed standards records, for example:
  • forge_standards
  • forge_standard_rules
  • forge_standard_profiles
• Seed at least one default profile such as p1-technical-hardening or default-code-quality.
• Standards should cover:
  • readability
  • modularity
  • scalability
  • avoiding hardcoded values when DB/config is appropriate
  • small reversible changes
  • testability
  • validation and error handling
  • security-sensitive handling of secrets, auth, filesystem, GitHub, and command execution
  • documentation quality
• Inject active standards into Architect prompts, work package handoffs, QA prompts, Reviewer prompts, and documentation output.
• Store the applied standards profile/version in artifact metadata.

Acceptance target:

• Every Forge-generated implementation plan, handoff, review artifact, and documentation artifact can reference the active standards profile.
• Generated documents should explicitly check whether code avoids unnecessary hardcoded values and follows the repository's modularity/readability standards.

4. Replace text-only status comments with enforced status types

The schema uses multiple text() status fields with comments describing allowed values.

Recommendation:

• Use Drizzle pgEnum or database check constraints for:
  • task status
  • task attempt status
  • work package status
  • review requirement
  • agent run status
  • artifact type
  • approval gate status
  • VCS change status
  • repository command risk class
  • task question status
• Export shared TypeScript unions from the same source used by schema/API/UI.
• Replace raw status-string UI logic with typed status maps.

Acceptance target:

• Invalid statuses fail at compile time where possible and at the database layer where necessary.

5. Reduce duplicated defaults across TypeScript and SQL literals

Example: project MCP defaults exist as both a TypeScript object and a raw SQL JSON default.

Recommendation:

• Centralize defaults in DB-backed seed records where editable.
• Where a DB default must exist, add tests that assert runtime defaults and DB defaults remain aligned.
• Avoid copy/pasting JSON blobs across schema, migrations, UI, and docs.

Acceptance target:

• Runtime defaults are declared once or verified by tests if unavoidable duplication remains.

6. Refactor queue definitions and worker runtime processing

web/worker/queue.ts hardcodes Redis queue names for tasks, approvals, and answers. web/worker/runtime.ts repeats claim/process/retry/dead-letter/ack logic per queue type.

Recommendation:

• Add queue definition records/config, for example:

const QUEUE_DEFINITIONS = {
  tasks: {
    queueKey: 'forge:tasks',
    processingKey: 'forge:tasks:processing',
    retryKey: 'forge:tasks:retry',
    deadKey: 'forge:tasks:dead',
    claimsKey: 'forge:tasks:claims',
  },
  approvals: { ... },
  answers: { ... },
} satisfies Record<string, QueueDefinition>

• Keep Redis key names as stable protocol-ish values, but define them once and reuse everywhere.
• Use Zod schemas for job payload validation instead of repeated manual parse checks.
• Extract a generic processClaimedJob() helper for:
  • start attempt
  • processor call
  • finish attempt
  • retry/dead-letter
  • ack/final cleanup
• Move retry/backoff/timeouts into validated runtime config.

Acceptance target:

• Adding another queue does not require copy-pasting another full claim/process/retry block.

7. Move Architect prompts, capability taxonomy, and MCP capability data into DB-backed editable records

The Architect prompt builder embeds long instruction text, required JSON block examples, capability strings, MCP IDs, MCP permission examples, and output-shape guidance directly in TypeScript.

Recommendation:

• Add DB-backed prompt/template records, for example:
  • prompt_templates
  • prompt_template_versions
  • capability_catalog
  • mcp_capability_catalog
• Seed current Architect instructions and schemas as initial records.
• Store active template/schema/catalog versions in generated artifact metadata.
• Keep parser/validator logic in TypeScript, but load catalog values from the DB.
• Use Zod or JSON Schema validation for planner JSON blocks.

Acceptance target:

• Updating the Architect instruction text, capability taxonomy, or MCP capability examples does not require editing web/worker/orchestrator.ts.

8. Keep security-sensitive allowlists explicit and tested

Not every value should become user-editable.

Recommendation:

• Keep security-sensitive allowlists explicit, typed, and tested.
• Examples likely to remain code-level or strongly guarded:
  • approved provider adapter names
  • safe env var names
  • forbidden GitHub/merge capabilities
  • filesystem execution boundaries
  • protocol fence names
  • secret-handling rules
• If editable records influence security-sensitive behavior, add validation and review gates.

Acceptance target:

• DB-backed flexibility does not weaken auth, secrets, filesystem, command execution, GitHub token handling, or merge automation controls.

9. Split large dashboard pages into focused modules, including task detail

The roadmap already identifies the large dashboard pages. This issue should explicitly include the task detail page in the first implementation wave.

Refactor these files:

• web/app/dashboard/providers/page.tsx
• web/app/dashboard/agents/page.tsx
• web/app/dashboard/tasks/[id]/page.tsx

Suggested module split:

For providers:

• ProviderList
• ProviderForm
• ProviderHealthIndicator
• ProviderDiscoveryPanel
• useProviders
• useProviderHealth
• useModelListing
• provider-status-view-model.ts

For agents/workforces:

• AgentEditor
• WorkforceEditor
• RecommendedProviderPicker
• useAgentsAndWorkforces
• agent-form-schema.ts
• workforce-form-schema.ts

For task detail:

• TaskHeader
• RunTimeline
• ArtifactViewer
• OpenQuestionsPanel
• WorkforcePanel
• ApprovalGateControls
• CommandAuditPanel
• useTaskDetail
• task-status-view-model.ts

Where practical, use server-owned initial data and focused client components for mutation/refresh instead of keeping entire pages as large client components.

Acceptance target:

• Page files become route shells.
• Domain parsing/formatting logic is testable outside React components.
• Status and artifact rendering uses typed maps instead of repeated conditionals.

10. Split seed scripts into parser, policy, DB write, and export modules

web/db/seed-agents.ts currently handles prompt discovery, prompt copying/backups, legacy Claude prompt parsing, custom TOML parsing, DB upserts, default workforce creation, and workspace export.

Recommendation:

• Split into:
  • prompt file discovery
  • prompt parser
  • seed manifest loader
  • DB upsert service
  • workspace export service
• Move default agent/workforce/provider data into DB-backed seed records.
• Add a small TOML parser dependency if it improves correctness and reduces bespoke parsing risk.
• Continue validating parsed prompt files with Zod or an equivalent schema.

Acceptance target:

• Default agents/workforces/providers can be changed through seed records/migrations.
• Parser behavior has isolated tests.
• DB upsert behavior has isolated tests.

11. Centralize runtime magic values

Examples include worker claim timeouts, max attempts, stuck-job recovery interval, provider health interval, retry backoff cap, LM Studio load/list timeouts, generation temperature, and UI preview/truncation limits.

Recommendation:

• Create a runtime config module such as web/lib/config/runtime-defaults.ts.
• Validate env overrides once.
• Use descriptive names for values that remain in code.
• Move operator-tunable values into DB-backed settings or env-backed runtime config.

Acceptance target:

• No unexplained numeric/string literals in shared worker/provider/orchestrator paths.

Language-level improvements to use where appropriate

• satisfies for static adapter/config maps to preserve literal types while checking shape.
• as const plus derived unions for true code-level invariants.
• Zod schemas for API bodies, queue job payloads, prompt/template inputs, planner JSON blocks, and seed data.
• Drizzle pgEnum or DB check constraints for persisted lifecycle states.
• Discriminated unions for queue jobs, provider adapters, artifacts, approval decisions, and task lifecycle events.
• Adapter maps instead of large switches where the set is extensible.
• React server components for initial data where useful, with focused client mutation components.
• Status-to-view-model maps for labels, colors, and icons.
• A TOML parser dependency if it removes custom parser risk and improves maintainability.

Suggested implementation phases

Phase 1 — Standards and source-of-truth foundations

• Add DB-backed Forge standards profile and rules.
• Add runtime defaults/config module.
• Add shared status/domain constants or Drizzle enum modules.
• Add validation helpers for DB-backed config/templates.

Phase 2 — Provider and recommendation data

• Add DB-backed provider preset/recommendation tables.
• Migrate existing TypeScript presets/recommendations into seed data.
• Consolidate provider metadata and introduce provider adapter registry.

Phase 3 — Schema and status safety

• Add DB enums/check constraints.
• Replace raw status strings across API, worker, and UI with shared typed constants.
• Add migration and regression tests.

Phase 4 — Worker modularity

• Add queue definition catalog.
• Add generic claim/process/retry/dead-letter helper.
• Replace repeated runtime queue blocks.
• Add focused retry/dead-letter tests.

Phase 5 — Architect prompt, standards injection, and Workforce routing data

• Move Architect prompt/templates to DB-backed prompt records.
• Move capability taxonomy and MCP capability examples to DB-backed catalogs.
• Inject active Forge standards into Architect, handoff, QA, Reviewer, and documentation prompts.
• Store standards/template/schema versions in artifact metadata.

Phase 6 — Dashboard modularity, including task detail

• Split providers, agents/workforces, and task detail pages into feature modules.
• Extract data hooks and view-model helpers.
• Keep current UX behavior while making components easier to test.

Acceptance criteria

• Provider/model recommendations are no longer maintained as one large hardcoded TypeScript object.
• Provider metadata has one source of truth; runtime behavior is isolated in adapters.
• Forge coding/documentation standards are DB-backed, active, versioned, and applied to generated plans/handoffs/reviews/docs.
• Status values are enforced through TypeScript and database constraints.
• Queue definitions are declared once and reused by runtime, queue classes, tests, and docs.
• Architect capability/MCP catalogs are not embedded inside the long prompt-building function.
• Large dashboard pages, including tasks/[id]/page.tsx, are split into feature modules and route shells.
• Seed defaults for agents/workforces/providers are data-driven and migration/seed-backed.
• Runtime magic values are named, validated, and documented.
• Existing behavior remains intact.
• These commands pass from web/:
  • npm run lint
  • npx tsc --noEmit
  • npm test
  • npm run build

Reviewed anchors

• README.md
• AGENTS.md
• docs/developer-guide.md
• docs/roadmap.md
• web/lib/recommendations.ts
• web/lib/providers/types.ts
• web/lib/providers/catalog.ts
• web/lib/providers/registry.ts
• web/db/schema.ts
• web/db/seed-agents.ts
• web/db/seed-providers.ts
• web/worker/queue.ts
• web/worker/runtime.ts
• web/worker/orchestrator.ts
• web/worker/capability-classification.ts
• web/worker/mcp-execution-design.ts
• web/app/dashboard/providers/page.tsx
• web/app/dashboard/agents/page.tsx
• web/app/dashboard/tasks/[id]/page.tsx

Notes

This issue intentionally pushes straight toward DB-backed editable records rather than an intermediate reference-JSON source, because Forge's direction is an operator-controlled orchestration product. JSON files can still be useful for migration fixtures, tests, or seed manifests, but the runtime/editor source of truth should be the database.