Implementation Blueprint - Full System Architecture

[NalinDalal]

DriftLock Implementation Blueprint

Overview

Build a complete API drift detection system: parser → agent → CLI+UI → git tracker → Docker sandbox → auto-fix PRs.

---

Tech Stack

Layer	Technology
Runtime	Bun
Monorepo	Turborepo + Bun workspaces
Parser	tree-sitter + @babel/parser
Agent	OpenAI/Anthropic API
CLI	Bun + Commander.js
UI	Next.js 14 (App Router)
Database	PostgreSQL + Drizzle ORM
Queue	BullMQ + Redis
Sandbox	Docker + dockerode
Git	simple-git + Octokit

---

Implementation Steps

Step 1: Project Scaffolding

• Bun monorepo with workspaces
• Turborepo pipeline config
• Core types: CallSite, Snapshot, DriftEvent, Fix
• Docker Compose (PostgreSQL + Redis)

Step 2: Parser Package

• Tree-sitter TypeScript extractor
• stripe.* call site detection
• Request/response shape inference
• Endpoint mapping

Step 3: Git Tracker

• Change detection (added/modified/deleted)
• Call site diff tracking
• Snapshot comparison

Step 4: Sandbox

• Docker container orchestration
• HTTPS proxy for traffic capture
• Test classification (mock vs real)
• Resource limits + timeouts

Step 5: Agent

• OpenAI-powered change analysis
• Fix generation with alternatives
• Confidence scoring

Step 6: CLI

• driftlock analyze - Find API calls
• driftlock test - Sandbox execution
• driftlock diff - Compare snapshots
• driftlock fix - Generate fixes

Step 7: Web Dashboard

• Overview page (drift events, coverage)
• Code viewer with diff highlighting
• Fix preview (before/after)
• GitHub OAuth

Step 8: PR Generation

• Template-based PRs
• Branch management
• Auto-cleanup on merge

---

Open Questions

1. Multi-language support — TypeScript first, then Python. How to structure the extractor interface?
2. Sandbox security — Network isolation approach? Allowlist vs blocklist?
3. Fix confidence — When to auto-generate vs ask for confirmation?
4. Pricing model — Per-repo? Per-drift-event? Usage-based?

---

Next Actions

• Create custom discussion categories (Architecture Decisions, Implementation Plans)
• Complete Step 1: Project Scaffolding
• Test parser against 3-5 open-source Stripe integrations
• Design proxy capture format

---

Created: $(date '+2026-09-12')
Last Edited: $(date '+2026-09-14')
Status: Building

[NalinDalal]

Implementation Log — Step 1: Project Scaffolding

Status: Partially complete
Date: 2026-09-14
Related PR: #6 — fix configs

Objective

Create the initial DriftLock monorepo structure and define the shared contracts that later services will use.

Completed Work

1. Bun monorepo with workspaces

The root package.json defines the Bun package manager:

"packageManager": "bun@1.3.11"

The repository uses Bun workspaces:

"workspaces": [
  "packages/*",
  "apps/*"
]

The current workspace packages are:

• @driftlock/cli
• @driftlock/core
• @driftlock/agent
• @driftlock/git
• @driftlock/parser
• @driftlock/sandbox
• @driftlock/tests

This structure separates shared domain contracts, parsing, Git operations, sandbox execution, AI-agent logic, testing, and the CLI application.

2. Turborepo pipeline configuration

The repository includes turbo.json.

The pipeline defines tasks for:

• build
• dev
• test
• lint
• typecheck

The build task depends on build tasks from workspace dependencies and stores output in dist/**.

This configuration provides a common task interface for all packages and allows Turborepo to coordinate workspace commands.

3. Core domain types

The shared domain types are defined in packages/core/types.ts.

The primary types are:

• CallSite
  Represents an external API call found in repository source code.

• Snapshot
  Represents captured request and response information from a test run.

• DriftEvent
  Represents a detected difference between an old snapshot and a new snapshot.

• Fix
  Represents a generated remediation proposal for a drift event.

Supporting types also include:

• DiffSummary
• Repository
• AnalysisResult
• TestClassification
• CoverageReport

These types define the data contract between the parser, Git tracker, sandbox runner, agent, CLI, and test packages.

Pending Work

4. Docker Compose for PostgreSQL and Redis

This item is not complete.

The repository does not currently contain a Docker Compose configuration file. PostgreSQL and Redis services are therefore not yet available through local container orchestration.

The next implementation task is to add a Compose file that defines:

• A PostgreSQL service for persistent DriftLock data.
• A Redis service for caching, queues, or job coordination.
• Named volumes for persistent local development data.
• Environment-variable configuration for service credentials and ports.
• Health checks for service readiness.

Notes and Rationale

The project starts with shared types because each subsystem must use the same data model. This reduces contract drift between source analysis, test execution, drift detection, and automated fix generation.

The workspace structure keeps each subsystem independently maintainable while allowing shared local dependencies through Bun workspaces.

The Turborepo pipeline provides a single build, test, lint, and type-check workflow across the monorepo.

Next Step

Complete the Docker Compose configuration for PostgreSQL and Redis. Then verify that all workspace entry points, lint tasks, and database configuration are executable in CI.

[NalinDalal]

TODO: Add a demo video at last of project, so that product is easily understandable

[NalinDalal]

Implementation Log 001: Project Scaffolding and Initial Components

Date: 2026-09-13
Baseline: main, merge commit 73ce555
Related PR: #6: fix configs
Status: Core scaffolding implemented; local service setup remains outstanding.

Step 1: What is complete

• Bun monorepo with workspaces

  • Configured apps/* and packages/* workspaces with Bun 1.3.11.
  • Established shared packages for core types, parsing, Git tracking, sandbox execution, agent functionality, and tests, plus the CLI application.
  • Why: Keep related components together while maintaining separate responsibilities and reusable internal packages.
  • Reference: package.json:5.

• Turborepo pipeline configuration

  • Defined build, development, test, lint, and typecheck tasks.
  • Builds declare upstream build dependencies and dist/** outputs; development tasks are persistent and uncached.
  • Why: Provide consistent task orchestration and dependency-aware build caching across workspaces.
  • Reference: turbo.json:4.
  • Note: Pipeline declarations do not, by themselves, mean every package implements every task.

• Shared core types

  • CallSite: where and how application code calls an API.
  • Snapshot: captured request/response shapes and execution metadata.
  • DriftEvent: differences between snapshots, confidence, and resolution status.
  • Fix: a proposed correction, its diff, affected files, and confidence.
  • Added supporting types for repositories, diff summaries, analysis, test classification, and coverage.
  • Why: Give components consistent data contracts before connecting the full detection and remediation workflow.
  • References: packages/core/types.ts:1, :17, :29, and :65.

• Docker Compose for PostgreSQL + Redis

  • Not present in the merged repository.
  • The quick-start guide references a Compose file that does not exist.
  • CI provisions PostgreSQL 16, but that is not equivalent to local PostgreSQL + Redis setup.
  • Intended role: PostgreSQL for persistent application data; Redis for queued/background work.
  • References: QUICKSTART.md:19, .github/workflows/ci.yml:12.

Additional implementation already started

These are partial implementations, not completed blueprint milestones:

Area	Present today	Still outstanding
Parser	TypeScript AST traversal, basic stripe.* detection, endpoint mappings, initial request-shape extraction	Complete shape inference and response-field extraction
Git tracker	File change detection, explicit base comparisons, basic Git operations	Historical call-site comparison and API snapshot comparison
Sandbox	Docker execution scaffold with resource limits, timeout handling, and cleanup	Integrated traffic capture, endpoint restrictions, and test classification
Agent	Initial change-analysis and fix-suggestion methods	Robust response validation and integration into the end-to-end workflow
CLI	Initial analyze, test, diff, and init commands	Snapshot-based drift analysis and working fix command

The CLI currently reports Git file changes rather than API snapshot differences. Its fix command explicitly reports that it is unavailable instead of claiming success.

The web dashboard and automated PR generation remain outstanding.

Stabilization included in PR #6

• Corrected workspace entrypoints, runtime dependencies, and test/coverage command paths.
• Improved analysis-response parsing and explicit Git base comparisons.
• Removed credential collection and plaintext credential storage from initialization.
• Disabled networking in CLI-launched sandboxes until endpoint restrictions are enforced.
• Removed premature database configuration and the inactive CI lint step.
• Added tests and improved isolation of external network and Docker dependencies.

Why: Make the initial components more reliable while avoiding unsafe defaults, broken configuration, and misleading success messages.

Validation

The CI job on PR #6 completed successfully before merge.

This confirms the configured CI checks passed, not that the complete drift-detection system is implemented or production-ready.

Follow-up

• Add and verify PostgreSQL + Redis Compose setup.
• Correct the blueprint checklist and setup documentation to match the repository.
• Complete parser request/response extraction and validate against real Stripe integrations.
• Implement snapshot persistence and comparison before connecting automated fixes.

Milestone conclusion: The shared project foundation is in place, and several later components have initial implementations. Step 1 should remain partially complete until local PostgreSQL + Redis setup is delivered.

[NalinDalal]

Database wiring for installations:

• installations table in the schema (we have repositories but not the GitHub App installation itself)
• Webhook handlers don't save to database
• Webhook server doesn't connect to database

[NalinDalal]

@NalinDalal

Implementation Log 001: Project Scaffolding and Initial Components

Date: 2026-09-13
Baseline: main, merge commit 73ce555
Related PR: [#6: fix configs](https://github.com/nerdev...

Implementation Log 002: Database Foundation and Product Research

Date: 2026-09-15
Baseline: main, merge commit 304305c, compared with previous baseline 73ce555
Related PR: #7: db and some more research
Status: Merged. PostgreSQL configuration, database schema, and migration tooling are implemented; application persistence and Redis remain outstanding.

What is complete

• Database workspace package

  • Added @driftlock/db using Drizzle ORM and the postgres client.
  • Implemented cached, lazy initialization through getDb(), which requires DATABASE_URL.
  • Added createDb(connectionString) for independently configured instances, plus schema and database-type exports.
  • Why: Establish a shared database access layer before connecting application components to persistent storage.
  • References: packages/db/package.json:2, packages/db/index.ts:7.

• Initial PostgreSQL schema

  • repositories: repository identity, installation ID, default branch, languages, and analysis timestamps.
  • call_sites: repository-linked API usages, locations, endpoints, request shapes, response fields, and associated tests.
  • snapshots: captured request/response shapes and test-execution metadata.
  • drift_events: links to old/new snapshots, diff summaries, suggested fixes, confidence, PR numbers, and status.
  • Added enums, foreign keys, defaults, and cascading deletion on selected parent relationships.
  • Why: Provide the storage structure needed for historical API observations and drift tracking.
  • References: packages/db/schema.ts:39, packages/db/schema.ts:52, packages/db/schema.ts:70, packages/db/schema.ts:84.
  • Note: Defining these tables does not mean the application currently writes or compares snapshots.

• Migration and database-development tooling

  • Added PostgreSQL Drizzle configuration and a checked-in initial SQL migration with migration metadata.
  • Added db:generate, db:migrate, db:push, and db:studio package scripts.
  • Why: Make schema changes reproducible and provide an explicit workflow for applying and inspecting them.
  • References: packages/db/drizzle.config.ts:3, packages/db/package.json:11, packages/db/migrations/0000_salty_rumiko_fujikawa.sql:1.
  • Note: Migration files and commands are present; successful migration execution was not demonstrated by the configured CI checks.

• Local PostgreSQL Compose configuration

  • Added a root-level Compose file using PostgreSQL 16.
  • Configured a named volume for database persistence and published port 5432.
  • Added a matching DATABASE_URL example.
  • Why: Supply the local PostgreSQL configuration missing from the previous milestone.
  • References: docker-compose.yml:1, .env.example:1.
  • Note: This is PostgreSQL-only configuration, not a verified PostgreSQL + Redis setup.

Research and documentation added

• Updated the README with the intended discover, classify, probe, compare, fix, and report workflow.
• Added competitive research covering other approaches, potential differentiators, and ideas such as scheduled scanning, Git co-change analysis, and stricter fix validation.
• Updated the YC application’s product description.
• Why: Clarify the product direction and record research that can inform implementation priorities.
• References: README.md:47, docs/competitive-analysis.md:21, docs/competitive-analysis.md:204, docs/yc-application.md:7.

These are documentation and research changes, not evidence that the described monitoring, classification, or automated PR workflow is implemented. Competitor claims were not independently revalidated for this log.

Integration status after PR #7

Area	Present today	Still outstanding
Database access	Schema, connection helpers, and migration tooling	Application consumers and database-backed persistence operations
Local services	PostgreSQL Compose configuration and persistent volume	Redis, readiness checks, and verified setup instructions
Snapshots and drift	Storage tables and existing sandbox scaffold	Integrated traffic capture, snapshot writes, and API-shape comparison
Fix workflow and dashboard	Existing suggestion scaffold and expanded product documentation	Working CLI fixes, automated PR generation, and web dashboard

No current application code consumes the new database package. The sandbox still returns an empty traffic-capture array, and CLI diff still reports Git file changes rather than API snapshot differences. The fix command explicitly remains unavailable.

References: packages/sandbox/runner.ts:80, apps/cli/index.ts:147, apps/cli/index.ts:194.

Stabilization included in PR #7

• Corrected an export regression introduced during the PR so SandboxRunner remains a directly constructable class.
  • Reference: packages/sandbox/index.ts:2.
• Removed test and test:watch scripts from the CLI, agent, core, Git, parser, and sandbox package manifests. Existing centralized test commands remain in packages/tests.
  • References: packages/tests/package.json:11, package.json:13.
• No new test files were added in this PR.

Why: Restore compatibility with sandbox callers and retain test execution through the existing centralized test package.

Validation

The CI job on PR #7 completed successfully on final PR commit 1b44213 before merge.

The configured typecheck, build, unit-test, integration-test, E2E-test, and coverage steps passed.

This does not verify the new Compose setup or migration: CI provisions PostgreSQL separately, contains no migration step, and the existing tests do not exercise the new database package. No local services or migrations were run for this log.

References: .github/workflows/ci.yml:12, .github/workflows/ci.yml:40, packages/tests/package.json:18.

Follow-up

• Add Redis to complete the originally planned local service setup.
• Verify PostgreSQL startup, migration application, and persistence across restarts; add database integration tests.
• Restrict the development database port to localhost and add readiness checks.
• Correct QUICKSTART.md:19, which still points to docker/docker-compose.yml and promises Redis.
• Update stale package-specific test commands in QUICKSTART.md:106 and distinguish implemented features from planned capabilities.
• Connect call-site and snapshot persistence, then implement snapshot comparison before automated fixes.

Milestone conclusion: PR #7 establishes the database foundation and supplies the previously missing PostgreSQL Compose configuration. Step 1 remains partially complete because Redis and local setup verification are outstanding; the end-to-end drift-detection and remediation workflow is not yet implemented.