LineageGuard

Turn DataHub impact evidence into a deterministic schema-change decision and a validated migration pull request.

LineageGuard protects analytical data-platform changes before merge. It combines DataHub lineage, query usage, ownership, and ML metadata with deterministic policy, bounded LLM planning, exact canonical artifact generation, executable validation, a draft GitHub pull request, and verified DataHub write-back.

Accepted LIVE evidence: ALLOW → BLOCK · 4 consumer groups · 10 DataHub evidence items · 8 artifacts · 8/8 PASS · Draft PR #7 · verified write-back.

From a local change to organizational impact

The canonical scenario starts with a warehouse-field rename in source PR #3:

ALTER TABLE commerce.orders
RENAME COLUMN customer_id TO buyer_id;

Looking only at the proposed change, the baseline returns ALLOW. DataHub then adds the organizational context missing from that view:

commerce.orders.customer_id
  └─ analytics.stg_orders.customer_id
       ├─ analytics.customer_revenue
       │    └─ Finance Revenue Dashboard
       └─ fraud.customer_features.customer_id
            └─ Fraud Model v3

Observed query evidence: finance-monthly-close.sql

LineageGuard normalizes this into four consumer groups and ten evidence items. Its deterministic policy evaluates rules LG001–LG004 and changes the original in-place rename to BLOCK.

BLOCK is the LineageGuard policy decision for the unsafe rename—not GitHub branch protection and not a failed pipeline. The workflow continues with a compatible replacement:

DataHub evidence
  → LineageGuard policy: BLOCK
  → LLM: structured migration plan + rationale
  → Zod validation
  → deterministic canonical generation: 8 exact artifacts
  → executable validation: 8 / 8 PASS
  → Draft PR #7 for human review
  → verified DataHub decision memory

DataHub supplies evidence. LineageGuard policy owns ALLOW | REVIEW | BLOCK. The LLM proposes the structured remediation plan and rationale; it does not independently author the eight published files. Application code builds those canonical artifacts deterministically, validators own pass/fail, and a human retains merge authority.

Verified LIVE proof

The accepted result is bound to runtime commit c589f69580c6c56744703ec089554df117e6b0dd and LIVE run run_000000000000019fe1aafd0a. The product video walks through this previously verified LIVE run; recording does not launch new external effects.

Stage	Observed result	Evidence
Source	PR #3 is OPEN at the accepted head	Source PR #3
Baseline	Proposed-change view returns ALLOW / low risk	Risk comparison
DataHub	4 consumer groups, 10 evidence items, 0 synthetic LIVE evidence	Impact context
Policy	ALLOW → BLOCK, rules LG001–LG004	Golden run
Remediation	8 deterministic canonical artifacts	Accepted generation response
Validation	8/8 executable checks pass	Final execution report
Review	PR #7 is OPEN + DRAFT; published bytes match validated bytes	Generated Draft PR #7
Memory	DataHub write-back succeeded and read-back was verified	Run summary

Explore the complete proof:

• accepted Mission Control screenshots
• canonical LIVE run bundle
• accepted generation response
• operation walkthrough
• ready-to-paste Devpost project copy

Quick verification

Prerequisites: Node.js 24 LTS, pnpm 11.20, Python 3.12 with uv, Docker, PostgreSQL, and DataHub OSS. The full LIVE path also requires separately scoped GitHub and DataHub credentials.

pnpm install --frozen-lockfile
pnpm walkthrough:verify

walkthrough:verify runs the environment, format, lint, type, unit, build, and browser gates.

For the external-service-backed canonical walkthrough:

cp .env.example .env
# Configure the least-privilege values documented in docs/demo-walkthrough.md.

pnpm demo:preflight
pnpm demo:bootstrap
pnpm demo:run
pnpm demo:verify -- --runId <run-id>

See Demo Walkthrough for the exact sequence and Troubleshooting for environment-specific recovery.

Architecture

• TypeScript / Node.js 24 LTS: domain contracts, policy, worker orchestration, adapters, persistence, validation control, and Mission Control.
• Next.js + React: the operational evidence workspace.
• DataHub OSS + official MCP Server: read-side organizational context.
• PostgreSQL + dbt Core + Docker: analytical data and isolated executable checks.
• Python 3.12 + uv: narrow DataHub ingestion and metadata-seeding utilities.

Read Architecture, Deterministic Control Plane, and Data Platform Boundary for the detailed design.

Current scope

This release deliberately proves one high-polish vertical slice: commerce.orders.customer_id → buyer_id.

• Analytical warehouse/data-product scope only; operational service schemas and API/event contracts are outside the boundary.
• One canonical rename, not arbitrary schemas, SQL dialects, or a generic migration engine.
• The exact canonical artifacts come from a deterministic generator.
• Validation requires Docker and pinned content-addressed images.
• Mission Control has no multi-tenant authentication.
• DataHub write-back may run automatically after validation; autonomous merge is not enabled, and generated changes remain Draft for human review.
• Verified replay is a clearly labeled contingency for previously accepted LIVE evidence.

LineageGuard is available under the Apache License 2.0.