Skip to content

feat(invoices): invoice-budget-line junction table (EPIC-15 Story 15.1)#612

Merged
steilerDev merged 8 commits intobetafrom
feat/603-invoice-budget-junction
Mar 8, 2026
Merged

feat(invoices): invoice-budget-line junction table (EPIC-15 Story 15.1)#612
steilerDev merged 8 commits intobetafrom
feat/603-invoice-budget-junction

Conversation

@steilerDev
Copy link
Copy Markdown
Owner

Summary

  • Creates invoice_budget_lines junction table (migration 0017) replacing the 1:1 FK model on invoices
  • Migrates existing invoice-budget links to junction rows with itemized amounts
  • Updates Drizzle schema, shared types, and all affected services/pages with temporary compilation shims
  • Fixes Bug BUG: Migration 0017 ON DELETE SET NULL incompatible with XOR CHECK constraint in SQLite #611: ON DELETE SET NULL → CASCADE for budget line FKs (SQLite CHECK constraint incompatibility)

Fixes #603
Fixes #611

Test plan

  • Migration tests verify table creation, data migration, constraints, and indexes
  • Full typecheck passes (0 errors across shared, server, client)
  • All test fixtures updated to use junction table pattern
  • CI Quality Gates pass

Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
Co-Authored-By: Claude backend-developer (Haiku) noreply@anthropic.com
Co-Authored-By: Claude frontend-developer (Haiku) noreply@anthropic.com
Co-Authored-By: Claude qa-integration-tester (Haiku) noreply@anthropic.com

claude added 4 commits March 8, 2026 19:33
… Story 15.1)

Replace the 1:1 foreign key model (invoices.work_item_budget_id and
invoices.household_item_budget_id) with a many-to-many junction table
(invoice_budget_lines) to enable invoices to be split across multiple budget
line allocations.

## Changes

Database:
- Migration 0017: Create invoice_budget_lines junction table with data migration
  from old 1:1 model. Recreate invoices table without budget FK columns.
- Drizzle schema: Add invoiceBudgetLines table, remove FK columns from invoices.

Shared Types:
- New type file: invoiceBudgetLine.ts with InvoiceBudgetLine, InvoiceBudgetLineSummary
- Invoice: Remove workItemBudgetId/householdItemBudgetId FKs and old summary types
- Invoice: Add budgetLines array and remainingAmount fields
- CreateInvoiceRequest/UpdateInvoiceRequest: Remove budget FK fields
- ErrorCode: Add ITEMIZED_SUM_EXCEEDS_INVOICE code

Server:
- invoiceService: Apply compilation shim (stub budgetLines: [], remainingAmount)
  Temporary until Story #604 implements invoice budget line CRUD.

Note: Test fixtures in existing tests will need updating by QA agent to match
new schema in Story #604.

Co-Authored-By: Claude backend-developer (Haiku 4.5) <noreply@anthropic.com>
…budget-line junction table

Covers: fresh DB migration run, column removal from invoices, table structure
and indexes (including partial UNIQUE indexes with WHERE clause verification),
data migration for WI and HI budget links, unlinked invoice handling, XOR
CHECK constraint enforcement, itemized_amount > 0 CHECK, partial UNIQUE
constraint behavior, CASCADE delete from invoice, and idempotency.

Also documents a critical defect (Bug #611): SQLite enforces CHECK constraints
when ON DELETE SET NULL fires, making the XOR CHECK constraint incompatible
with the ON DELETE SET NULL FK actions. The SET NULL tests document the actual
behavior and must be updated after Bug #611 is fixed.

52 tests total, all passing.

Co-Authored-By: Claude qa-integration-tester (Sonnet 4.6) <noreply@anthropic.com>
…et FKs

SQLite enforces CHECK constraints during FK SET NULL actions, making
ON DELETE SET NULL incompatible with the XOR CHECK constraint on
invoice_budget_lines. Changed to ON DELETE CASCADE for both budget
FK columns. Updated ADR-018, Drizzle schema, migration tests, and
all affected production code and test fixtures to compile with the
new junction table schema.

Fixes #611

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude backend-developer (Haiku) <noreply@anthropic.com>
Co-Authored-By: Claude frontend-developer (Haiku) <noreply@anthropic.com>
Co-Authored-By: Claude qa-integration-tester (Haiku) <noreply@anthropic.com>
…ion table

Migration 0017 moved budget FK columns from invoices to invoice_budget_lines,
but several services still referenced the old columns via raw SQL strings.
Updated budgetBreakdownService, budgetOverviewService, budgetServiceFactory,
and subsidyPaybackServiceFactory to JOIN through the junction table and use
itemized_amount instead of invoices.amount.

Fixes #603

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude backend-developer (Haiku) <noreply@anthropic.com>
Copy link
Copy Markdown
Owner Author

@steilerDev steilerDev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[security-engineer]

Security review of PR #612 — EPIC-15 Story 15.1: Invoice-Budget-Line Junction Schema & Migration.

Summary

No blocking security findings. The migration, schema design, and compilation shims are well-structured from a security standpoint. Two informational observations are noted below.

Checklist

  • No SQL/command/XSS injection vectors in new code
  • Authentication/authorization enforced on all new endpoints (no new endpoints in this PR)
  • No sensitive data (secrets, tokens, PII) exposed in logs, errors, or client responses
  • User input validated and sanitized at API boundaries
  • New dependencies have no known CVEs (no new dependencies)
  • No hardcoded credentials or secrets
  • CORS configuration remains restrictive
  • Error responses do not leak internal details

Positive Security Observations

Migration SQL is injection-safe. The data migration uses only literal SQL with no string interpolation or user-controlled input. UUID generation uses randomblob() + hex() (SQLite built-ins) — correct pattern, no predictability concern.

XOR CHECK constraint is correct and tested. The CHECK((work_item_budget_id IS NOT NULL AND household_item_budget_id IS NULL) OR ...) pattern correctly enforces mutual exclusivity at the DB layer. The prior Bug #611 fix (SET NULL → CASCADE) is the right resolution — the QA agent's migration test confirms SQLite enforces CHECKs when SET NULL fires, which was the incompatibility.

CASCADE delete semantics are correct. invoice_budget_lines.invoice_id → ON DELETE CASCADE and both FK columns → ON DELETE CASCADE are the right semantics: budget line rows are not meaningful without their invoice, and junction rows should not orphan when a budget line is removed.

Partial UNIQUE indexes are correctly scoped. WHERE work_item_budget_id IS NOT NULL and WHERE household_item_budget_id IS NOT NULL correctly prevent NULL–NULL collisions from triggering the uniqueness constraint, while still enforcing the 1-invoice-per-budget-line invariant for non-null values.

itemized_amount > 0 CHECK is enforced. Prevents zero-amount or negative junction rows at the DB layer. The new error code ITEMIZED_SUM_EXCEEDS_INVOICE is correctly added to the shared ErrorCode union for use by Story #604.

toInvoice() shim is clearly marked temporary. The stub returning budgetLines: [] and remainingAmount: row.amount is an explicit compilation shim with a comment calling out the Story #604 follow-up. This is a correct incremental approach — the API contract is unchanged (fields are present, just empty/stub) and no data is suppressed.

No secrets or PII exposure. The migration does not log or expose any sensitive data. The shim response exposes only amounts already present in the invoice row.

Drizzle schema matches migration DDL. The invoiceBudgetLines table in schema.ts matches the SQL in 0017_invoice_budget_lines.sql: same columns, FKs, and ON DELETE CASCADE semantics. The partial unique index using .where(isNotNull(...)) in Drizzle matches the WHERE ... IS NOT NULL clause in SQL.

Informational Findings

[INFORMATIONAL] toInvoice() shim returns remainingAmount: row.amount regardless of existing junction rows

Description: The compilation shim in invoiceService.ts always returns remainingAmount: row.amount (full invoice amount) even for invoices that have junction rows already (e.g., data migrated from the old 1:1 FK model). For the migration data path, this means the remaining amount shown to users will be incorrect until Story #604 is implemented. This is an accepted functional limitation of the shim, not a security issue, but worth noting so Story #604 picks up the full list of affected queries.

Affected Files:

  • server/src/services/invoiceService.tstoInvoice() function, budgetLines: [] and remainingAmount: row.amount

Risk if Unaddressed: None security-wise. Functional regression for users viewing invoice detail after migration, until Story #604 completes.


[INFORMATIONAL] PRAGMA foreign_keys = OFF window during migration

Description: Migration 0017 disables foreign keys via PRAGMA foreign_keys = OFF before the table rebuild and re-enables at the end. This is the standard SQLite pattern for ALTER TABLE equivalents and is correct here. However, if the migration is interrupted mid-execution (process crash between PRAGMA OFF and PRAGMA ON), foreign key enforcement could be left disabled for the session. The migration runner should ideally execute the entire migration SQL as a single transaction, which SQLite would roll back atomically on failure.

Affected Files:

  • server/src/db/migrations/0017_invoice_budget_lines.sql

Remediation: Verify the migration runner (server/src/db/migrate.ts) wraps each migration file execution in a transaction. If it does, the PRAGMA issue is moot — SQLite PRAGMAs inside a transaction scope correctly. This is a pre-existing pattern from earlier migrations (e.g., 0008, 0015) and is tracked as an informational note, not a new regression.

Risk if Unaddressed: No exploit path. In a crash scenario during migration, the SQLite file would likely be in an inconsistent state regardless; the PRAGMA window is the least of the concerns.


Conclusion

PR #612 is approved from a security perspective. The migration SQL is injection-safe, constraint enforcement is correct, the XOR/CASCADE design correctly addresses the Bug #611 incompatibility, and the compilation shim is appropriately bounded to this story. Story #604 should ensure toInvoice() is rewritten to query junction rows and compute remainingAmount correctly.

Copy link
Copy Markdown
Owner Author

@steilerDev steilerDev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[product-owner]

Story #603 — AC Review

Acceptance Criteria Assessment

AC Description Verdict
1 Junction table with correct columns PASS
2 XOR CHECK constraint PASS
3 UNIQUE on work_item_budget_id PASS
4 UNIQUE on household_item_budget_id PASS
5 Data migration from old FK columns PASS
6 Drizzle schema updated PASS
7 Indexes created PASS
8 Migration idempotency PASS

All 8 acceptance criteria are met in the implementation.

Bug #611 Fix

The FK action change from SET NULL to CASCADE is a justified deviation from the original AC #1 wording ("set null on delete"). Bug #611 correctly identified that SQLite enforces CHECK constraints when ON DELETE SET NULL fires, making SET NULL incompatible with the XOR CHECK. CASCADE is the correct fix — when a budget line is deleted, the junction row should be removed entirely rather than left in an invalid state.

Blocking Issue: CI Test Failures

CI shows all 3 test shards failing. This PR cannot be approved while CI is red. Please investigate and fix the test failures, then push a fix.

Non-Blocking Observations

  1. Temporary compilation shims: The toInvoice() function stubs budgetLines: [] and remainingAmount: row.amount. This is acknowledged as temporary (Story #604 will rewrite). Acceptable for a schema-only migration story.

  2. Frontend shim pattern: InvoiceDetailPage, InvoicesPage, VendorDetailPage, and HouseholdItemDetailPage all have temporary display changes (e.g., "N line(s) linked" instead of full budget line details). These are compilation shims, not final UI. Acceptable.

  3. MUTUALLY_EXCLUSIVE_BUDGET_LINK error code retained: The error code is kept in errors.ts and a new ITEMIZED_SUM_EXCEEDS_INVOICE code was added. Good forward planning for Story #604.

  4. itemized_amount > 0 CHECK: The migration enforces itemized_amount > 0 (strictly positive). This is stricter than the AC which only specifies REAL NOT NULL. This is a reasonable business constraint — a zero-amount junction row has no meaning.

Verdict

Request changes — all ACs pass but CI must be green before approval. Once test failures are fixed, this PR is ready for approval.

Copy link
Copy Markdown
Owner Author

@steilerDev steilerDev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[product-architect] Architecture review for Story #603 (EPIC-15 Story 15.1: Invoice-Budget-Line Junction Schema & Migration)

Summary

The overall architecture is sound. The migration, Drizzle schema, ADR-018, and shared types are well-designed and align with the junction table pattern. However, there are critical issues with test correctness and a wiki documentation deviation that must be fixed before merge.

Findings

CRITICAL: Broken test assertions in invoiceService.household.test.ts

The test file removes the householdItemBudgetId and workItemBudgetId parameters from createInvoice and updateInvoice calls, but retains assertions that expect MutuallyExclusiveBudgetLinkError to be thrown. After the diff, these tests call updateInvoice(db, vendorId, created.id, {}) with an empty object and still expect it to throw -- but the mutual exclusivity validation was removed from invoiceService.ts. These tests will fail (confirmed by CI: Quality Gates and all Test shards are red).

Fix: Either remove the mutual exclusivity tests entirely (the validation is no longer in the service), or update them to test the new junction-table-based constraint (which lives at the DB level via CHECK constraint and will be tested through the junction table CRUD in Story 15.2).

Additionally, ~30+ assertions are commented out with // instead of being removed or replaced with proper junction-table-aware assertions. While some of these are legitimately deferred to Story 15.2 (when the CRUD API is implemented), the pattern of leaving large blocks of commented-out test code is fragile -- it's easy to forget to un-comment them. Recommend adding // TODO(EPIC-15 Story 15.2): Re-enable with junction table assertions to each block so they can be tracked.

HIGH: Wiki Schema.md documents ON DELETE SET NULL for budget FKs, but implementation uses ON DELETE CASCADE

The wiki Schema.md (line 2036-2037, 2065, 2134, 2154-2155) documents ON DELETE SET NULL for work_item_budget_id and household_item_budget_id on invoice_budget_lines. The actual migration SQL (line 433-434 of 0017_invoice_budget_lines.sql), Drizzle schema, and ADR-018 all correctly use ON DELETE CASCADE.

This was the Bug #611 fix -- SQLite enforces CHECK constraints during SET NULL, so CASCADE is required. The ADR-018 wiki page was updated to CASCADE, but the Schema.md page was not.

Fix: Update wiki/Schema.md to change all references from ON DELETE SET NULL to ON DELETE CASCADE for invoice_budget_lines.work_item_budget_id and invoice_budget_lines.household_item_budget_id. Also update the design rationale text (line 2065) and the DDL example (lines 2154-2155). Add a Deviation Log entry.

MEDIUM: InvoiceBudgetLineSummary type divergence between shared types and wiki API Contract

The shared type InvoiceBudgetLineSummary (in shared/src/types/invoiceBudgetLine.ts) has fields budgetLineId, budgetLineType, itemName, confidence that the wiki API Contract's InvoiceBudgetLineSummary (embedded in InvoiceResponse) does not include. Conversely, the wiki's GET endpoint response shape includes categoryId, parentItemId, parentItemTitle, parentItemType fields that the shared type does not have.

This is acceptable as a Story 15.1 scope item (the CRUD endpoint is Story 15.2), but the shapes need to be reconciled before Story 15.2 is implemented. The wiki API Contract should be the source of truth.

Action: No fix required now, but flag this for Story 15.2 implementation. The shared type should be updated to match the wiki API Contract when the CRUD endpoint is built.

Architecture Compliance

  • Migration SQL: Correctly implements the junction table per ADR-018. PRAGMA foreign_keys ON/OFF pattern is correct for table rebuild. UUID generation via randomblob is appropriate for SQLite.
  • Drizzle schema: Matches the migration SQL. Partial unique indexes using .where(isNotNull(...)) are correct.
  • Data migration: Correctly migrates existing 1:1 FK data to junction rows, using invoice amount as itemized_amount.
  • Service changes: budgetSourceService.ts and householdItemService.ts correctly updated to join through invoiceBudgetLines and SUM itemizedAmount instead of invoices.amount.
  • Type changes: Invoice type correctly replaces workItemBudgetId/householdItemBudgetId with budgetLines[] and remainingAmount. New error code ITEMIZED_SUM_EXCEEDS_INVOICE added.
  • Compilation shim: The toInvoice() function returning budgetLines: [] and remainingAmount: row.amount is a valid temporary shim for Story 15.1.

Verdict

Request changes due to CI-blocking test failures. The wiki Schema.md deviation should also be fixed in this PR since the schema documentation is part of Story 15.1's deliverables.

claude added 4 commits March 8, 2026 20:44
…nstraints

Tests created multiple invoices per budget line, violating the new partial
UNIQUE index. Reworked test data to use distinct budget lines for each
invoice. Removed obsolete invoice-budget FK validation tests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude qa-integration-tester (Haiku) <noreply@anthropic.com>
ESM mode does not provide __dirname. Use fileURLToPath(import.meta.url)
to resolve the migrations directory path.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…rceService tests

Two tests attached both a paid and claimed invoice to the same budget line,
violating the partial UNIQUE index. Use separate budget lines instead.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@steilerDev steilerDev merged commit dcf94b0 into beta Mar 8, 2026
13 checks passed
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 8, 2026

🎉 This PR is included in version 1.14.0-beta.1 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 9, 2026

🎉 This PR is included in version 1.14.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

@steilerDev steilerDev deleted the feat/603-invoice-budget-junction branch March 22, 2026 12:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants