feat(invoices): invoice-budget-line junction table (EPIC-15 Story 15.1)#612
feat(invoices): invoice-budget-line junction table (EPIC-15 Story 15.1)#612steilerDev merged 8 commits intobetafrom
Conversation
… Story 15.1) Replace the 1:1 foreign key model (invoices.work_item_budget_id and invoices.household_item_budget_id) with a many-to-many junction table (invoice_budget_lines) to enable invoices to be split across multiple budget line allocations. ## Changes Database: - Migration 0017: Create invoice_budget_lines junction table with data migration from old 1:1 model. Recreate invoices table without budget FK columns. - Drizzle schema: Add invoiceBudgetLines table, remove FK columns from invoices. Shared Types: - New type file: invoiceBudgetLine.ts with InvoiceBudgetLine, InvoiceBudgetLineSummary - Invoice: Remove workItemBudgetId/householdItemBudgetId FKs and old summary types - Invoice: Add budgetLines array and remainingAmount fields - CreateInvoiceRequest/UpdateInvoiceRequest: Remove budget FK fields - ErrorCode: Add ITEMIZED_SUM_EXCEEDS_INVOICE code Server: - invoiceService: Apply compilation shim (stub budgetLines: [], remainingAmount) Temporary until Story #604 implements invoice budget line CRUD. Note: Test fixtures in existing tests will need updating by QA agent to match new schema in Story #604. Co-Authored-By: Claude backend-developer (Haiku 4.5) <noreply@anthropic.com>
…budget-line junction table Covers: fresh DB migration run, column removal from invoices, table structure and indexes (including partial UNIQUE indexes with WHERE clause verification), data migration for WI and HI budget links, unlinked invoice handling, XOR CHECK constraint enforcement, itemized_amount > 0 CHECK, partial UNIQUE constraint behavior, CASCADE delete from invoice, and idempotency. Also documents a critical defect (Bug #611): SQLite enforces CHECK constraints when ON DELETE SET NULL fires, making the XOR CHECK constraint incompatible with the ON DELETE SET NULL FK actions. The SET NULL tests document the actual behavior and must be updated after Bug #611 is fixed. 52 tests total, all passing. Co-Authored-By: Claude qa-integration-tester (Sonnet 4.6) <noreply@anthropic.com>
…et FKs SQLite enforces CHECK constraints during FK SET NULL actions, making ON DELETE SET NULL incompatible with the XOR CHECK constraint on invoice_budget_lines. Changed to ON DELETE CASCADE for both budget FK columns. Updated ADR-018, Drizzle schema, migration tests, and all affected production code and test fixtures to compile with the new junction table schema. Fixes #611 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Co-Authored-By: Claude backend-developer (Haiku) <noreply@anthropic.com> Co-Authored-By: Claude frontend-developer (Haiku) <noreply@anthropic.com> Co-Authored-By: Claude qa-integration-tester (Haiku) <noreply@anthropic.com>
…ion table Migration 0017 moved budget FK columns from invoices to invoice_budget_lines, but several services still referenced the old columns via raw SQL strings. Updated budgetBreakdownService, budgetOverviewService, budgetServiceFactory, and subsidyPaybackServiceFactory to JOIN through the junction table and use itemized_amount instead of invoices.amount. Fixes #603 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Co-Authored-By: Claude backend-developer (Haiku) <noreply@anthropic.com>
steilerDev
left a comment
There was a problem hiding this comment.
[security-engineer]
Security review of PR #612 — EPIC-15 Story 15.1: Invoice-Budget-Line Junction Schema & Migration.
Summary
No blocking security findings. The migration, schema design, and compilation shims are well-structured from a security standpoint. Two informational observations are noted below.
Checklist
- No SQL/command/XSS injection vectors in new code
- Authentication/authorization enforced on all new endpoints (no new endpoints in this PR)
- No sensitive data (secrets, tokens, PII) exposed in logs, errors, or client responses
- User input validated and sanitized at API boundaries
- New dependencies have no known CVEs (no new dependencies)
- No hardcoded credentials or secrets
- CORS configuration remains restrictive
- Error responses do not leak internal details
Positive Security Observations
Migration SQL is injection-safe. The data migration uses only literal SQL with no string interpolation or user-controlled input. UUID generation uses randomblob() + hex() (SQLite built-ins) — correct pattern, no predictability concern.
XOR CHECK constraint is correct and tested. The CHECK((work_item_budget_id IS NOT NULL AND household_item_budget_id IS NULL) OR ...) pattern correctly enforces mutual exclusivity at the DB layer. The prior Bug #611 fix (SET NULL → CASCADE) is the right resolution — the QA agent's migration test confirms SQLite enforces CHECKs when SET NULL fires, which was the incompatibility.
CASCADE delete semantics are correct. invoice_budget_lines.invoice_id → ON DELETE CASCADE and both FK columns → ON DELETE CASCADE are the right semantics: budget line rows are not meaningful without their invoice, and junction rows should not orphan when a budget line is removed.
Partial UNIQUE indexes are correctly scoped. WHERE work_item_budget_id IS NOT NULL and WHERE household_item_budget_id IS NOT NULL correctly prevent NULL–NULL collisions from triggering the uniqueness constraint, while still enforcing the 1-invoice-per-budget-line invariant for non-null values.
itemized_amount > 0 CHECK is enforced. Prevents zero-amount or negative junction rows at the DB layer. The new error code ITEMIZED_SUM_EXCEEDS_INVOICE is correctly added to the shared ErrorCode union for use by Story #604.
toInvoice() shim is clearly marked temporary. The stub returning budgetLines: [] and remainingAmount: row.amount is an explicit compilation shim with a comment calling out the Story #604 follow-up. This is a correct incremental approach — the API contract is unchanged (fields are present, just empty/stub) and no data is suppressed.
No secrets or PII exposure. The migration does not log or expose any sensitive data. The shim response exposes only amounts already present in the invoice row.
Drizzle schema matches migration DDL. The invoiceBudgetLines table in schema.ts matches the SQL in 0017_invoice_budget_lines.sql: same columns, FKs, and ON DELETE CASCADE semantics. The partial unique index using .where(isNotNull(...)) in Drizzle matches the WHERE ... IS NOT NULL clause in SQL.
Informational Findings
[INFORMATIONAL] toInvoice() shim returns remainingAmount: row.amount regardless of existing junction rows
Description: The compilation shim in invoiceService.ts always returns remainingAmount: row.amount (full invoice amount) even for invoices that have junction rows already (e.g., data migrated from the old 1:1 FK model). For the migration data path, this means the remaining amount shown to users will be incorrect until Story #604 is implemented. This is an accepted functional limitation of the shim, not a security issue, but worth noting so Story #604 picks up the full list of affected queries.
Affected Files:
server/src/services/invoiceService.ts—toInvoice()function,budgetLines: []andremainingAmount: row.amount
Risk if Unaddressed: None security-wise. Functional regression for users viewing invoice detail after migration, until Story #604 completes.
[INFORMATIONAL] PRAGMA foreign_keys = OFF window during migration
Description: Migration 0017 disables foreign keys via PRAGMA foreign_keys = OFF before the table rebuild and re-enables at the end. This is the standard SQLite pattern for ALTER TABLE equivalents and is correct here. However, if the migration is interrupted mid-execution (process crash between PRAGMA OFF and PRAGMA ON), foreign key enforcement could be left disabled for the session. The migration runner should ideally execute the entire migration SQL as a single transaction, which SQLite would roll back atomically on failure.
Affected Files:
server/src/db/migrations/0017_invoice_budget_lines.sql
Remediation: Verify the migration runner (server/src/db/migrate.ts) wraps each migration file execution in a transaction. If it does, the PRAGMA issue is moot — SQLite PRAGMAs inside a transaction scope correctly. This is a pre-existing pattern from earlier migrations (e.g., 0008, 0015) and is tracked as an informational note, not a new regression.
Risk if Unaddressed: No exploit path. In a crash scenario during migration, the SQLite file would likely be in an inconsistent state regardless; the PRAGMA window is the least of the concerns.
Conclusion
PR #612 is approved from a security perspective. The migration SQL is injection-safe, constraint enforcement is correct, the XOR/CASCADE design correctly addresses the Bug #611 incompatibility, and the compilation shim is appropriately bounded to this story. Story #604 should ensure toInvoice() is rewritten to query junction rows and compute remainingAmount correctly.
steilerDev
left a comment
There was a problem hiding this comment.
[product-owner]
Story #603 — AC Review
Acceptance Criteria Assessment
| AC | Description | Verdict |
|---|---|---|
| 1 | Junction table with correct columns | PASS |
| 2 | XOR CHECK constraint | PASS |
| 3 | UNIQUE on work_item_budget_id | PASS |
| 4 | UNIQUE on household_item_budget_id | PASS |
| 5 | Data migration from old FK columns | PASS |
| 6 | Drizzle schema updated | PASS |
| 7 | Indexes created | PASS |
| 8 | Migration idempotency | PASS |
All 8 acceptance criteria are met in the implementation.
Bug #611 Fix
The FK action change from SET NULL to CASCADE is a justified deviation from the original AC #1 wording ("set null on delete"). Bug #611 correctly identified that SQLite enforces CHECK constraints when ON DELETE SET NULL fires, making SET NULL incompatible with the XOR CHECK. CASCADE is the correct fix — when a budget line is deleted, the junction row should be removed entirely rather than left in an invalid state.
Blocking Issue: CI Test Failures
CI shows all 3 test shards failing. This PR cannot be approved while CI is red. Please investigate and fix the test failures, then push a fix.
Non-Blocking Observations
-
Temporary compilation shims: The
toInvoice()function stubsbudgetLines: []andremainingAmount: row.amount. This is acknowledged as temporary (Story #604 will rewrite). Acceptable for a schema-only migration story. -
Frontend shim pattern: InvoiceDetailPage, InvoicesPage, VendorDetailPage, and HouseholdItemDetailPage all have temporary display changes (e.g., "N line(s) linked" instead of full budget line details). These are compilation shims, not final UI. Acceptable.
-
MUTUALLY_EXCLUSIVE_BUDGET_LINKerror code retained: The error code is kept inerrors.tsand a newITEMIZED_SUM_EXCEEDS_INVOICEcode was added. Good forward planning for Story #604. -
itemized_amount > 0CHECK: The migration enforcesitemized_amount > 0(strictly positive). This is stricter than the AC which only specifies REAL NOT NULL. This is a reasonable business constraint — a zero-amount junction row has no meaning.
Verdict
Request changes — all ACs pass but CI must be green before approval. Once test failures are fixed, this PR is ready for approval.
steilerDev
left a comment
There was a problem hiding this comment.
[product-architect] Architecture review for Story #603 (EPIC-15 Story 15.1: Invoice-Budget-Line Junction Schema & Migration)
Summary
The overall architecture is sound. The migration, Drizzle schema, ADR-018, and shared types are well-designed and align with the junction table pattern. However, there are critical issues with test correctness and a wiki documentation deviation that must be fixed before merge.
Findings
CRITICAL: Broken test assertions in invoiceService.household.test.ts
The test file removes the householdItemBudgetId and workItemBudgetId parameters from createInvoice and updateInvoice calls, but retains assertions that expect MutuallyExclusiveBudgetLinkError to be thrown. After the diff, these tests call updateInvoice(db, vendorId, created.id, {}) with an empty object and still expect it to throw -- but the mutual exclusivity validation was removed from invoiceService.ts. These tests will fail (confirmed by CI: Quality Gates and all Test shards are red).
Fix: Either remove the mutual exclusivity tests entirely (the validation is no longer in the service), or update them to test the new junction-table-based constraint (which lives at the DB level via CHECK constraint and will be tested through the junction table CRUD in Story 15.2).
Additionally, ~30+ assertions are commented out with // instead of being removed or replaced with proper junction-table-aware assertions. While some of these are legitimately deferred to Story 15.2 (when the CRUD API is implemented), the pattern of leaving large blocks of commented-out test code is fragile -- it's easy to forget to un-comment them. Recommend adding // TODO(EPIC-15 Story 15.2): Re-enable with junction table assertions to each block so they can be tracked.
HIGH: Wiki Schema.md documents ON DELETE SET NULL for budget FKs, but implementation uses ON DELETE CASCADE
The wiki Schema.md (line 2036-2037, 2065, 2134, 2154-2155) documents ON DELETE SET NULL for work_item_budget_id and household_item_budget_id on invoice_budget_lines. The actual migration SQL (line 433-434 of 0017_invoice_budget_lines.sql), Drizzle schema, and ADR-018 all correctly use ON DELETE CASCADE.
This was the Bug #611 fix -- SQLite enforces CHECK constraints during SET NULL, so CASCADE is required. The ADR-018 wiki page was updated to CASCADE, but the Schema.md page was not.
Fix: Update wiki/Schema.md to change all references from ON DELETE SET NULL to ON DELETE CASCADE for invoice_budget_lines.work_item_budget_id and invoice_budget_lines.household_item_budget_id. Also update the design rationale text (line 2065) and the DDL example (lines 2154-2155). Add a Deviation Log entry.
MEDIUM: InvoiceBudgetLineSummary type divergence between shared types and wiki API Contract
The shared type InvoiceBudgetLineSummary (in shared/src/types/invoiceBudgetLine.ts) has fields budgetLineId, budgetLineType, itemName, confidence that the wiki API Contract's InvoiceBudgetLineSummary (embedded in InvoiceResponse) does not include. Conversely, the wiki's GET endpoint response shape includes categoryId, parentItemId, parentItemTitle, parentItemType fields that the shared type does not have.
This is acceptable as a Story 15.1 scope item (the CRUD endpoint is Story 15.2), but the shapes need to be reconciled before Story 15.2 is implemented. The wiki API Contract should be the source of truth.
Action: No fix required now, but flag this for Story 15.2 implementation. The shared type should be updated to match the wiki API Contract when the CRUD endpoint is built.
Architecture Compliance
- Migration SQL: Correctly implements the junction table per ADR-018. PRAGMA foreign_keys ON/OFF pattern is correct for table rebuild. UUID generation via randomblob is appropriate for SQLite.
- Drizzle schema: Matches the migration SQL. Partial unique indexes using
.where(isNotNull(...))are correct. - Data migration: Correctly migrates existing 1:1 FK data to junction rows, using invoice amount as itemized_amount.
- Service changes:
budgetSourceService.tsandhouseholdItemService.tscorrectly updated to join throughinvoiceBudgetLinesand SUMitemizedAmountinstead ofinvoices.amount. - Type changes: Invoice type correctly replaces
workItemBudgetId/householdItemBudgetIdwithbudgetLines[]andremainingAmount. New error codeITEMIZED_SUM_EXCEEDS_INVOICEadded. - Compilation shim: The
toInvoice()function returningbudgetLines: []andremainingAmount: row.amountis a valid temporary shim for Story 15.1.
Verdict
Request changes due to CI-blocking test failures. The wiki Schema.md deviation should also be fixed in this PR since the schema documentation is part of Story 15.1's deliverables.
…nstraints Tests created multiple invoices per budget line, violating the new partial UNIQUE index. Reworked test data to use distinct budget lines for each invoice. Removed obsolete invoice-budget FK validation tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Co-Authored-By: Claude qa-integration-tester (Haiku) <noreply@anthropic.com>
ESM mode does not provide __dirname. Use fileURLToPath(import.meta.url) to resolve the migrations directory path. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…rceService tests Two tests attached both a paid and claimed invoice to the same budget line, violating the partial UNIQUE index. Use separate budget lines instead. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
🎉 This PR is included in version 1.14.0-beta.1 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
|
🎉 This PR is included in version 1.14.0 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
Summary
invoice_budget_linesjunction table (migration 0017) replacing the 1:1 FK model on invoicesFixes #603
Fixes #611
Test plan
Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
Co-Authored-By: Claude backend-developer (Haiku) noreply@anthropic.com
Co-Authored-By: Claude frontend-developer (Haiku) noreply@anthropic.com
Co-Authored-By: Claude qa-integration-tester (Haiku) noreply@anthropic.com