Skip to content

ADR 032 Paperless First Invoice Creation

Jump to bottom
Frank Steiler edited this page Jun 15, 2026 · 1 revision

ADR-032: Paperless-First Invoice Creation via Stateless Extraction and Create-on-Confirm Commit

Status

Accepted

Context

Story #1679 ("Paperless-first invoice creation with LLM auto-itemize") introduces a new invoice-creation entry point: the user picks a Paperless-ngx document first, the app's LLM auto-extracts the line items, document metadata, and best-matching vendor, and the user reviews everything before the invoice is created.

The existing auto-itemize feature (EPIC-16, ADR-031, Story #1547) is keyed on an existing invoice:

  • POST /api/invoices/:invoiceId/auto-itemize loads the invoice up front (invoiceAutoItemizeService.ts ~line 110), seeds extraction hints from invoice.vendorId and invoice.amount, and verifies the paperlessDocumentId is already linked to that invoice via document_links.
  • On commit (dryRun: false), it creates budget lines on that invoice and optionally patches it.

The new flow is extract-first, create-on-confirm: there is no invoice (and no document link) when the user picks a document. Per the product-owner's acceptance criteria (AC23–AC26), the invoice must be created only after the human confirms, and abandoning the review must leave no orphan invoice, document link, or budget line.

Three sub-problems must be solved without breaking the existing invoice-keyed flow:

  1. Stateless extraction — extract from a document with no invoice context, and let the LLM choose the best-matching app vendor from a server-provided list.
  2. Atomic create-on-confirm — create the invoice, link the document, and itemize the reviewed lines in a single transaction with all-or-nothing semantics.
  3. Correspondent listing — a new Paperless proxy endpoint to populate the picker's vendor-filter dropdown.

Alternatives considered

  • (A) Extend the existing /:invoiceId/auto-itemize route to accept a null invoiceId. Rejected: the route's params schema requires invoiceId, the service body assumes a loaded invoice for hints and the document-link check, and overloading dry-run vs. commit semantics for two fundamentally different lifecycles (patch-existing vs. create-new) would make the service a tangle of branches. Backward-compatibility risk is high.
  • (B) Persist a draft invoice immediately, then finalize on confirm. Rejected by the product-owner (AC23/AC25): a draft would be an orphan if the user abandons review, and the draft would pollute the invoices list and budget aggregations.
  • (C) Two dedicated stateless endpoints — preview (extraction) and commit (atomic create) — that share extraction internals with the existing flow. Chosen. Clean separation of lifecycles, no change to the existing endpoint's contract, and maximal reuse of the LLM extraction + line-persistence internals.

Decision

Add two new stateless endpoints plus one Paperless proxy endpoint, and refactor invoiceAutoItemizeService.ts to extract its reusable internals into shared helpers.

1. Stateless extraction — POST /api/invoices/auto-itemize/preview

  • No invoiceId. Request: { paperlessDocumentId, locale? }.
  • Server fetches the document OCR text (existing paperlessService.getDocument), runs the LLM provider (existing getProvider), maps category names to IDs (existing mapCategoryNameToId), and computes the due-date fallback (existing computeDueDateFallback) — identical to the existing dry-run path.
  • Vendor matching: the server loads all app vendors as { id, name }, injects the list into the LLM prompt, and instructs the LLM to return chosenVendorName (a verbatim copy of one provided name, or null). The server resolves chosenVendorName back to a vendor id by exact case-insensitive name match against the provided list. If the LLM returns null or a name that does not exactly match a provided vendor, suggestedVendorId is null (AC22). Matching is performed server-side against the canonical list — the LLM never invents an id.
  • No warnings array (there is no invoice total to compare against yet). The review UI computes the extracted-line total locally.
  • Response: { lines, suggestedVendorId, extractedInvoiceNumber?, extractedInvoiceDate?, extractedDueDate?, extractedNotes? }.

2. Atomic create-on-confirm — POST /api/invoices/auto-itemize/commit

A dedicated endpoint (not an extension of createInvoice, which is vendor-scoped and knows nothing about document links or itemization). In a single SQLite transaction:

  1. Validate vendorId exists (vendor is required — AC21).
  2. Create the invoice (createInvoice internals: vendor, amount, date, dueDate, invoiceNumber, notes, status).
  3. Create the document_links row linking the selected Paperless document to the new invoice (entity_type='invoice', entity_id=<new invoice id>).
  4. Itemize the reviewed lines into invoice_budget_lines / work_item_budgets using the same line-persistence helper extracted from the existing commit path (mode: 'append' semantics — there are never pre-existing lines on a brand-new invoice).
  5. Validate Σ itemized ≤ invoice amount (existing ItemizedSumExceedsInvoiceError).

If any step throws, db.transaction() rolls back the entire unit: no invoice, no document link, no budget lines persist (AC24/AC26). The endpoint is not idempotent — each successful call creates a new invoice; the client must guard against double-submit (the review screen disables confirm while the request is in flight). Because there is no invoice before confirm, abandoning the review is a pure client-side discard — nothing to clean up (AC25).

Response (201 Created): { invoice, budgetLines, remainingAmount } — the created invoice plus the itemization result, so the client can navigate straight to the invoice without a follow-up fetch.

3. Correspondents — GET /api/paperless/correspondents

Mirrors GET /api/paperless/tags exactly: returns { correspondents: { id, name }[] }, requires auth, returns 503 PAPERLESS_NOT_CONFIGURED when Paperless is not configured, 502 on upstream failure. Fetched on demand (paginated page_size=1000 like tags). Not cached initially — correspondent lists are small (<~100) and the picker fetches once on open; if profiling later shows pressure, a short-TTL cache mirroring the filter-tag cache can be added (deferred, noted for future review).

4. Reuse strategy (invoiceAutoItemizeService.ts)

The existing invoice-keyed flow stays unchanged in behavior. The following internals are extracted into reusable, non-invoice-coupled helpers (same file or a sibling module) and shared by both flows:

  • Line persistence — the per-line loop that resolves assignment mode, creates work_item_budgets (origin 'auto') or assigns existing budget lines, inserts invoice_budget_lines junction rows, and accumulates totalItemized. Refactored to take (db, invoiceId, vendorId, userId, lines) instead of reading them off a loaded invoice object. The existing commit path calls it with the loaded invoice's id/vendorId; the new commit path calls it with the freshly-created invoice's id/vendorId.
  • Extraction core — fetch document, truncate OCR to MAX_OCR_CHARS, run provider, map categories, compute due-date fallback. Shared by both dry-run paths. The only difference is hint seeding (existing flow seeds from the loaded invoice; the new flow seeds from the document + provided locale and additionally injects the vendor list).
  • Vendor-list prompt augmentation — new code: gather { id, name }[], render into the user prompt, parse chosenVendorName, resolve to id. Used only by the new preview endpoint.

Genuinely new code: the two route handlers, the two thin service entry points (previewAutoItemize, commitAutoItemizeCreate), the vendor-list prompt augmentation + parsing, and the correspondents proxy. The bulk of persistence and extraction logic is shared.

Consequences

Easier:

  • The existing /:invoiceId/auto-itemize contract is untouched — zero regression risk to the EPIC-16 flow and its tests.
  • Atomicity is trivially correct: a single db.transaction() wraps create + link + itemize, so partial failure cannot leave orphans (AC26), and there is no draft-cleanup machinery to maintain (AC25).
  • Vendor resolution is deterministic and safe — the LLM can only echo a name from the server-provided list, and the id is resolved server-side, so the LLM cannot fabricate a vendor id.
  • The correspondents endpoint follows the established /tags pattern, so it is consistent and quick to implement/test.

More difficult / trade-offs:

  • Two extraction entry points now exist (/:invoiceId/auto-itemize dry-run and /auto-itemize/preview). They share an extraction core but diverge in hint seeding and the vendor-list step; this divergence must be kept in sync if the prompt evolves. Mitigated by extracting the shared core into one helper.
  • The commit endpoint duplicates the "create invoice" surface that createInvoice already owns. We deliberately call createInvoice's internals rather than re-implementing validation, but the transaction boundary lives in the new endpoint. If invoice-creation rules change, both call sites must be considered (they share the underlying helper, so the risk is contained).
  • Exact-match vendor resolution may under-match (e.g., "ACME GmbH" vs. "ACME"); the user override (AC21) is the safety net. Fuzzy matching is intentionally not done server-side to keep resolution predictable; the LLM is the fuzzy layer and is constrained to echo a provided name.
  • Correspondents are not cached; a picker open triggers one upstream call. Acceptable at <5 users; revisit if it becomes a hot path.

Related

  • ADR-015 (Paperless-ngx Integration) — proxy pattern, polymorphic document_links.
  • ADR-018 (Invoice-Budget-Line Junction) — invoice_budget_lines model, ON DELETE CASCADE.
  • ADR-031 (Outbound LLM Provider Integration) — OpenAI-compatible gateway, autoItemizeEnabled.

Clone this wiki locally