Skip to content

ADR 015 Paperless ngx Integration Architecture

Jump to bottom
Claude product-architect (Opus 4.6) edited this page Mar 5, 2026 · 2 revisions

ADR-015: Paperless-ngx Integration Architecture

Status

Accepted

Context

Cornerstone needs to integrate with Paperless-ngx for document management. Users want to link documents stored in Paperless-ngx to work items, household items, and vendor invoices. The application must browse, search, and display document metadata and thumbnails without storing documents locally.

Several architectural decisions need to be made:

  1. Communication pattern: Should the browser communicate directly with Paperless-ngx, or should Cornerstone's server proxy the requests?
  2. Document linking schema: Should we use a single polymorphic document_links table with an entity_type discriminator, or separate junction tables per entity type (e.g., work_item_documents, household_item_documents, invoice_documents)?
  3. Caching strategy: Paperless-ngx document metadata and thumbnails are fetched on demand -- should we cache them, and if so, where?
  4. Configuration: How should the Paperless-ngx connection be configured?

Decision

1. Server-Side Proxy Pattern

All Paperless-ngx API requests are proxied through Cornerstone's Fastify server. The browser never communicates directly with Paperless-ngx.

Alternatives considered:

  • Direct client-to-Paperless-ngx: The browser would need the Paperless-ngx API token, exposing it in client-side JavaScript. This is a security risk. It also requires CORS configuration on Paperless-ngx and exposes the Paperless-ngx URL to the browser, which may not be routable from the client network.
  • Server proxy (chosen): The Fastify server holds the Paperless-ngx API token securely. The client only talks to Cornerstone's /api/paperless/* endpoints. This centralizes authentication, allows rate limiting, and simplifies CORS.

Rationale: Security is paramount. The API token grants full access to all documents in Paperless-ngx. Keeping it server-side follows the principle of least privilege. The proxy also allows Cornerstone to transform Paperless-ngx responses into a stable contract that won't break if the upstream API changes.

2. Single Polymorphic Document Links Table

We use a single document_links table with an entity_type discriminator column rather than separate junction tables per entity type.

Alternatives considered:

  • Separate junction tables (work_item_documents, household_item_documents, invoice_documents): Each table has a clear foreign key to its parent entity. This provides strong referential integrity via SQLite foreign keys. However, it requires three nearly identical tables with duplicate DDL, duplicate Drizzle schema definitions, and largely duplicate CRUD routes.
  • Single polymorphic table (chosen): One document_links table with columns entity_type (enum: work_item, household_item, invoice) and entity_id (TEXT). This centralizes all document linking logic, reduces schema duplication, and makes it easy to add new entity types in the future.

Trade-off: The polymorphic approach cannot use SQLite foreign key constraints on entity_id because it references different tables depending on entity_type. Referential integrity for the entity side is enforced at the application layer (check that the referenced entity exists on insert, and cascade-delete links when an entity is deleted). The paperless_document_id column has no foreign key either (it references an external system). This is acceptable because:

  • Document links are lightweight references, not critical business data
  • Orphaned links (pointing to deleted entities) are harmless and can be cleaned up lazily
  • The application layer already validates entity existence on link creation
  • A composite unique constraint on (entity_type, entity_id, paperless_document_id) prevents duplicate links

3. No Server-Side Cache (Phase 1)

For the initial implementation, there is no caching layer. Each proxy request to Paperless-ngx results in a live upstream call.

Alternatives considered:

  • In-memory LRU cache: Cache document metadata and thumbnails in a Node.js Map with TTL. This reduces latency and upstream load.
  • SQLite cache table: Store fetched metadata in a local table with expiry timestamps.
  • No cache (chosen for Phase 1): The simplest approach. With fewer than 5 users, the request volume to Paperless-ngx is minimal. The Paperless-ngx instance is typically on the same network (or even the same machine), so latency is low.

Future extensibility: If performance becomes a concern, an in-memory LRU cache with a configurable TTL (e.g., 5 minutes for metadata, 1 hour for thumbnails) can be added as a Fastify plugin without changing the API contract. The proxy endpoints would check the cache before calling upstream. This is noted but not built.

4. Environment Variable Configuration

Paperless-ngx connection is configured via two environment variables:

Variable Default Description
PAPERLESS_URL (none) Base URL of the Paperless-ngx instance (e.g., http://paperless:8000)
PAPERLESS_API_TOKEN (none) API authentication token for Paperless-ngx

The integration is enabled when both variables are set. If either is missing, all /api/paperless/* endpoints return 503 SERVICE_UNAVAILABLE with error code PAPERLESS_NOT_CONFIGURED.

Rationale: Environment variables are the established configuration pattern in Cornerstone (consistent with OIDC configuration). The API token is sensitive and should never be stored in the database or exposed to the frontend. The paperlessEnabled flag is derived at startup and made available via fastify.config.

5. Paperless-ngx API Version

Cornerstone targets Paperless-ngx API version 5 (supported since Paperless-ngx 2.x). All proxy requests include the Accept: application/json; version=5 header. This pins Cornerstone to a known API shape and prevents breaking changes from upstream version bumps.

6. Proxy Endpoint Design

Proxy endpoints are scoped under /api/paperless/ and map to a curated subset of the Paperless-ngx API:

  • GET /api/paperless/documents -- Search/browse documents
  • GET /api/paperless/documents/:id -- Single document metadata
  • GET /api/paperless/documents/:id/thumb -- Document thumbnail (binary passthrough)
  • GET /api/paperless/documents/:id/preview -- Document preview/PDF (binary passthrough)
  • GET /api/paperless/tags -- List all Paperless-ngx tags

These proxy endpoints transform Paperless-ngx responses into Cornerstone's own response shapes, decoupling the client from the upstream API format. Only the fields needed by the Cornerstone UI are included.

Consequences

Easier

  • Security: The Paperless-ngx token never reaches the browser
  • Consistency: All API calls go through /api/*, maintaining a single origin for the SPA
  • Schema simplicity: One document_links table instead of three junction tables
  • Future entity types: Adding document links to new entities (e.g., subsidy programs, budget sources) requires only a new entity_type enum value and application-layer handlers
  • Configuration: Follows established env var patterns; admins familiar with OIDC config will recognize the pattern

More Difficult

  • No referential integrity on entity_id: Application layer must handle orphan cleanup when entities are deleted
  • Proxy maintenance: If Paperless-ngx changes their API significantly, the proxy layer needs updating (mitigated by version pinning)
  • No offline access: Without caching, Paperless-ngx must be reachable for any document-related feature to work
  • Binary passthrough for thumbnails/previews: The server must buffer and forward binary responses, adding memory pressure for large PDFs (mitigated by the small user count)

Clone this wiki locally