Documents and Storage

Suite test cases can attach documents without storing the file bytes in PostgreSQL. Aludel validates uploads, records metadata and a storage reference, then loads content only for execution.

Supported Files

• PDF
• PNG
• JPEG
• JSON
• CSV
• plain text

Validation checks the claimed content type against signatures or valid text/JSON content before persistence.

Storage Flow

flowchart LR
    U[Upload] --> V[Validate]
    V --> S[Storage adapter]
    S --> M[(Document metadata in PostgreSQL)]
    S --> B[(Local, S3, or GCS bytes)]
    M --> E[Suite execution]
    B --> E

Loading

Document rows store filename, content type, size, storage key, and backend. Deletes remove both the database row and the external object.

Local Development

config :aludel, Aludel.Storage,
  adapter: Aludel.Interfaces.Storage.Adapters.Local,
  backends: [{Aludel.Interfaces.Storage.Adapters.Local, root: "tmp/aludel_uploads"}]

The default local root is an operating-system temporary directory.

For a standalone production release, choose a persistent mounted path explicitly:

export ALUDEL_STORAGE_BACKEND=local
export ALUDEL_STORAGE_PATH=/data/aludel_uploads

AWS S3

export ALUDEL_STORAGE_BACKEND=aws
export AWS_S3_BUCKET=aludel-uploads
export AWS_REGION=us-east-1

The standalone release uses the AWS runtime identity provider by default. Set explicit credentials only when the deployment does not supply a runtime identity:

export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
export AWS_SESSION_TOKEN=...

The access-key pair must be set together; the session token is optional. The AWS adapter uses ExAws S3. Use an identity limited to the configured bucket and required object actions.

Google Cloud Storage

export ALUDEL_STORAGE_BACKEND=gcs
export GCS_BUCKET=aludel-uploads
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json
export GCS_USER_PROJECT=optional-requester-pays-project

GOOGLE_APPLICATION_CREDENTIALS_JSON is also supported. The adapter uses Goth and Google application credentials. Set GCS_USER_PROJECT only for requester-pays buckets.

Standalone startup validates storage before the endpoint starts. Missing backend-specific values, unsupported backends, relative local paths, and partial explicit AWS credentials fail with a configuration error.

Changing ALUDEL_STORAGE_BACKEND does not move existing objects. Keep the former backend variables configured until its documents have been removed or migrated. The standalone release retains every completely configured backend so existing document rows can still be read and cleaned up.

Library API

Embedded applications can use the storage facade directly after configuring an adapter:

document_id = Ecto.UUID.generate()
key = Aludel.Storage.storage_key(document_id, "customer brief.pdf")

{:ok, ^key} =
  Aludel.Storage.put(key, pdf_bytes, "application/pdf")

{:ok, ^pdf_bytes} = Aludel.Storage.get(key)
:ok = Aludel.Storage.delete(key)

Use a document row when Aludel should select the backend that originally owned the object:

{:ok, bytes} = Aludel.Storage.read(test_case_document)

The persisted backend remains authoritative after an active-backend switch. Direct reads and deletes can make the same selection with storage_backend: test_case_document.storage_backend.

PDF Conversion

Anthropic supports native PDF input. Provider paths that require images use the document converter:

config :aludel, :document_converter,
  adapter: Aludel.Interfaces.DocumentConverter.Adapters.Imagemagick,
  density: 150,
  timeout_ms: 30_000,
  max_input_bytes: 10_485_760,
  max_output_bytes: 20_971_520,
  max_diagnostic_bytes: 16_384

The included adapter renders only the first page, preferring ImageMagick's magick executable and falling back to convert on ImageMagick 6 installations. Every conversion receives a cryptographically random private workspace and a dedicated OS process group. Timeouts terminate the converter and its delegates before Aludel removes the workspace.

The defaults also bound source and result bytes, retained diagnostics, dimensions, memory, mapped memory, temporary disk, open files, worker threads, and ImageMagick execution time. Applications can lower the byte limits, choose a density from 72 through 300 DPI, or set a timeout from 100 through 60,000 milliseconds. Trusted deployments can set explicit :executable and existing :temporary_directory paths.

The dependency that manages converter process groups needs a C++17 toolchain at compilation time.

Custom Backends

Implement Aludel.Interfaces.Storage.Behaviour for another object store. The adapter receives a storage key, bytes, content type, and backend configuration for writes, and exposes get/delete operations for later execution and cleanup.

Implement Aludel.Interfaces.DocumentConverter.Behaviour when document transformation needs another tool or supports additional formats.

Related Pages

• Evaluation Suites
• Installation
• Architecture