[Feat]: chat attachments backed by Cloudflare R2

[FireTable]

Feature: chat attachments backed by Cloudflare R2

Let users drag-drop / paste a file into the composer and have it round-trip through the chat:

1. File uploads to a Cloudflare R2 bucket.
2. The assistant-ui runtime receives a CompleteAttachment whose content is the message parts the model will see (image URL / file URL / extracted text, depending on type).
3. We persist an attachments row keyed to (user_id, thread_id, message_id) so we can render the attachment in the thread later, dedupe per-thread, and (later) feed it into the knowledge base.

This issue is just upload + storage + per-message reference. Knowledge-base ingestion is a separate follow-up — see Out of scope.

Goals

• Wire assistant-ui's AttachmentAdapter (no custom file-picker UI; the <Composer /> already supports attachments when an adapter is provided).
• Store files in Cloudflare R2, not on the Next.js server (zero disk, zero proxying of file bytes through us).
• Persist the public/signed URL + metadata in Postgres so the file is recoverable across reloads, deletable, and consumable by future tools (KB ingestion, search, etc.).
• Per-user isolation: file lookups MUST scope by user_id; cross-user access returns 404 (matches the existing data-isolation pattern — see docs/AUTH.md).
• Self-host-friendly: env-driven config, no Cloudflare-specific business logic outside the S3 client wrapper. A future drop-in to Backblaze B2 / MinIO / AWS S3 should be a one-line endpoint change.

Frontend wiring — assistant-ui AttachmentAdapter

The adapter contract (verified against @assistant-ui/core@0.2.20/dist/adapters/attachment.d.ts installed in this repo):

type AttachmentAdapter = {
  accept: string;                                          // MIME pattern
  add(state: { file: File }): Promise<PendingAttachment>   // upload; AsyncGenerator also OK
        | AsyncGenerator<PendingAttachment, void>;         //   for progress streaming
  remove(attachment: Attachment): Promise<void>;            // cancel / abort upload
  send(attachment: PendingAttachment): Promise<CompleteAttachment>; // finalize → content parts
};

PendingAttachment has status: { type: "running", progress: number } so the <Composer /> can render a progress bar. CompleteAttachment has content: ThreadUserMessagePart[] — these parts are what the model sees. For images we send { type: "image", image: "<public-url>" }; for PDFs/docs we send { type: "file", data: "<public-url>", mimeType: "...", filename: "..." } (the content-part field is data, not url — per the assistant-ui content-part shape; verified against the installed @assistant-ui/core types).

@assistant-ui/react-langgraph's useLangGraphRuntime({ ... }) accepts attachments?: AttachmentAdapter under adapters.attachments (dist/types.d.ts:228-229). Existing built-ins (SimpleImageAttachmentAdapter, SimpleTextAttachmentAdapter, CompositeAttachmentAdapter) are too narrow — they inline the bytes into the prompt via base64, which is exactly what we DON'T want for KB / large files. Verified: there is no ServerUploadAdapter exported in @assistant-ui/core@0.2.20 — the docs reference one but it's illustrative code, not a class. We must write our own.

Docs to read before implementing (in order):

1. assistant-ui — https://www.assistant-ui.com/docs/guides/Attachments (canonical guide for custom adapters).
2. assistant-ui — AttachmentAdapter reference: https://www.assistant-ui.com/docs/api-reference/runtimes/AttachmentAdapter.
3. assistant-ui — attachment UI rendering: https://www.assistant-ui.com/docs/ui/attachment (composer's AttachmentPrimitive.Root / .Name / .Remove give the chip for free; thread-side rendering is out of scope).
4. Cloudflare R2 — S3 API compatibility: https://developers.cloudflare.com/r2/api/s3/api/.

Backend — community-standard R2 wiring

Don't hand-roll an upload handler that streams the file through Next.js. The community-standard pattern (also documented in Vercel's "Using R2 with Next.js") is:

1. Presigned PUT URL — the client asks our API for a short-lived URL scoped to a specific R2 object key.
2. Direct upload — the client PUTs the file bytes straight to R2. Next.js never sees the bytes.
3. Confirm — client calls our API again to say "upload finished" with the key; we verify the object exists and persist a row in attachments.

For the S3 client, the de-facto choice is the AWS SDK v3:

• @aws-sdk/client-s3 — list / head / delete objects, validate size + content-type after upload.
• @aws-sdk/s3-request-presigner — generate the presigned PUT URL the client uses.

Both work against R2's S3-compatible endpoint (https://<accountid>.r2.cloudflarestorage.com) without any R2-specific code. No custom upload library needed — this is the well-trodden path and avoids the uploadthing / Uppy / etc. abstractions until we actually need them.

Reference implementations to crib from:

• Vercel's guide: https://vercel.com/guides/using-r2-with-nextjs — Next.js + R2 presigned URLs.
• Cloudflare's official sample: https://developers.cloudflare.com/r2/examples/aws-sdk-js/ — @aws-sdk/client-s3 against R2.
• awesome-cloudflare list for R2 patterns: https://github.com/piotrmurach/awesome-cloudflare#r2.

Proposed architecture

┌─────────┐  1. POST /api/attachments/presign   ┌──────────────┐
│ Browser │  ─────────────────────────────────→ │  Next.js     │
│(assistant│   { fileName, contentType, size }    │  (withAuth)  │
│  -ui    │   ← presigned PUT URL + key          └──────┬───────┘
│ adapter)│                                            │
│         │  2. PUT <url>  (file bytes)                │  3. head_object(key)
│         │  ─────────────────────────────────→        ▼
│         │                                  ┌──────────────────┐
│         │                                  │  Cloudflare R2   │
│         │                                  │  (attachments-   │
│         │                                  │   bucket)        │
│         │                                  └──────────────────┘
│         │  4. POST /api/attachments/confirm
│         │   { key, contentType, name, size }
│         │   ─────────────────────────────→ ┌──────────────┐
│         │   ← { id, publicUrl }            │ INSERT INTO  │
│         │                                  │ attachments  │
│         │                                  └──────────────┘
│         │  5. <Composer /> send → chat runtime receives
│            CompleteAttachment(content: [image / file parts])

Step 5 lives entirely in the browser; the LLM receives URLs and (in the image case) can fetch them. We'll rely on the same rule #10 redaction as observability spans (no token / secret leakage in spans), so the URLs R2 returns are safe to put in the prompt.

DB schema

New attachments table (Drizzle, alongside the existing lib/threads/schema.ts / lib/auth/schema.ts modules). Reasonable starter shape — adjust on review:

export const attachment = pgTable(
  "attachment",
  {
    id: text("id").primaryKey(),                          // nanoid, 12 chars
    userId: text("user_id").notNull().references(() => user.id, { onDelete: "cascade" }),
    threadId: text("thread_id").references(() => thread.id, { onDelete: "set null" }),
    messageId: text("message_id"),                        // nullable; set after the chat run commits
    r2Key: text("r2_key").notNull().unique(),             // e.g. "u/<user-id>/<nanoid>-<safe-filename>"
    name: text("name").notNull(),                         // original filename
    contentType: text("content_type").notNull(),
    size: integer("size").notNull(),                      // bytes; reject over a cap
    publicUrl: text("public_url").notNull(),              // what's emitted to the model
    createdAt: timestamp("created_at").defaultNow().notNull(),
    deletedAt: timestamp("deleted_at"),                   // soft delete; cron reaps after N days
  },
  (table) => [index("attachment_userId_idx").on(table.userId),
              index("attachment_threadId_idx").on(table.threadId)],
);

ON DELETE CASCADE on user_id (matches existing pattern, see docs/DB.md). ON DELETE SET NULL on thread_id so deleting a thread doesn't take the underlying R2 object with it (the cron job will).

API surface (new routes under app/api/attachments/)

Route	Method	Auth	Body / response	Notes
/api/attachments/presign	POST	yes	{ fileName, contentType, size } → { url, key, publicUrl }	Issues presigned PUT (5 min TTL). Rejects if size > cap or contentType not on ALLOWED_CONTENT_TYPES.
/api/attachments/confirm	POST	yes	{ key, contentType, name, size } → { id, publicUrl }	Heads the R2 object: re-reads ContentType from R2, asserts it matches the server-chosen value, asserts ContentLength matches the declared size; inserts row.
/api/attachments/[id]	GET	yes	→ { id, name, contentType, size, publicUrl, createdAt }	Per-row lookup. 404 on cross-user (rule from docs/AUTH.md).
/api/attachments/[id]	DELETE	yes	204	Soft-deletes row + deletes R2 object.

Rule #9: every route wrapped in withAuth. No exception (login is for /api/auth/* only).

Update docs/APIS.md in the same commit (rule #1).

Security — XSS / injection

This is the single biggest risk of an attachment feature and is worth designing in from day one. Five vectors to cover:

1. Spoofed Content-Type — the browser's File.type is client-supplied; a user can upload evil.svg and call it image/png. If R2 then serves it as image/svg+xml and the browser renders it inline, the SVG's <script> runs in our origin. Mitigation:

   • Server-side allow-list of MIME types (the env var R2_ALLOWED_CONTENT_TYPES defaults above already excludes SVG / HTML / XML). Reject anything else at presign time.
   • Server-chosen content type, re-verified after upload: in confirm, HeadObject the R2 object and read ContentType back; the value MUST equal the ContentType we signed into the presigned PUT URL (which the server picked from the allow-list). The browser cannot influence this because we never read its claim — we trust the server's signed choice, re-verified from R2's stored metadata. No libmagic — the server-decided allow-list makes sniffing redundant.
   • R2 upload via presigned PUT must be told the content type from the server side, not the client: pass ContentType in the PutObjectCommand when generating the presigned URL. The browser can lie but the signed URL enforces what it signed.

2. Dangerous formats that LOOK like images — SVG, HTML, XHTML, XML, EPUB. Even with the allow-list, defense-in-depth:

   • Set Content-Disposition: attachment on the R2 bucket policy for the bucket (or per-object metadata) so the browser downloads instead of rendering. Especially important for any format the browser interprets as HTML/JS.
   • If a future use case legitimately needs SVG upload, serve it from a separate origin (e.g. attachments-cdn.<PUBLIC_DOMAIN> with no cookies, no app session) so an XSS in the SVG can't reach the app's authenticated state.

3. Filename rendering — the original file.name is user-supplied and may contain <script>alert(1)</script>.pdf. Anywhere we render a filename in the UI (attachment chips in <Thread />, file picker chrome, error messages), we MUST escape it. The assistant-ui <Thread /> uses React's default JSX escaping, so plain {name} is safe; the danger is if we ever pass filenames through dangerouslySetInnerHTML, markdown rendering without escaping, or URL-construction without encoding.

   • Filenames in URLs: always encodeURIComponent. Filenames in attributes: always escape quotes.
   • Filenames as text: plain JSX {name} is fine; if we use a Markdown / rich-text renderer, ensure it doesn't allow raw HTML.

4. Public URL handling — the publicUrl we hand to the model and to the frontend is server-controlled (we generate the URL from R2_PUBLIC_BASE_URL + our own key, never from user input). Still:

   • The assistant-ui renderer will fetch the URL on the user's behalf. Confirm CSP img-src / connect-src allowlists the R2 public domain; nothing else should be reachable from the chat runtime.
   • Don't pass user-controlled strings into the URL path. The key is server-constructed (<userId>/<nanoid>-<safe-filename>); the safe-filename step strips path separators and control chars.

5. CSP — review next.config.* (or wherever the security headers live) and add:

   • Content-Security-Policy: img-src 'self' <R2_PUBLIC_BASE_URL>; connect-src 'self' <R2_PUBLIC_BASE_URL>; object-src 'none'; frame-ancestors 'none'.
   • No unsafe-inline scripts anywhere.

6. Markdown / link rendering — if the model's reply contains a link to the attachment, the renderer must not let javascript: URLs through. assistant-ui's default markdown renderer is OK, but if we add a custom link handler we have to re-check it.

Add to the TDD plan below: at minimum an XSS-filename escaping test, a server-side MIME-validation test, and a CSP-header test on the upload route.

Environment

New env vars (extend .env.example + docs/AUTH.md table):

Var	Purpose
R2_ACCOUNT_ID	Cloudflare account ID (endpoint base)
R2_ACCESS_KEY_ID	R2 API token access key
R2_SECRET_ACCESS_KEY	R2 API token secret
R2_BUCKET	bucket name (e.g. langgraph-app-attachments)
R2_PUBLIC_BASE_URL	custom domain OR https://pub-<bucket>.r2.dev
R2_MAX_BYTES	per-file cap, default 10485760 (10 MiB)
R2_ALLOWED_CONTENT_TYPES	comma-separated MIME list, default image/png,image/jpeg,image/webp,application/pdf

CD.yml and the Dockerfile stay unchanged — env-only.

Composer's adapter wiring

In app/assistant.tsx (or wherever useLangGraphRuntime({ ... }) is called):

const attachments = useMemo(
  () => new R2AttachmentAdapter({ presign: "/api/attachments/presign", confirm: "/api/attachments/confirm" }),
  [],
);

const runtime = useLangGraphRuntime({
  // ...
  adapters: { attachments },
});

The adapter implements AttachmentAdapter from dist/adapters/attachment.d.ts. Wrap it in a class so future @assistant-ui/core API bumps touch one import.

Out of scope (separate issues)

• Knowledge-base ingestion (the user already flagged this in the request). Add a tool that, given a message_id, looks up its attachments and (for PDFs / text files) extracts chunks into the existing backend/store.ts Postgres-backed store. Follow-up issue, separate lifecycle.
• Image OCR / vision model for non-image attachments.
• Attachment rendering in the assistant-ui <Thread /> (displaying thumbnails / file chips alongside messages). The composer-side chip is provided by AttachmentPrimitive.Root / .Name for free; the thread-side MessagePrimitive.Attachments + AttachmentComponents map is a separate polish item.
• Server-side virus scanning before persisting.
• Resumable / chunked uploads for files > R2's presigned PUT URL limits.

Acceptance criteria

• User can drag-drop / paste a file into <Composer /> and the upload progress shows in the UI.
• On send, the model receives image / file message parts and can reference the URL.
• Reloading the thread still shows the attached files (row in attachments is keyed to thread_id).
• DELETE /api/attachments/[id] removes both the DB row and the R2 object.
• Cross-user GET returns 404 (rule from docs/AUTH.md).
• Upload size + content-type caps are enforced server-side; oversized / disallowed uploads 4xx with a clear error surfaced to <Composer />.
• Server-side Content-Type validation: the value stored in R2 and the value in the attachments row MATCH one of the allowed MIME types (no SVG / HTML / XHTML / XML, no client-supplied types). Verified by re-reading ContentType from HeadObject in tests/api/attachments/confirm.test.ts.
• Filename rendering is XSS-safe anywhere the UI displays a filename (assistant-ui <Thread />, error toasts, MCP tool responses). React JSX default escaping is sufficient; covered by a render test with <script>-bearing filenames.
• Content-Disposition: attachment set on the R2 bucket (or per-object) so the browser downloads instead of inline-rendering anything that could carry scripts.
• CSP updated: img-src / connect-src allowlist the R2 public domain; object-src 'none'; no unsafe-inline.
• No new secrets in the client bundle — R2_* env vars are server-only; only R2_PUBLIC_BASE_URL may be exposed if needed for direct rendering (decide based on public-bucket vs signed-URL strategy).
• pnpm lint, pnpm typecheck, pnpm test all green; new tests under tests/api/attachments/ cover presign / confirm / GET / DELETE.
• docs/APIS.md updated (rule feat(001): stage 1 — user auth (Better Auth + email verification + thread ownership) #1).
• docs/AUTH.md env table updated with the new R2_* vars.

TDD plan (rule #2)

1. Red: tests/api/attachments/presign.test.ts — mock the S3 client; assert the presigner is called with the right Bucket / Key / ContentType, and that the response shape is { url, key, publicUrl }. Mock withAuth to throw a 401 path test.
2. Green: minimal app/api/attachments/presign/route.ts calling getSignedUrl(s3, new PutObjectCommand(...), { expiresIn: 300 }).
3. Red: tests/api/attachments/confirm.test.ts — mock s3.send(new HeadObjectCommand(...)) to return the expected ContentLength / ContentType; assert the DB row is inserted with the right fields. Negative case: HeadObject returns mismatched ContentType or ContentLength → 400, no row.
4. Green: confirm route handler.
5. Red: tests/frontend/assistant/r2-adapter.test.tsx — mock fetch for presign + confirm. Two critical assertions:
   • add() is implemented as an async * generator (per the AsyncGenerator<PendingAttachment, void> overload in @assistant-ui/core@0.2.20 dist/adapters/attachment.d.ts:5-12) and must explicitly yield { type: "running", progress: 1 } as its final state before returning — per assistant-ui docs: "Yield final progress so the 100% state reaches the composer." Without this yield the composer's progress UI stalls below 100%.
   • send() returns a CompleteAttachment whose content[0] is { type: "image", image: "<public-url>" } for images, or { type: "file", data: "<public-url>", mimeType, filename } for non-images — note data, not url.
6. Green: the R2AttachmentAdapter class.
7. Refactor with green tests: extract a single getS3Client() helper, add the env-cap validation.
8. XSS coverage (RED → GREEN):
   • tests/api/attachments/confirm.test.ts — extend: send ContentType: "image/svg+xml" from the client side; assert the route rejects with 400 (server-side allow-list wins, not the browser's claim).
   • tests/api/attachments/presign.test.ts — extend: request a presigned URL with ContentType: "text/html"; assert the route rejects with 400.
   • tests/frontend/assistant/attachment-render.test.tsx — render the attachment chip with name = "<img src=x onerror=alert(1)>.png"; assert the literal string appears as text and is NOT parsed into a DOM node.
   • tests/api/headers.test.ts (or whichever test covers response headers) — assert CSP includes img-src 'self' <R2_PUBLIC_BASE_URL> and object-src 'none'.

Notes for the implementer

• R2 key convention: <userId>/<nanoid>-<safe-filename> — keeps objects namespaced for easy debugging and lets us prefix-list for cleanup.
• Filename sanitization: strip path separators; refuse names longer than 256 chars.
• Public URL vs signed URL: start with a public bucket + custom domain (simplest). If security review demands it, switch to presigned GET URLs with a longer TTL and refresh-on-render.
• Avoid streaming through Next.js: even for 1 MiB files. The whole point of presigned URLs is that the browser talks to R2 directly — putting a Next.js route in the middle kills throughput and eats serverless function duration.
• No built-in server-upload adapter in @assistant-ui/core@0.2.20 — verified dist/adapters/index.d.ts only exports SimpleImageAttachmentAdapter, SimpleTextAttachmentAdapter, CompositeAttachmentAdapter. We must write our own R2AttachmentAdapter. (assistant-ui docs reference a ServerUploadAdapter — it's illustrative code, not an exported class.)
• Rule [Bug]: Memory tab Socials row missing email per linked provider #10 alignment: the attachment tool surface (when KB ingestion lands) must follow the same lazy-registration pattern — addAttachmentTool: StructuredTool | null = process.env.R2_BUCKET ? tool(...) : null. Out of scope for this issue but worth pre-empting.
• Rule feat(backend): code sub-agent + lazy-register key-needing tools + shared UI primitives #5: comments only where the R2 / S3 SDK quirks matter (e.g. "R2 ignores ACL; object is public via bucket policy, not per-object").
• XSS first-class: any line of code that takes a File.name / File.type and shoves it into a URL, attribute, or text node must include the safety check at the same time. Don't add a separate "hardening pass" later.

Related

• @assistant-ui/react-langgraph useLangGraphRuntime({ adapters.attachments }) — dist/types.d.ts:228-229.
• @assistant-ui/core AttachmentAdapter — dist/adapters/attachment.d.ts:5-12.
• assistant-ui AttachmentPrimitive (Root, Name, Remove) — composer chip primitives for free.
• lib/auth/with-auth.ts — every new route goes through this (rule [Feat]: Email verification has no result page between verify and login redirect #9).
• lib/threads/schema.ts — the existing Drizzle module pattern to mirror.
• docs/AUTH.md § Data isolation — cross-user 404 rule.
• docs/DB.md — schema conventions; new attachment table follows the same shape.
• docs/APIS.md — new routes must be added in the same commit (rule feat(001): stage 1 — user auth (Better Auth + email verification + thread ownership) #1).

[FireTable]

Follow-up decisions (2026-07-09):

Confirmed

1. Public bucket + custom domain — not presigned GET. Rationale: 10MB cap + self-hosted + small user counts; simplicity wins. Content-Disposition: attachment is set via bucket policy so browsers download instead of inline-rendering any image-resembling format that could carry scripts (SVG etc.).

2. R2 bucket CORS is required — original issue body did not include this; without a CORS rule the browser PUT to R2 is blocked before it leaves the user's tab. Add this to the bucket:

   [{
     "AllowedOrigins": ["<YOUR_PUBLIC_DOMAIN>"],
     "AllowedMethods": ["PUT"],
     "AllowedHeaders": ["Content-Type"],
     "MaxAgeSeconds": 3000
   }]

   Local dev (pnpm dev) needs http://localhost:3000 added to the same rule. Do not ship a localhost-only rule to prod.

3. messageId backfill mechanism — dropped from this issue. Research result: assistant-ui has no documented mechanism to correlate an attachment with the resulting message_id after send. Attachments become content parts inline; there is no out-of-band id exposed to the backend. Concretely:

   • Drop message_id from the attachments schema. The original promise of "set after chat run commits" was not implementable without custom useLangGraphRuntime run-metadata plumbing — out of scope here.
   • Adopted strategy (A): attachments has no messageId column. Thread-side rendering joins attachments to LangGraph messages via (thread_id, created_at) — find attachments created within the user message's send timestamp window. Slightly fuzzy but sufficient for thread-side rendering.
   • Strategy B (later, if needed): the adapter passes a custom key in runConfig.metadata; backend commits and backfills messageId. Add when we need exact attachment→messageId mapping (e.g. for KB ingestion correlation).

4. userId in R2 key — keep as drafted in the issue body. Community standard (Vercel guide, AWS blog, SO answers) is uploads/<userId>/<nanoid>-<safe-filename>. The "exposes user activity via bucket list" concern is bounded: R2 list operations require IAM regardless of the bucket's public-read policy. We keep u/<user-id>/<nanoid>-<safe-filename>. Belt-and-braces option (SHA-256 prefix of userId) deferred — not enough benefit; add only if bucket listing widens in scope.

Lazy-registered when env is missing (mirrors CLAUDE.md rule #10 for tools)

• Backend: lib/r2/client.ts throws R2NotConfiguredError when required R2_* vars are missing. Each route under app/api/attachments/ catches that error and returns 503 with body { "error": "attachments_not_configured", "message": "..." }. Not 404 (route exists, just not configured) and not 500 (not a bug). Documented in docs/APIS.md.
• Frontend: single client-visible flag NEXT_PUBLIC_ATTACHMENTS_ENABLED (default "false" in .env.example). When "false", useLangGraphRuntime({ adapters: { /* no attachments key */ } }) is called, and assistant-ui's <Composer /> skips the attachment UI automatically — no custom render gating needed.

Doc sync (same commits as code per rule #1)

• docs/附件.md — new design doc mirroring the pattern of docs/MEMORY.md / docs/OBSERVABILITY.md (architecture, R2 key, allow-list, messageId strategy, CORS gotcha, public-bucket trade-off).
• .claude/skills/langgraph-app-maintain.md — R2 key convention, messageId backfill decision, CORS requirement, Content-Disposition bucket policy reminder.
• .env.example — 7 R2_* vars + NEXT_PUBLIC_ATTACHMENTS_ENABLED="false".
• README.md — environment table gets R2 rows.
• (Code lands) docs/APIS.md — 4 routes added in the same commit (rule feat(001): stage 1 — user auth (Better Auth + email verification + thread ownership) #1).
• (Code lands) docs/DB.md — attachments table follows the existing Drizzle module convention.

[FireTable]

Implementation complete — feat/attachments-r2 (2026-07-09)

Closed the issue end-to-end on feat/attachments-r2. Three commits, 699 tests passing (19 new + 11 new adapter), DB migrated to dev.

What shipped

• Backend — POST /api/attachments/presign, POST /api/attachments/[id]/confirm, DELETE /api/attachments/[id]. All wrapped in withAuth (rule [Feat]: Email verification has no result page between verify and login redirect #9), all return 503 ATTACHMENTS_NOT_CONFIGURED on missing R2_* (rule [Bug]: Memory tab Socials row missing email per linked provider #10 lazy-register), 404 on cross-user access (no existence leak).
• DB — attachments table with FK → user (CASCADE) + FK → threads (CASCADE, nullable for the presign→confirm window). drizzle-kit migration 0002_workable_senator_kelly.sql applied.
• Frontend — R2AttachmentAdapter implementing assistant-ui's AttachmentAdapter: presign → PUT straight to R2 → confirm → returns CompleteAttachment with image/file content parts pointing at the public URL. Gated on NEXT_PUBLIC_ATTACHMENTS_ENABLED === "true" so flipping the env on both sides is the entire rollout.
• Docs — docs/ATTACHMENTS.md (architecture, key convention, messageId-deferred rationale, CORS gotcha, lazy-register), docs/APIS.md Attachments section, docs/DB.md table + cascade row, README env table + bullet + index, skills/langgraph-app-maintain.md .env.vps + CORS block.

Per-object Content-Disposition (XSS guard)

Images → inline; filename="...". Everything else → attachment; filename="...". Set on PutObjectCommand and baked into the signature. R2 has no bucket-level override, so this is the only knob — confirmed during the smoke test that GETs return the exact header that was PUT. SVG / HTML / PDF never execute inline.

Browser-verified

Picked a 68-byte PNG via the composer:

• POST /api/attachments/presign → 201 (id ab7nf3xtag1n, signed URL with X-Amz-Signature)
• PUT https://langgraph-app.*.r2.cloudflarestorage.com/... → 200
• Composer chip renders ("Image attachment"), remove button works
• Public URL: https://file.ai.firetable.tech/u/<userId>/ab7nf3xtag1n-pixel.png

Commit log

6aead41 feat(attachments): wire assistant-ui composer + design doc
51016e2 feat(attachments): presign/confirm/delete routes + tests + docs
dafaf0b feat(attachments): backend foundation — schema, R2 ops, migration
1d86c82 docs(env): document R2 vars + attachment toggle; index ATTACHMENTS.md
8558958 feat(attachments): R2 client with lazy-register on missing env
75747da chore(deps): add @aws-sdk/client-s3 + s3-request-presigner

Ready to merge after final review. Tracking next-step follow-ups (messageId backfill via runConfig.metadata, orphan-pending retention sweep) in docs/ATTACHMENTS.md § Watch-outs when extending.

[FireTable]

PDF handling research (2026-07-09)

Follow-up to "Implementation complete" above: the shipped R2 adapter emits { type: "file", data: <r2-url> } for non-image files. The OpenAI-compatible chat backend rejects file_data that's a plain HTTPS URL — it requires data:<mime>;base64,.... So PDF/markdown/text attachments hit a 400 the moment they reach the model. Probed 7 libraries + 3 path families; documenting results so we can pick the right follow-up.

What we tested (3-page Chinese resume, 1.4 MiB, mixed column layout)

#	Path	Output size	Layout preserved?	Notes
01	pdfjs-dist + manual y-bucket line recovery	5324 chars / ~1331 tok	❌ column structure lost	Solid baseline, what we shipped with
02	unpdf.extractText (mergePages: true)	5327 / ~1332	❌ same	One-line API, zero worker config
03	unpdf.extractText (mergePages: false)	5325	❌ same	Per-page strings
04	unpdf raw text items	7075	❌ same	One line per pdfjs text item
05	mupdf asText() (bad flags)	9249	❌ char-per-line	Wrong option flags
06	libpdf (Rust + NAPI)	—	—	Failed to load on macOS ARM64: prebuilt .node binary is x86_64 only. Production risk on Graviton / Apple Silicon serverless / Windows ARM. Pass.
07	mupdf asJSON() (bad flags)	8205 + bbox	❌	Same flag issue
08	mupdf render→PNG (150 DPI)	977 KiB PNG / ~2900 vision tok/page	✅ perfect	Version of choice for layout fidelity
09	mupdf asHTML()	606 KiB HTML	❌	One <p> per char, useless for LLM
10	pdfjs-dist + canvas render→base64	PNG (similar to #8)	✅	2026 mainstream stack; needs node-canvas in Node, plain <canvas> in browser
11	mupdf asText() with correct flag ("preserve-whitespace" only, NOT preserve-spans)	5487 / ~1372	❌ columns lost	Key finding: extra preserve-spans flag was forcing char-per-line. Drop it and mupdf does proper line joining.

The ceiling

No text extraction library (pdfjs, unpdf, mupdf, lopdf, anything) can recover the column structure of a multi-column PDF. The PDF format stores text as a character stream with transform matrices; extraction APIs return chars + bbox, but assembling them back into "left column then right column" requires layout analysis (clustering by x-bbox, sorting within clusters) that no JS library does for you. Both pdfjs y-bucket and mupdf preserve-whitespace produce ~1300 tokens of the right characters in roughly the right order — that is the practical ceiling for the text path.

For a two-column Chinese résumé, this means the model sees "姓名： ... 已从事研发工作近 9 年 ... Web3 DApp：熟悉 DApp 与智能合约开发；" — left column header fused into right column body. The model can usually figure it out from context, but it's not pretty.

Two viable paths

Path A — text extraction (cheap)

• 1300-1400 tokens, ~150ms parse
• Layout lost on complex PDFs
• ❌ Scanned/image-only PDFs return empty
• Implementation: pdfjs-dist + y-bucket (already shipped in this issue)
• Trade-off: ~free, but garbage for invoices / reports / anything columnar

Path B — render to image, send to vision (expensive)

• ~2900 vision tokens per page (≈6.7× text), 75-290ms render per page
• ✅ Layout perfect, including columns, tables, scanned pages
• ✅ Scanned PDFs work — LLM OCRs the image itself
• Implementation: mupdf page.toPixmap(Matrix.scale(150/72), DeviceRGB).asPNG() → base64 → existing image content channel
• Bundle: 5 MB wasm (vs 500 KB for pdfjs) — lazy-import only on PDF send

Path C — hybrid (not recommended)

• Detect "simple text-layer PDF" → Path A; "complex / scanned" → Path B
• Detection logic itself has edge cases; doubles the surface area
• Skipped: ship one path, learn from real usage, split later if needed

Decision

Going with Path B (mupdf render → image) for PDFs + keep Path A for plain .txt / .md (no need to render what is already text). Reasoning:

• The model needs the actual layout for PDFs to be useful (invoices, reports, contracts are column-heavy)
• The same image channel that already works for screenshots/photos handles the rendered pages — zero new wire format
• Scanned PDFs become free: the model reads text from the image
• Vision token cost is 6.7× text but acceptable for an opt-in attachment path; users uploading a 10-page doc are signaling "I want the model to see this"

Follow-ups parked

• libpdf: confirmed unsafe for production (no ARM prebuilt). Skip.
• MuPDF wasi / wasm-pack based alternatives: not in npm; would need a custom build. Out of scope.
• OCR-as-a-service (Google Document AI / AWS Textract / Cloudflare Workers AI 文档理解): deferred. Path B handles most of the scanned-PDF case via vision OCR already; only matters if a user complains about OCR accuracy on a specific doc, at which point we evaluate.
• Layout analysis layer (x-bbox clustering) for Path A: not pursued — Path B already wins on layout fidelity at acceptable cost.

Files (artifacts, not committed)

All probes are at ~/.local/pdfjs-probe/outputs/ (outside the repo):

• 01-pdfjs-dist.md … 11-mupdf-official.md
• 08-mupdf-render/ — 3 rendered PNGs (1240×1754 each)
• 09-mupdf-html/ — mupdf's HTML output (unusable for LLM, kept for reference)
• 10-pdfjs-render/ — pdfjs+canvas render outputs

Re-run any probe with cd ~/.local/pdfjs-probe && node probe-<name>.mjs.

[FireTable]

Implementation plan — PDF → markdown (M1, no KB)

Scope (this issue, M1 only): fix PDF uploads by converting to markdown before sending to the LLM. KB / RAG / Vectorize / Document Agent are explicitly deferred (YAGNI).

Flow

User uploads PDF in chat
    ↓
Existing R2 upload (already shipped) → { type: "file", data: <r2-url> }
    ↓
NEW: POST /api/attachments/parse { attachment_id, thread_id, message_id }
    ↓
1. Fetch bytes from R2
2. mupdf render: each page → PNG (default 150 DPI, scale=150/72)
3. Call vision LLM with N pages per request (cap ~5-8 to stay under
   vision-context limits, multi-round for long PDFs)
4. Aggregate per-page markdown → single markdown doc
5. Write markdown to R2: threads/{thread_id}/{message_id}.md
6. Update attachments row: markdown_path, page_count, parsed_at
7. Replace the { type: "file" } content in the message with
   { type: "text", text: <markdown> } before it reaches the model

Key decisions

• mupdf server-side (Node, not browser). Reasons: (a) consistent across clients, (b) prepares for future KB extraction, (c) avoids shipping 5 MB wasm to the browser just for upload-time rendering.
• Vision LLM is the OCR. No separate OCR service (PaddleOCR-VL / Unlimited-OCR / Gemini-OCR) for M1 — main LLM's vision handles scanned + structured PDFs.
• Batched vision calls. 5-8 pages per request to balance context cost vs API roundtrips. Long PDFs get multiple rounds; we stitch per-page markdown back into one doc.
• Markdown stored alongside the file in R2. Cheap, reusable for re-prompts, downloadable, renderable by the frontend.
• Defer (NOT in this issue): KB / Vectorize / Document Agent / ColPali / multi-tenant metadata / async job queue / page-stitched images.

Files (planned)

• lib/pdf/render.ts — mupdf wrapper, returns Buffer[] per page
• lib/pdf/to-markdown.ts — vision-LLM call with batched pages
• lib/pdf/index.ts — orchestrator (fetch → render → md → write R2 → update DB)
• app/api/attachments/parse/route.ts — API entry, withAuth, validates input
• tests/api/attachments/parse.test.ts — 401 / 400 / happy path / batch boundary
• lib/pdf/*.test.ts — render output shape, batch chunking logic
• DB: add markdown_path, page_count, parsed_at columns to attachments (migration)

Acceptance

• Upload PDF in chat → model receives markdown, not a 400
• Upload TXT / MD / image — unchanged behavior
• 10-page PDF → ≤3 vision calls, single round-trip feel to user
• Re-upload same file → idempotent (cache by R2 key hash)
• Markdown lives in R2, frontend can render via existing markdown viewer