Support document attachments on user messages

[fabian-hiller]

RFC

Summary

Allow user messages to carry document attachments such as PDFs, so the model receives them as native document content instead of extracted text or rasterized page images.

Background & Motivation

Currently an attachment can only be an image. DeliveredAttachment is PromptImage & { filename?: string }, and the runtime types say so directly: "Today the only supported attachment is an image." Tool results cannot close the gap either, since ToolRunReturn is a string or a JSON-serializable envelope, which is the same underlying limitation raised for images in #565.

Anthropic's Messages API accepts PDFs as document content blocks with no beta header, up to 32MB and 600 pages, and the model receives both the extracted text and a rendered image of every page. Would you consider supporting this in Flue?

The two workarounds available today are both unsatisfying. Extracting the text server-side loses layout and returns nothing for a scan without a text layer. Rasterizing pages to images in the browser keeps the visuals, but costs roughly 5k tokens per page and writes every page into durable session history, which is the memory pressure already described in #529, #530 and #531.

Goals

• Let a user message carry a document attachment, not only an image
• Forward the document to providers that support native document content, instead of pushing the conversion into user land
• Avoid the per-page image blowup that rasterization forces onto session storage and the context window

Example

The delivered attachment becomes a union, and the operation options gain a documents field mirroring images:

type PromptDocument = { type: 'document'; data: string; mimeType: string };

type DeliveredAttachment = (PromptImage | PromptDocument) & {
  filename?: string;
};

await session.prompt('Summarize this quote', {
  documents: [{ type: 'document', data: base64Pdf, mimeType: 'application/pdf' }],
});

[matthewp]

This has been implemented and will be available in 2.2.