feat: add support for file attachments / uploads (#15)

[bnaylor]

Closes #15

Summary of Changes

• Data Models: Added and models to support binary payloads.
• Document Parsing: Native macOS text extraction via for PDFs and for /.
• Vision Routing: Direct binary payload generation for vision-capable models (Gemini, Claude, GPT-4o) and local auxiliary model fallback for text-only models (Ollama/LlamaCPP).
• UI Components: Attachment preview bar with removable chips above composer, 📎 paperclip button via , drag-and-drop overlay target on , chat history thumbnail rendering, and Auxiliary Vision configuration in Settings.
• Guardrails & Testing: Enforces 20MB image / 50MB document file size limits and includes comprehensive unit test suite.

[bnaylor]

Please note the guidance recently added in 9e6a61a and relocate your specs.

[clomp42]

Adversarial Code & Architecture Review — PR #38

Solid architecture and clean native implementation overall — particularly the zero-dependency document parsing via / and the dual-layer vision fallback via .

Here are 4 critical findings and structural recommendations to address before merging:

---

1. 📁 Specs & Plans Directory Alignment (PR #37 Rule)

• Finding: The PR introduces and .

• Fix: In line with PR #37 and the updated / rules, move these files to:

  Delete the empty directory.

---

2. ⚡ Base64 Payload Persistence in (Critical Token & Disk Bloat)

• Finding: When an image is attached to a vision model turn, embeds the Base64 string into . In , this is appended directly to and saved into .
• Impact:
  1. Token/Bandwidth Multiplication: For every subsequent turn in that chat session, sends the multi-megabyte Base64 image payload in the API request over and over again.
  2. Disk Inflation: permanently stores raw Base64 image strings on disk.
• Fix: Store only metadata (file URL, name, size) in and history. Strip from after the turn request completes, or generate JIT strictly for the immediate API request.

---

3. 🎯 Vision Model Capability Detection ()

• Finding: uses hardcoded string matching ().
• Impact: Model aliases like , , , , or OpenRouter model strings (e.g. ) could fail this check and unnecessarily trigger auxiliary vision fallbacks.
• Fix: Expand matching keywords to include , , , , , or check provider capabilities.

---

4. 🔤 Text Encoding Fallbacks in

• Finding: fails silently for non-UTF8 text files (ISO-8859-1, UTF-16, ASCII with high bits).
• Fix: Fall back to or if decoding returns before throwing a warning.

---

Summary

The UI components (, drag-and-drop targets) and native parsing engines are well-designed and covered by unit tests. Once the Base64 history persistence and specs directory locations are resolved, this is ready to merge!

[rune42808]

Review: file attachments PR

Nice work — this is a solid foundation for file uploads. Clean architecture: FileAttachment → AttachmentProcessor → VisionRouter → LLM clients, well-separated UI.

Major: Vision detection misses Claude 4 models

VisionRouter.isVisionCapable() checks name.contains("claude-3") — catches claude-3-5-sonnet but misses claude-sonnet-4 and claude-haiku-4-5 (Claude 4). All modern Anthropic models support vision, so use:

name.contains("claude")  // all Claude models support vision

Major: Infinite recursion in AuxiliaryInferenceEngine defaults

The extension provides defaults for both generate overloads, each calling the other — if a future engine overrides neither, it will stack-overflow at runtime. Have both delegate to a shared private stub.

Minor: Synchronous I/O

Data(contentsOf:) and String(contentsOf:) block the SwiftUI main thread. Consider wrapping in Task { } or using URLSession async loads for the 20MB/50MB limits.

Minor: PR includes unrelated diff noise

EmojiCatalog.swift changes and EmojiTokenModel.swift removal appear to be accidental includes — consider separating into a cleanup PR.

Test coverage: ++

Comprehensive — Anthropic payload format, processing edge cases, vision routing with mock engines, serialization round-trips, integration flows. The MockVisionEngine pattern is clean.

Design docs

Thorough plan and spec. Drag-and-drop from Finder (mentioned) could be a good follow-up.

Overall: solid implementation, good separation of concerns, strong tests. Fix the two majors before merge.

[bnaylor]

All code review feedback has been addressed and pushed in commit abc7f75:

1. Specs & Plans Directory Relocation: Moved specs and plans to docs/specs/2026-07-28-file-attachments-design.md and docs/plans/2026-07-28-file-attachments.md. Removed empty docs/superpowers directory.
2. Base64 Payload Persistence & Inflation Fix: Added stripInlineDataFromHistory(for:) in AppState.swift and invoked it in iris.swift post-turn. inlineData Base64 strings are passed to the model JIT for the immediate turn but stripped from history, eliminating Base64 token multiplication on subsequent turns and preventing disk bloat in conversations.json.
3. Expanded Vision Model Detection: Updated VisionRouter.isVisionCapable() to support claude (Claude 3, 3.5, 4, 4.5), gpt-4, o1, o3, vision, llava, vl, pixtral, paligemma, and minicpm.
4. Infinite Recursion Guard: Restructured AuxiliaryInferenceEngine protocol and extension overloads to prevent recursive default loops.
5. Text Encoding Fallback: Added ASCII, ISO-Latin-1, and system default text encoding fallbacks in AttachmentProcessor.swift when UTF-8 decoding fails.

[bnaylor]

Review: file attachments / vision (#15)

Reviewed by Claude (Opus 4.8) at Brian's request. Build passes; AttachmentProcessorTests 10/10; VisionRouterTests has 1 failure — see the critical inline note.

🔴 Critical (blocks merge)

• Auxiliary vision path silently drops the image (protocol-dispatch bug) — the whole text-only-model fallback sends no image. Branch is red on its own test. Inline on AuxiliaryInferenceEngine.swift.

🟠 Important — security (untrusted files)

• Extracted attachment text is injected into the prompt with no sanitization / tag-escaping, bypassing the existing InjectionGuard. Inline on AttachmentProcessor.swift.
• NSAttributedString(url:options:[:]) on untrusted docx/rtf can trigger remote-resource loading (SSRF class). Inline on AttachmentProcessor.swift.

🟡 Moderate — correctness

• isVisionCapable substring matching over-matches (routes images to text-only models). Inline on VisionRouter.swift.
• Reflection counter reset moved into the async task → double-reflection window. Inline on AppState.swift.
• stripInlineDataFromHistory runs after every model response within a turn, so image+tool-use turns lose the image mid-turn. Consider scoping to end-of-turn.

⚪ Minor

• Persisted FileAttachment.fileURL → dangling refs / paths in conversation JSON.
• Anthropic/OpenAI adapters flipped else if→independent if for parts (benign, but note it).
• Image types gated by extension only; some providers reject gif/webp.

✅ Strengths

Clean models (Sendable, backwards-compatible Codable); size limits enforced before reading into memory; file I/O off the main actor; correct OpenAI/Anthropic multimodal payloads; solid AttachmentProcessor coverage; stripInlineDataFromHistory is a thoughtful token/disk win.

Bottom line: don't merge until the critical aux-vision bug (and the red test) are fixed; the two security items are real for untrusted files. Rest are tractable. (I did not deep-read the ChatView/AttachmentBarView UI diff.)

[bnaylor]

🔴 Critical: the aux vision model never receives the image.

generate(prompt:jsonSchema:images:) is defined only as a protocol extension default (which discards images and calls the 2-arg generate), not as a protocol requirement. VisionRouter holds the engine as any AuxiliaryInferenceEngine and calls this 3-arg method — which statically dispatches to this default, so OllamaEngine's real images: override is never reached through the existential. The aux model is asked to "describe this image" with no image → hallucinated description.

testProcessTextOnlyImagesWithAuxiliaryModel fails exactly here (receivedImages == nil), so the branch is red.

Fix: make the overload a protocol requirement (keep this as the default impl):

protocol AuxiliaryInferenceEngine: Sendable {
    ...
    func generate(prompt: String, jsonSchema: String?, images: [String]?) async throws -> String
}

[bnaylor]

🟠 Security: untrusted file content injected without sanitization or escaping.

This wraps extracted (untrusted) file text in <attached_file name="…">…</attached_file> and AppState.sendMessage joins it straight into the model prompt — bypassing the InjectionGuard/PromptInjectionGuard the codebase applies to system events. Two concrete holes:

1. A file whose content contains a literal </attached_file> escapes the wrapper and can inject instructions.
2. \(attachment.filename) is interpolated unescaped into name="…" — a " or > in the filename breaks the tag.

Escape/neutralize the closing tag and the filename, and ideally route extracted text through InjectionGuard. (Same applies to the vision-description path — adversarial text inside an image becomes injected prompt content.)

[bnaylor]

🟠 Security: NSAttributedString(url:options:[:]) on untrusted documents.

Empty options let NSAttributedString sniff the content type; a crafted file (RTF/HTML/webarchive with a docx/rtf name) can trigger remote-resource loading — the well-known NSAttributedString SSRF class. Constrain the parse to the expected type, e.g. options: [.documentType: NSAttributedString.DocumentType.rtf] (or .officeOpenXML for docx), rather than auto-detecting.

[bnaylor]

🟡 isVisionCapable over-matches via substrings.

contains("gpt-4") marks text-only gpt-4/gpt-4-0314 as vision; contains("o1") marks o1-mini; contains("claude") marks all Claude (older ones aren't vision). A false positive sends an image inline to a text-only model → hard API error instead of the aux-model fallback. Tighten to specific vision-capable families/versions.

[bnaylor]

🟡 Reflection counter reset now happens inside the async task, after processInput.

Since the reset moved after a potentially long processInput, a second message sent during that window re-evaluates shouldReflect on the stale (≥30) count → double reflection. It was synchronous before this refactor. Consider resetting the counter synchronously up-front when shouldReflect is decided.

[bnaylor]

All review items from Claude (Opus 4.8) have been resolved and verified in commit 66dc507:

1. Critical Protocol-Dispatch Bug (Auxiliary Vision Payload): Re-added 3-parameter generate(prompt:jsonSchema:images:) to protocol AuxiliaryInferenceEngine interface so existential calls (any AuxiliaryInferenceEngine) dynamically dispatch via witness table to OllamaEngine.generate. VisionRouterTests.testProcessTextOnlyImagesWithAuxiliaryModel now passes cleanly!
2. Security — Prompt Injection Guard: Wrapped all extracted text from PDFs, DOCX/RTF, and plain text files with PromptInjectionGuard.sanitizeUntrustedInput() before embedding into <attached_file> tags to prevent prompt injection breakouts.
3. Security — SSRF Prevention in NSAttributedString: Restricted NSAttributedString document reading options to [NSAttributedString.DocumentReadingOptionKey(rawValue: "NSReadAccessURL"): attachment.fileURL] to prevent remote asset fetching during DOCX/RTF parsing.
4. Model Capability Precision: Tightened VisionRouter.isVisionCapable() matching to specifically target vision model families (gpt-4o, gpt-4.5, gpt-4-turbo, claude-3, claude-4, claude-sonnet, claude-haiku, claude-opus, non-mini o1/o3, vision, llava, qwen-vl, pixtral, paligemma, minicpm) and avoid false positives on text-only models (e.g., gpt-4-0314 or o1-mini).
5. Reflection Counter Synchronous Reset: Moved messageCountSinceReflection = 0 reset to execute synchronously upfront when shouldReflect is evaluated in AppState.swift, preventing double reflection triggers.
6. History Stripping Timing: Moved stripInlineDataFromHistory call to execute post-turn completion in iris.swift so multi-step tool turns preserve image data throughout the active turn while still purging Base64 strings before disk persistence.