v2.3.1

What's Changed

Aivory 2.3.1 improves how the Python sandbox handles the files a conversation uses. Every conversation upload is now staged in its original form — so the model can edit an existing PDF, DOCX, PPTX, or XLSX without reconstructing it — and it adds a fetch_image tool that downloads public images into the sandbox through the backend's SSRF-safe client. This release also restores OpenAI chat reasoning across tool rounds, hardens web_fetch against unreachable origins with a reader-service fallback, and refines image handling on text-only models and across turns.

Sandbox Conversation Files

• Stages every conversation upload into /workspace/uploads/ in its original bytes, removing the earlier format allowlist. The model can now read and write the actual Office/PDF files the user attached instead of a regenerated approximation.
• Preserves document layout and formatting for targeted edits, guiding the model to prefer Python libraries such as python-docx, python-pptx, and pypdf over reconstructing content.
• Keeps uploads bounded by the existing conversation-upload size limit and resets the staging namespace between calls so stale or removed files do not linger.
• Advertises uploads in the sandbox guidance for every supported language (English, Simplified Chinese, Traditional Chinese, Japanese, French).

Public Image Downloads (fetch_image)

• Re-enables image downloads into the conversation sandbox with a fetch_image tool. The Python runner stays network-isolated; all outbound access goes through the backend's SSRF-safe client.
• Validates the URL scheme, host, and port, rejects private/loopback/metadata targets, enforces a byte cap, and verifies the bytes are a supported image before staging.
• Stages verified images under /workspace/downloads/, persists the sandbox session across turns, and rebuilds the session if the container is reclaimed.
• Adds signal and data coverage for image boundaries in the sandbox, including session-gone rebuilds and workspace policy denial.
• Adds per-turn and Deep Research limits for fetch_image and a per-invocation download timeout, all env-tunable and documented in docs/config-reference.md.
• Blocks fetch_image in fast turns alongside python_execute and subjects it to the AllowSandbox workspace policy.
• Adds the "Download image" entry to the selectable built-in-tool catalog.

OpenAI Chat Reasoning Across Tool Rounds

• Replays reasoning_content on request when replaying prior OpenAI chat turns, so reasoning blocks produced in earlier rounds are forwarded correctly.
• Streams OpenAI chat reasoning deltas into the same tool-loop events as non-reasoning content, restoring reasoning display continuity in longer tool-enabled conversations.
• Adds regression coverage for reasoning replay across a multi-turn tool loop.

Web Fetch Resilience

• When the origin is directly unreachable (filtered or black-holed route), retries the read through a Jina Reader-compatible text-extraction endpoint instead of hanging until the tool budget is exhausted.
• Runs the direct attempt on a short sub-timeout so a black-hole cannot consume the whole tool budget, and keeps the reader hop SSRF-safe — private, loopback, and metadata targets are never handed off.
• Relaxes the web_fetch timeouts (15s→30s tool, 25s→40s client) and adds regression coverage for the fallback path.

Chat, Images, and Error Handling

• On text-only models, warns that images will be ignored and strips image attachments from the request instead of blocking the send; the model switch itself warns when conversation history contains images.
• Converts phone formats vision providers cannot reliably decode (HEIC/HEIF, TIFF, BMP, AVIF, ICO) even when small, while keeping compatible formats byte-for-byte through 3 MiB.
• Supports paste-to-attach anywhere in the composer from clipboardData.files, deduped against the editor's own paste handler.
• Preserves image edit context across turns and skips restore for optimistic new-chat temp ids so new conversations no longer 404 with a spurious restore error.
• Sanitizes failed tool output and user-visible errors so internal paths, URLs, and secrets are never streamed or persisted.
• Wraps multiple image attachments to prevent overflow, and tightens admin users row alignment.

Upgrade Notes

• Database schema is unchanged in this release; no manual SQL is required.
• No configuration changes are required. New optional env vars (AIVORY_TOOLS_WEB_FETCH_JINA_FALLBACK, AIVORY_TOOLS_WEB_FETCH_JINA_BASE, AIVORY_TOOLS_WEB_FETCH_DIRECT_TIMEOUT, AIVORY_LLM_PER_TURN_TOOL_LIMITS_FETCH_IMAGE, AIVORY_LLM_DEEP_RESEARCH_TOOL_LIMITS_FETCH_IMAGE, AIVORY_LLM_TOOL_TIMEOUTS_FETCH_IMAGE) default to safe values and are documented in docs/config-reference.md.
• fetch_image requires the Python sandbox to be configured and to be enabled by the active workspace policy.

Validation

• npm run typecheck
• npm run lint
• npm run test
• npm run build
• go test ./... from server/
• go build ./... from server/
• Locale JSON validation with jq empty
• git diff --check

Release Scope

• Sandbox conversation files and fetch_image tool: cafe633 (fix(images): preserve edit context across turns) and d1f9098 (feat(sandbox): stage all conversation uploads).
• OpenAI chat reasoning replay: 7b135c3 (fix(openai): replay chat reasoning content).
• Web fetch reader fallback: dcd74bc (fix(tools): fall back to a reader service when web_fetch can't reach the origin).
• Text-only model images and paste-to-attach: e525bc3 (fix(chat): ignore images on text-only models and support paste-to-attach).
• Image attachment wrapping: b1d9b14 (fix(ui): wrap multiple image attachments).
• Admin users row alignment: 39c9cf3 (fix(ui): tighten admin users row alignment).

Full Changelog: v2.3.0...v2.3.1