fix: Ask Agent indexes and attaches files before sending to chat#725
fix: Ask Agent indexes and attaches files before sending to chat#725kovtcharov merged 3 commits intomainfrom
Conversation
The Ask Agent button in the file browser closed the modal but dispatched a CustomEvent that nothing listened for, and performed no file indexing. The agent received no document context. - Extract shared `indexAndAttachFiles` helper from `handleIndexSelected` - Rewrite `handleAskAgent` to index files, attach to session, then set `pendingPrompt` in the Zustand store - Make ChatView's `pendingPrompt` consumption reactive so it works when the chat is already open (not just on mount) - Add try/finally to ensure loading state is always cleared - Add backend tests for index→attach→chat document_ids flow Closes #583
…surface partial failures (#583) - ChatView: add cancelled flag to requestAnimationFrame cleanup to prevent deferred send firing on unmounted component - FileBrowser: wrap handleIndexSelected body in try/catch/finally so indexingFiles is always cleared on unexpected errors (stuck spinner fix) - FileBrowser: surface partial-failure warning in handleAskAgent before closing modal so user knows when some selected files failed to index
|
@claude review this PR |
|
Claude finished @itomek's task —— View job Reviewing PR #725
Code Review SummaryThis PR correctly fixes the "Ask Agent" functionality which was dispatching events that nothing listened to (dead code). The fix is well-structured and the approach is sound. Positive Observations
Issues Found🟡 Important
🟢 Minor
Suggestions for Future Improvements
VerdictThe PR successfully fixes the Ask Agent workflow and follows good React patterns. The filename extraction bug (item 1) is worth addressing but doesn't block the fix - the worst case is showing the wrong filename in the prompt, which the LLM can still process correctly since the documents are properly attached. Approved for merge with optional fix for the filename alignment issue. |
…mismatch (#583) The previous prompt-building logic used index-based slicing (supported.filter((_, i) => i < docIds.length)) which produced wrong filenames when a middle file failed — e.g. if B fails from [A,B,C], docIds=[A_id,C_id] but the slice yielded names [A,B] instead of [A,C]. Fix: add succeededFiles[] to IndexResult, populated only when indexing actually produces document IDs. handleAskAgent now uses succeededFiles directly, eliminating the index/identity mismatch entirely. Also add test_upload_without_attach_has_no_rag_context to verify that uploading a document without attaching it to the session does not leak RAG context into the chat response.
|
@claude review this PR |
|
Claude finished @itomek's task —— View job Reviewing PR #725
Code Review SummaryThis PR correctly fixes the "Ask Agent" functionality which was dispatching events that nothing listened to (dead code). The fix is well-structured and the approach is sound. Positive Observations
Issues Found🟢 Minor
Suggestions for Future Improvements
VerdictThe PR successfully fixes the Ask Agent workflow and follows good React patterns. The minor issues noted above are non-blocking improvements. Approved for merge ✅ |
Fixes #728. ## What was broken Drag-and-drop document upload failed with *"Indexing error: Not found. The file or folder may have been moved or deleted."* in any browser, and in the packaged Electron desktop app (Electron 40). **Root cause:** `src/gaia/apps/webui/src/components/DocumentLibrary.tsx:265` relied on `(file as any).path`, an Electron-only augmentation that's `undefined` in every browser and was [removed in Electron 32+](https://www.electronjs.org/blog/electron-32-0). The fallback `file.name` (bare filename, no directory) was POSTed to `/api/documents/upload-path`, which couldn't resolve it → HTTP 404 → misleading frontend error. ## Fix New `POST /api/documents/upload` endpoint that accepts the file content as multipart form data, streams it to `~/.gaia/documents/<shortid>_<original_name>.<ext>` with a 20 MB cap, dedupes by SHA-256 hash, and indexes via the standard RAG pipeline. Drag-and-drop in both DocumentLibrary and ChatView now use this endpoint. Delete cleans up server-owned files; user-owned files from `upload-path` are left alone. Also fixes `getFriendlyError()` in `api.ts` to prefer the backend's `detail` over the canned 404 string, so future upload errors surface meaningful messages. ### Details worth knowing - **On-disk filename embeds the original stem** (`<12-hex-id>_<sanitized_name>.<ext>`) so the agent LLM and RAG tools can recognize the file by its human-readable name. A pure-UUID filename caused the LLM to hallucinate paths (e.g. `C:\Users\14255\.gaia\documents\<uuid>.pdf` → `/Users/14255/Documents/<uuid>.pdf`) and fail tool lookups. - **Filename sanitizer** strips Windows-illegal chars (`<>:"/\|?*`), control chars, leading/trailing whitespace and dots, and caps length at 80 chars — keeps total path well under Windows MAX_PATH. - **12-hex-char ID** instead of full UUID. We dedup by SHA-256 first, so identical-content collisions are impossible; the only theoretical collision is concurrent uploads of *different* content drawing the same ID, with probability ~1 in 1.4e14 even at 1M concurrent uploads. Saves 24 chars per filename. - **Reliability**: streaming write with abort-on-overflow, try/finally partial cleanup (survives ClientDisconnect), atomic `os.replace`, orphan cleanup on indexing/db/race failures. - **Chat-view drop UX**: dropping onto the chat area indexes quietly without forcing the Document Library side panel open. Dropping onto the library panel itself still opens it (correct for users intentionally managing their library). ### Bundled cleanup This PR also includes a doc-consistency pass auto-applied by `util/lint.py --all --fix` (step 8) that aligns the supported-extension allowlist across backend (`utils.py`, `files.py`), frontend (`UnsupportedFeature.tsx`), and tests. ### What's NOT changed - `POST /api/documents/upload-path` is untouched — still used by the server-side file browser in `FileBrowser.tsx`, where the user explicitly picks a real filesystem path. - The Electron preload — deliberately not adding `webUtils.getPathForFile` since it's [broken for drag-drop in Electron 30-33+](electron/electron#44600) and the blob path works everywhere. ### Known limitation / follow-up The agent LLM still occasionally cites the on-disk filename (e.g. `9c59c981f8e7_report.pdf`) when answering questions about a document, instead of the user's display name (`report.pdf`). This is because `src/gaia/agents/chat/tools/rag_tools.py` returns chunk metadata using raw filepaths from `self.rag.indexed_files` rather than translating them to the display name stored in `documents.filename`. **This is a separate, pre-existing concern in the RAG tool layer** and fixing it requires a focused refactor of `rag_tools.py` (~1800 lines, touches `query_documents`, `query_specific_file`, `summarize_document`, `search_indexed_chunks`, `list_indexed_documents`). Tracking it as a follow-up rather than expanding this PR's scope. ### Conflict notes for reviewers - **#723** (surface indexing failures) also touches `src/gaia/ui/routers/documents.py`, but edits the body of `upload_by_path` — this PR only adds new top-of-file constants, helpers, and a new `upload_document_blob` function inserted after `upload_by_path`. Three-way merge should be clean. Semantically, #723's "raise on failure" change is **more correct** under my new endpoint than the previous silent-0-chunk behavior. - **#725** (Ask Agent file indexing) touches `ChatView.tsx` in a disjoint line range. No conflict. --- ## Steps to reproduce the bug (before this fix) ### One-time setup ```bash git checkout main uv venv uv pip install -e ".[dev,ui]" cd src/gaia/apps/webui && npm install && npm run build && cd - ``` > **Windows PowerShell users:** quote the extras → `uv pip install -e ".[ui]"`. Without quotes PowerShell silently strips them. ### Start the services ```bash # Terminal 1 lemonade-server serve # Terminal 2 gaia chat --ui ``` ### Reproduce 1. Open the UI in **Firefox** (most reliable repro; the bug also reproduces in Chrome and the packaged Electron 40 desktop app). 2. Open DevTools → Network tab, filter on `documents`. 3. Open or create a chat session. 4. Drag any `.pdf`/`.txt`/`.md`/`.docx` from your file manager onto the Document Library drop zone or the chat area. 5. **Observe:** - UI: *"Indexing error: Not found. The file or folder may have been moved or deleted."* - Network: `POST /api/documents/upload-path` → **HTTP 404** - Request body: `{"filepath":"report.pdf"}` (bare filename — the smoking gun) - Backend log: `HTTPException: File not found: report.pdf` ### Verify the fix ```bash git checkout kalin/fix-728-document-upload cd src/gaia/apps/webui && npm run build && cd - # Restart `gaia chat --ui` ``` Repeat the drop. Expected: - Network: `POST /api/documents/upload` (multipart, **not** `upload-path`) → **HTTP 200** - File persisted at `~/.gaia/documents/<shortid>_report.pdf` with the original name embedded - Backend log: `Blob upload indexed: report.pdf (<N> bytes, <M> chunks)` - Drop the same file twice → `Blob upload dedup: report.pdf matches existing doc <id>` (no second file on disk) - Ask the agent about the doc → answers correctly using the indexed content ## Test plan - [x] 10 new integration tests (`tests/integration/test_documents_router.py`): happy path, dedup, empty/oversized/bad-extension/illegal-chars rejection, indexing failure cleanup, delete unlinks server-owned, delete preserves user-owned, 404 on delete - [x] Adjacent regression sweep: 186 tests pass (files router, folder indexing, database, server unit tests) - [x] `python util/lint.py --all --fix` clean - [x] `tsc --noEmit` clean (no new errors in changed files) - [x] End-to-end smoke test via TestClient (upload → list → delete → file unlinked) - [x] **Manual (Windows + Firefox)**: drag-and-drop indexes successfully → ✅ validated - [x] **Manual**: dropped doc auto-attaches to chat session → ✅ validated - [x] **Manual**: dedup short-circuit verified → ✅ validated - [x] **Manual**: delete cleanup verified for both server-owned and user-owned files → ✅ validated - [x] **Manual**: agent answers questions correctly using the indexed document → ✅ validated (display-name citation polish tracked as follow-up; see "Known limitation" above) --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
# GAIA v0.17.3 Release Notes GAIA v0.17.3 is an extensibility and resilience release. You can now package your own agents into a custom GAIA installer and seed them on first launch, point GAIA at alternative OpenAI-compatible inference servers from the C++ library (Ollama, for example), and start from three new reference agents (weather, RAG Q&A, HTML mockup) that execute against real Lemonade hardware in CI. It also hardens the RAG cache against an insecure-deserialization class of bug (CWE-502) — all users should upgrade. **Why upgrade:** - **Ship your own GAIA** — Export and import agents between machines, follow a new guide to produce a custom installer that seeds your agents on first launch, and on Windows install everything in one step because the installer now includes the Lemonade Server MSI. - **Work with alternative inference backends** — The C++ library now preserves OpenAI-compatible `/v1` base URLs instead of rewriting them to `/api/v1`, so servers that expose the standard `/v1` path (Ollama, for example) work out of the box. - **Start from a working example** — Three new reference agents (weather via MCP, RAG document Q&A, HTML landing-page generator) with integration tests that actually execute against Lemonade on a Strix CI runner. - **Safer RAG cache** — Replaces `pickle` deserialization with JSON + HMAC-SHA256 (CWE-502). Unsigned or tampered caches are rejected and transparently rebuilt on the next query. - **Better document handling** — Encrypted or corrupted PDFs now produce distinct, actionable errors (`EncryptedPDFError`, `CorruptedPDFError`) instead of generic failures, and the RAG index is hardened for concurrent queries. --- ## What's New ### Custom Installers and Agent Portability You can now package a custom GAIA installer that ships with your own agents pre-loaded, and move agents between machines with export/import (PR #795). On Windows, the official installer now includes the Lemonade Server MSI and runs it during install, so a fresh machine has the complete local-LLM stack after a single download (PR #781). **What you can do:** - Export an agent from `~/.gaia/agents/` to a portable bundle with `gaia agents export` and import it on another machine with `gaia agents import` - Follow the new custom-installer playbook at [`docs/playbooks/custom-installer/index.mdx`](/playbooks/custom-installer) to distribute GAIA with your agents pre-loaded — useful for workshops, team deployments, and internal tooling - On Windows, the installer now includes Lemonade Server — no separate download for a complete first-run experience **Under the hood:** - `gaia agents export` / `gaia agents import` CLI commands round-trip agents between machines as portable bundles - First-launch agent seeder (`src/gaia/apps/webui/services/agent-seeder.cjs`) copies `<resourcesPath>/agents/<id>/` into `~/.gaia/agents/<id>/` the first time the app starts - Windows NSIS installer embeds `lemonade-server-minimal.msi` into `$PLUGINSDIR` and runs it via `msiexec /i ... /qn /norestart` during install (auto-cleaned on exit) --- ### Broader Backend Compatibility in the C++ Library The C++ library now preserves OpenAI-compatible `/v1` base URLs (PR #773) instead of rewriting them to `/api/v1`. That means inference servers that expose the standard OpenAI `/v1` path — for example, Ollama at `http://localhost:11434/v1` — work out of the box without needing a special adapter. --- ### Reference Agents and Real-Hardware Integration Tests Three new example agents and a Strix-runner CI workflow land together (PR #340). **What you can do:** - Copy `examples/weather_agent.py`, `examples/rag_doc_agent.py`, or `examples/product_mockup_agent.py` as a starting point for your own agents - Run the new integration tests locally against Lemonade to validate agents end-to-end, not just structurally **Under the hood:** - `tests/integration/test_example_agents.py` executes agents and validates responses with a 5-minute-per-test timeout - `.github/workflows/test_examples.yml` runs on the self-hosted Strix runner (`stx` label) with Lemonade serving `Qwen3-4B-Instruct-2507-GGUF` - Docs homepage refreshed with a technical value prop ("Agent SDK for AMD Ryzen AI") and MCP / CUA added to the capabilities list --- ### Smarter PDF Handling in RAG Encrypted and corrupted PDFs now surface as distinct, actionable errors (`EncryptedPDFError`, `CorruptedPDFError`, `EmptyPDFError`) instead of generic failures or silent 0-chunk indexes (PR #784, closes #451). Encrypted PDFs are detected before extraction; corrupted PDFs are caught during extraction with a clear message. Combined with the indexing-failure surfacing in PR #723, you get a visible indexing-failed status the moment a document fails — and the RAG index itself is now thread-safe under concurrent queries (PR #746). --- ## Security ### RAG Cache Deserialization Replaced with JSON + HMAC Fixes an insecure-deserialization issue in the RAG cache (CWE-502, PR #768). Previously, cached document indexes were serialized with Python `pickle`; if an attacker could write to `~/.gaia/` — via a shared drive, a sync conflict, or a malicious extension — loading that cache could execute arbitrary code. v0.17.3 replaces `pickle` with signed JSON: caches are now serialized as JSON and authenticated with HMAC-SHA256 using a per-install key stored at `~/.gaia/cache/hmac.key`. Unsigned or tampered caches are rejected and transparently rebuilt on the next query. Old `.pkl` caches from previous GAIA versions are ignored and re-indexed the next time you query a document. **You should upgrade if you** share `~/.gaia/` across machines (Dropbox, iCloud, network home directories), run GAIA in a multi-user environment, or have ever imported RAG caches from another source. --- ## Bug Fixes - **Ask Agent attaches files before sending to chat** (PR #725) — Dropped files are indexed into RAG and attached to the active session before the prompt is consumed, so the model sees the document on the first turn instead of the second. - **Document indexing failures are surfaced** (PR #723) — A document that produces 0 chunks now raises `RuntimeError` in the SDK and surfaces as `indexing_status: failed` in the UI, instead of looking like a silent success. Covers RAG SDK, background indexing, and re-index paths. - **Encrypted or corrupted PDFs produce actionable errors** (PR #784, closes #451) — RAG now raises distinct `EncryptedPDFError` and `CorruptedPDFError` exceptions instead of generic failures, so you see exactly what went wrong. - **RAG index thread safety hardened** (PR #746) — Adds `RLock` protection around index mutation paths and rebuilds chunk/index state atomically before publishing it, so concurrent queries read consistent snapshots and failed rebuilds no longer leak partial state. - **MCP JSON-RPC handler guards against non-dict bodies** (PR #803) — A malformed JSON-RPC payload (array, string, null) now returns HTTP 400 `Invalid Request: expected JSON object` instead of an HTTP 500 from a `TypeError`. - **File-search count aligned with accessible results** (PR #754) — The returned count now matches the number of files the tool actually surfaces, instead of a pre-filter total that over-reported results the caller could not access. - **Tracked block cursor replaces misplaced decorative cursor** (PR #727) — Fixes the mis-positioned blinking cursor in the chat input box, which now tracks the actual caret position via a mirror-div technique. - **Ad-hoc sign the macOS app bundle instead of skipping code signing** (PR #765) — The `.app` bundle inside the DMG now carries an ad-hoc signature, so Gatekeeper presents a single "Open Anyway" bypass in System Settings instead of the unrecoverable "is damaged" error. Full Apple Developer ID signing is still being finalized. --- ## Release & CI - **Publish workflow: single approval gate, no legacy Electron apps** (PR #758) — Removed the legacy jira and example standalone Electron apps from the publish pipeline; a single `publish` environment gate governs PyPI, npm, and installer publishing. - **Claude CI modernization** (PR #797, PR #799, PR #783) — Migrated all four `claude-code-action` call sites to `v1.0.99` (pinned by SHA, fixes an issue-handler hang), bumped `--max-turns` from 20 to 50 on both `pr-review` and `pr-comment` for deeper analysis, upgraded to Opus 4.7, standardized 23 subagent definitions with explicit when-to-use sections and tool allowlists, and added agent-builder tooling (manifest schema, `lint.py --agents`, BuilderAgent mixins). --- ## Docs - **Roadmap overhaul** (PR #710) — Milestone-aligned plans with voice-first as P0 and 9 new plan documents for upcoming initiatives. - **Plan: email triage agent** (PR #796) — Specification for an upcoming email triage agent. - **Docs/source drift resolved** (PR #794) — Fixed broken SDK examples across 15 docs, rewrote 5 spec files against the current source (including two that documented entire APIs that don't exist in code), added 20+ missing CLI flags to the CLI reference, and removed 2 already-shipped plan documents (installer, mcp-client). - **FAQ: data-privacy answer clarified for external LLM providers** (PR #798) — Sharper guidance on what leaves your machine when you point GAIA at Claude or OpenAI. --- ## Full Changelog **21 commits** since v0.17.2: - `6d3f3f71` — fix: replace misplaced decorative cursor with tracked terminal block cursor (#727) - `874cf2a3` — fix: Ask Agent indexes and attaches files before sending to chat (#725) - `4fa121e2` — fix: surface document indexing failures instead of silent 0-chunk success (#723) - `34b1d06e` — fix(ci): ad-hoc sign macOS DMG instead of skipping code signing (#765) - `7188b83c` — Roadmap overhaul: milestone-aligned plans with voice-first P0 and 9 new plan documents (#710) - `1beddac5` — cpp: support Ollama-compatible /v1 endpoints (#773) - `cf9ac995` — fix: harden rag index thread safety (#746) - `1c55c31b` — fix(ci): remove legacy electron apps from publish, single approval gate (#758) - `52946a7a` — feat(installer): bundle Lemonade Server MSI into Windows installer (#774) (#781) - `e96b3686` — ci(claude): review infra + conventions + subagent overhaul + agent-builder tooling (#783) - `058674b5` — fix(rag): detect encrypted and corrupted PDFs with actionable errors (#451) (#784) - `7bcb5d51` — fix: replace insecure pickle deserialization with JSON + HMAC in RAG cache (CWE-502) (#768) - `a5167e5f` — fix: keep file-search count aligned with accessible results (#754) - `da5ba458` — ci(claude): migrate to claude-code-action v1.0.99 + fix issue-handler hang (#797) - `03f546b9` — ci(claude): bump pr-review and pr-comment --max-turns 20 -> 50 (#799) - `4119d564` — docs(faq): clarify data-privacy answer re: external LLM providers (#798) - `0cfbcf41` — Add example agents and integration test workflow (#340) - `c4bd15fb` — docs: fix drift between docs and source (docs review pass 1 + 2) (#794) - `407ed5b8` — docs(plans): add email triage agent spec (#796) - `06fb04a4` — fix(mcp): guard JSON-RPC handler against non-dict body (#803) - `880ad603` — feat(installer): custom installer guide, agent export/import, first-launch seeder (#795) Full Changelog: [v0.17.2...v0.17.3](v0.17.2...v0.17.3) --- ## Release checklist - [x] `util/validate_release_notes.py docs/releases/v0.17.3.mdx --tag v0.17.3` passes - [x] `src/gaia/version.py` → `0.17.3` - [x] `src/gaia/apps/webui/package.json` → `0.17.3` - [x] Navbar label in `docs/docs.json` → `v0.17.3 · Lemonade 10.0.0` - [x] All 21 PRs in the range (v0.17.2..HEAD) are represented in the notes - [ ] Review from @kovtcharov-amd addressed
Summary
handleAskAgentin FileBrowser dispatched agaia:send-promptCustomEvent that nothing listened for (dead code), performing zero indexing or session attachmenthandleAskAgentto index files, attach them to the active session, and deliver the prompt via the existingpendingPromptZustand mechanismuseEffect([sessionId])in ChatView with reactiveuseEffect([pendingPrompt])+getState()StrictMode guard so prompts are consumed whether ChatView is already mounted or notChanges
FileBrowser.tsx: Extract sharedindexAndAttachFiles()helper, rewritehandleAskAgent, add try/catch/finally tohandleIndexSelected, surface partial-failure warningChatView.tsx: Reactive pendingPrompt useEffect with StrictMode guard and rAF cancellation on unmounttests/unit/chat/ui/test_ask_agent_workflow.py: 3 backend tests covering index→attach→chat pipeline (519 passing)Test plan
python -m pytest tests/unit/chat/ui/ --ignore=tests/unit/chat/ui/test_sse_confirmation.py— 519 passednpm run build— clean TypeScript compilepython util/lint.py --all— all checks passedCloses #583