Skip to content

feat(installer): custom installer guide, agent export/import, first-launch seeder#795

Merged
itomek merged 8 commits intomainfrom
776-docs-guide-for-building-a-custom-installer-with-a-custom-agent
Apr 20, 2026
Merged

feat(installer): custom installer guide, agent export/import, first-launch seeder#795
itomek merged 8 commits intomainfrom
776-docs-guide-for-building-a-custom-installer-with-a-custom-agent

Conversation

@itomek
Copy link
Copy Markdown
Collaborator

@itomek itomek commented Apr 17, 2026

Closes #776

Summary

  • Custom installer guide (docs/guides/custom-installer.mdx) — step-by-step walkthrough for OEMs/power-users building a branded GAIA installer: project scaffold, embedding a custom agent, electron-builder config, first-launch seeder, and cross-platform packaging.
  • Agent export/import (src/gaia/installer/export_import.py) — zip-bundle round-trip for custom agents under ~/.gaia/agents/, with zip-bomb defenses (per-file + aggregate streaming byte counters), symlink rejection, path-traversal guards, and atomic temp-file staging.
  • FastAPI endpoints (src/gaia/ui/routers/agents.py) — POST /api/agents/export and POST /api/agents/import behind three security guards: localhost-only, X-Gaia-UI header CSRF check, and tunnel-inactive check (503 when ngrok tunnel active).
  • First-launch agent seeder (src/gaia/apps/webui/services/agent-seeder.cjs) — copies <resourcesPath>/agents/<id>/ into ~/.gaia/agents/<id>/ on first launch using atomic rename + .seeded sentinel; symlink-safe, crash-safe, idempotent.
  • Settings UI (src/gaia/apps/webui/src/components/CustomAgentsSection.tsx) — Export All / Import buttons in the Settings modal; credentials warning before export; best-effort pre-read of bundle.json from zip to list agent IDs in the trust confirmation modal.
  • CLI (src/gaia/cli.py) — gaia agents export and gaia agents import commands.
  • Playbook (docs/playbooks/custom-installer/) — annotated reference files for the guide.

Security measures

  • Zip-bomb: per-file ≤ 50 MB + total ≤ 500 MB, both via streaming counters during extraction
  • Symlink entries rejected (checked via external_attr upper 16 bits)
  • Absolute paths and Windows drive letters rejected
  • Agent IDs validated against ^[a-z0-9]([a-z0-9-]{0,50}[a-z0-9])?$ plus reserved Windows device names
  • Path traversal double-checked on both pre-validation and during extraction
  • Bundle manifest capped at 1 MB before pre-read in CLI
  • Export/import endpoints require localhost + X-Gaia-UI header + tunnel inactive

Test plan

  • tests/unit/test_export_import.py — 14 tests covering round-trip, zip-slip, symlink, absolute path, oversized entry, too-many-entries, invalid agent IDs, zero-agent export, overwrite, atomicity, missing bundle.json, wrong format version
  • tests/unit/chat/ui/test_agents_router.py::TestExportImportSecurityGuards — 6 tests covering all three security guards on both endpoints
  • tests/electron/agent-seeder.test.cjs — 14 Jest tests covering seeder happy path, skip-if-seeded, user-owned skip, partial cleanup, symlink skip, missing source
  • All 1455 unit tests pass (40 pre-existing errors in unrelated test_uninstall_command.py)

@itomek itomek requested a review from kovtcharov-amd as a code owner April 17, 2026 18:31
@itomek itomek linked an issue Apr 17, 2026 that may be closed by this pull request
@github-actions github-actions bot added documentation Documentation changes devops DevOps/infrastructure changes cli CLI changes tests Test changes labels Apr 17, 2026
Comment thread src/gaia/ui/routers/agents.py Fixed
@itomek itomek self-assigned this Apr 17, 2026
@github-actions github-actions bot added rag RAG system changes performance Performance-critical changes labels Apr 17, 2026
Comment thread util/check_doc_citations.py Outdated
@kovtcharov
Copy link
Copy Markdown
Collaborator

@claude review this PR and identify why the tests are failing.

Comment thread .github/workflows/check_doc_links.yml
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 17, 2026

Claude finished @kovtcharov's task —— View job


Reviewing PR #795 — identifying test failures

  • Check CI status for failing workflows
  • Analyze failing test logs
  • Review code changes for correctness
  • Post findings

Job run · Branch

Comment thread docs/guides/custom-installer.mdx Outdated
itomek added a commit that referenced this pull request Apr 17, 2026
…citation-checker

Per @itomek review on PR #795:
- util/check_doc_citations.py: out of scope, removed (was in prior commit)
- .github/workflows/check_doc_links.yml: revert to origin/main
- docs/guides/custom-installer.mdx: removed in favor of the playbook at
  docs/playbooks/custom-installer/index.mdx (was in prior commit)

Updated internal links so the navigation and cross-references resolve:
- docs/docs.json: drop guide nav entry (playbook entry stays)
- docs/deployment/ui.mdx: Card href → /playbooks/custom-installer/index
- docs/guides/custom-agent.mdx: same Card href swap
- docs/playbooks/custom-installer/index.mdx: rewrite intro paragraph that
  self-linked to the now-removed guide
kovtcharov added a commit that referenced this pull request Apr 17, 2026
… hang

## What this fixes

1. **PR reviews haven't fired since #783 merged.** The workflow's `@beta` pin is
   stuck on a 2025-08-22 SHA that predates `pull_request_target` support (merged
   upstream 2025-09-22) and Opus 4.7 support (v1.0.98). The action's Prepare step
   has been rejecting `pull_request_target` with "Unsupported event type" on
   every run, and `continue-on-error: true` was hiding the failure as "success".

2. **`@claude` mentions post a TODO checklist and never update it with findings.**
   In v0 tag mode, large custom_instructions + low max_turns were exhausting the
   turn budget before Claude got to the final comment-update step. Visible on
   run 24581846289 for PR #795.

## Changes

- **Pin all 4 action call sites to `v1.0.99` by SHA** — unblocks
  `pull_request_target` and Opus 4.7. SHA-pin rather than tag-pin so a future
  floating-tag retarget can't silently repeat this class of breakage.

- **Migrate to the v1 API.** v1.0.99 drops the v0 inputs we use
  (`direct_prompt`, `custom_instructions`, `model`, `max_turns`). Merged
  `direct_prompt` + `custom_instructions` into a single `prompt` block per job;
  moved `model` and `max_turns` into `claude_args`. Migration guide:
  https://github.com/anthropics/claude-code-action/blob/main/docs/migration-guide.md

- **All 4 jobs now run in automation mode** (via `prompt` input) instead of tag
  mode. Two reasons:
  - Works around anthropics/claude-code-action#1223 (open bug): `--model` is
    silently ignored in tag mode, falling back to Sonnet 4.6. Automation mode
    honors `--model claude-opus-4-7` correctly.
  - Fixes the unchecked-TODO behavior: automation mode runs Claude to completion
    and posts one final comment, no progress tracker to forget to update.
  - Claude posts its reply via `gh pr comment` / `gh issue comment` / `gh api`
    from within the automation task.

- **`--max-turns` bumped for `issue-handler` from 30 to 50** — the TODO-unfilled
  behavior was often turn-budget exhaustion against the expanded
  custom_instructions added in #783.

- **`continue-on-error: true` removed from all 3 Claude action steps.** This
  masking is the same "No Silent Fallbacks" pattern #783 explicitly added to
  `CLAUDE.md` as prohibited — the workflow was the biggest violator.

- **Prompt-injection hardening on `pr-comment` and `issue-handler`** — instead
  of interpolating `github.event.comment.body` into the `prompt` (a classic
  Actions injection sink when user content lands inside another shell/markdown
  context), the prompt tells Claude to fetch the comment body itself via `gh
  api`. Workflow-context values in the prompt are limited to numeric IDs and
  repo names.
@itomek
Copy link
Copy Markdown
Collaborator Author

itomek commented Apr 17, 2026

Integration test findings (Windows NSIS installer, commit 38a2ceb)

Ran the PR's CI-built installer end-to-end on Windows (Ryzen AI MAX+ 395). Good news: the React/Electron side is wired correctly. Bad news: Export All is broken in this build, and several supporting pieces need attention. Flagging here so Claude Code / a follow-up PR can pick them up.

1. BLOCKER — Python side of the PR never ships in the installer

build/scripts/electron-install (or the equivalent uv install step) pins amd-gaia[ui]==0.17.2 from PyPI at install time. That version on PyPI predates this PR, so none of the new Python code is present in the installed venv.

Verified directly on the installed backend:

POST /api/agents/export → 405 Method Not Allowed
POST /api/agents/import → 405 Method Not Allowed
GET  /openapi.json      → only /api/agents and /api/agents/{agent_id}

And on the CLI:

$ gaia --help
positional arguments:
  {prompt,chat,talk,summarize,blender,sd,jira,docker,api,download,stats,test,
   youtube,kill,llm,groundtruth,create-template,eval,report,visualize,perf-vis,
   generate,batch-experiment,mcp,cache,init,install,uninstall}

No agent subcommand. Running it anyway:

$ gaia agent export bob-the-zookeeper
gaia: error: argument action: invalid choice: 'agent' (choose from prompt, chat,
talk, summarize, ..., uninstall)
$ echo $?
2

Direct import confirms:

$ python -c "import gaia.installer.export_import"
ModuleNotFoundError: No module named 'gaia.installer.export_import'
$ python -c "import importlib.metadata as m; print(m.version('amd-gaia'))"
0.17.2

Installed amd-gaia==0.17.2 from PyPI does not contain the PR's Python tree — no CLI dispatcher, no export_import.py, no FastAPI route handlers.

Fix options, in order of preference:

  1. Bundle the wheel with the installer (dist/amd_gaia-*.whlextraResources, then uv pip install from the local path). Best for reproducibility and offline installs.
  2. Publish a new PyPI release (e.g. 0.17.3) and bump the installer pin.
  3. uv pip install "git+https://github.com/amd/gaia@<sha>#egg=amd-gaia[ui]" — couples runtime install to GitHub, slow, not ideal.

2. HIGH — Export All fails silently in the UI when backend returns non-2xx

With a custom agent present in ~/.gaia/agents/, clicking Settings → Custom Agents → Export All → OK on the confirmation dialog produces: no native save dialog, no toast, no file written. The request hits the 405 from issue #1 and the UI shows the user nothing.

The React bundle at resources/dist/assets/index-Dhf8N9tu.js does have an error path:

const g = await fetch(`${Q0}/agents/export`, {
  method: "POST",
  headers: { "X-Gaia-UI": "1" }
});
if (!g.ok) {
  const T = await g.text().catch(() => "");
  let b = T;
  try { b = JSON.parse(T); } catch {}
  // ...supposed to surface an error
}

…but whatever it dispatches on !g.ok is not visible to the user. Either the toast portal is clipped by the closing confirmation dialog, the toast duration is too short, or the error dispatch path is broken.

This is a separate bug from #1 — any failure mode (405, 500, network timeout) will look identical to the user after #1 is fixed. Please:

  • Log the failing response to the renderer console unconditionally.
  • Render error toasts outside the confirmation dialog's portal with a 3–5s duration and a "See details" affordance.
  • Add a smoke test: mock POST /agents/export → 500, assert visible error UI.

UI wiring itself is correct — bundle targets exactly POST /agents/export and POST /agents/import with X-Gaia-UI: 1 CSRF header. No front-end rewiring needed once #1 lands.

3. HIGH — Silent install (/S) crashes in NSIS System.dll

Exception 0xC0000005 (access violation) in System.dll @ +0x1581

Reproduced twice with gaia-agent-ui-0.17.2-x64-setup.exe /S. GUI install is fine. Blocks any MDM / CI / unattended deployment. Likely an unchecked System::Call or nsExec::ExecToStack that only materializes a string argument in the UI-mode branch.

4. HIGH — gaia init Lemonade MSI race; failure is swallowed

~/.gaia/electron-install.log shows:

gaia init --profile minimal --yes
❌ Installation failed: Another MSI installation is in progress
Warning: gaia init exited with code 1. Continuing anyway.
state: ready

The installer reports state: ready even though Lemonade wasn't actually installed. First launch then shows the "Run gaia init --profile chat" banner and the user has to remediate manually.

Fixes: serialize on the MSI mutex before invoking gaia init, and/or retry on MSI 1618 with backoff. Do not swallow the failure silently — at minimum surface a dialog saying setup is incomplete.

5. MEDIUM — First-launch seeder is a no-op

~/.gaia/logs/seeder.log:

[INFO] No bundled agents directory at
  C:\Users\<user>\AppData\Local\Programs\gaia-desktop\resources\agents
  — nothing to seed

The seeder cjs is packaged inside app.asar and runs correctly; the resources/agents/ directory just isn't shipped. Either electron-builder.yml is missing an extraResources entry for build/bundled-agents/**, or the source directory is empty in CI.

Fixes:

  • Check in a minimal example bundle under build/bundled-agents/ (demonstrates the feature; the docs guide in this PR needs something to point at).
  • Add the extraResources entry.
  • CI assert: test -d build/bundled-agents && test -n "$(ls build/bundled-agents)".
  • When NODE_ENV=production and the directory is absent, log at WARN, not INFO.

6. LOW (pre-emptive) — Route ordering in gaia/ui/routers/agents.py

The existing /api/agents/{agent_id:path} wildcard will shadow the new /export and /import literals if they're registered after it. Please confirm in the router:

  1. /export and /import literal routes are registered before {agent_id:path}.
  2. Agent-ID validator rejects the reserved words export and import (or constrains to [a-z0-9_-]+ in a way that excludes them).
  3. Test coverage includes GET /api/agents/{id} with id ∈ {"export","import"} to catch a future reorder regression.

What did work

  • UI copy & conditional rendering (Export All only appears when a custom agent exists — behaving as designed).
  • Confirmation dialog's security warning about API keys/tokens in agent.py being included in the bundle is a thoughtful touch.
  • Native file picker for Import is correctly wired via showOpenDialog IPC.
  • React bundle contains all expected endpoints, CSRF header, and the requires_restart flag handling.

Full report with reproduction commands and evidence available on request. Happy to re-test once any of these land.

github-merge-queue bot pushed a commit that referenced this pull request Apr 17, 2026
… hang (#797)

## Why

Two real, verified-from-logs problems with the current Claude Code
setup:

**1. PR reviews haven't fired since #783 merged.** The workflow's
`@beta` pin points to a 2025-08-22 SHA that predates
`pull_request_target` support (merged in
[anthropics/claude-code-action#579](anthropics/claude-code-action#579)
on 2025-09-22) and Opus 4.7 support (fixed in v1.0.98). The action's
Prepare step rejects `pull_request_target` with `Unsupported event
type`, but `continue-on-error: true` was hiding the failure as a
"success" conclusion. Run
[24580730832](https://github.com/amd/gaia/actions/runs/24580730832) on
PR #795 is the concrete example.

**2. `@claude` mentions post a TODO checklist and never update it with
findings.** In v0 tag mode, large `custom_instructions` + `max_turns:
30` exhaust the turn budget before Claude reaches the final
comment-update step. Visible on run
[24581846289](https://github.com/amd/gaia/actions/runs/24581846289) for
PR #795.

## What changed

- **Pinned all 4 action call sites to v1.0.99 by SHA**
(`c3d45e8e941e1b2ad7b278c57482d9c5bf1f35b3`).
- **Full migration to the v1 API.** v1.0.99 drops the v0 inputs we use —
merged `direct_prompt` + `custom_instructions` into a single `prompt`
per job; moved `model` / `max_turns` into `claude_args`.
- **All 4 jobs now run in automation mode** (`prompt` input), not tag
mode, to work around
[anthropics/claude-code-action#1223](anthropics/claude-code-action#1223)
(tag-mode `--model` silently ignored) and to fix the unchecked-TODO
behavior.
- **`--max-turns` bumped for `issue-handler`** from 30 to 50.
- **`continue-on-error: true` removed** from the 3 Claude action steps.
- **Prompt-injection hardening** on `pr-comment` / `issue-handler` —
comment bodies fetched via `gh api` at runtime instead of interpolated
from `github.event.comment.body`.

## Validation (completed on this PR via temporary `pull_request`
trigger)

Tested end-to-end on PR #797 itself by temporarily adding a
`pull_request` trigger (since `pull_request_target` uses `main`'s
workflow, not the PR-head's). That trigger is now reverted — final diff
is migration-only.

| Path | Event | Run | Result |
|------|-------|-----|--------|
| `pr-review` | `pull_request` (synchronize) |
[24583825151](https://github.com/amd/gaia/actions/runs/24583825151) | ✅
Claude posted a full structured review as [this
comment](#797 (comment)) —
Summary / Issues / Strengths / Verdict format, referenced by
file.py:line |
| `issue-handler` | `issue_comment` (@claude mention) |
[24583975636](https://github.com/amd/gaia/actions/runs/24583975636) | ✅
Claude replied with actual findings to an @claude question, not an
unchecked TODO list |

**Claude's own review of this PR caught a real bug** in the
`issue-handler` prompt — on `issues.opened` events
`github.event.comment.id` is empty, so the prompt's `gh api
.../issues/comments/` URL would 404. Fixed in
[ad99674](ad99674) by
adding an explicit `COMMENT ID` field and instructing Claude to skip the
comment fetch when empty. Dogfooding worked.

Still unvalidated end-to-end (structural validation only):
- `pr-comment` (`pull_request_review_comment`) — uses the same
automation-mode pattern as the two validated paths
- `release-notes` (`workflow_run`) — only fires on `Publish Release`
completion; will self-validate on the next release

## Commits

4 commits in the branch. Net diff = migration + issue-handler fix.
**Recommend squash-merge** to collapse to one clean commit.

1. `ae1fb3f` — the v1 migration (main change)
2. `18efdc0` — TEST ONLY: add `pull_request` trigger (for validation)
3. `ad99674` — fix from Claude's own review: gate comment fetch on
COMMENT ID
4. `c7cbc59` — revert of (2)

## Vetting v1.0.99 against open upstream bugs

v1 is **not** bug-free. Assessment per issue:

| Issue | Severity | Affects GAIA? |
|-------|----------|---------------|
| [#1205](anthropics/claude-code-action#1205)
+ [#1187](anthropics/claude-code-action#1187)
/ [#1220](anthropics/claude-code-action#1220)
Bun tsconfig / cpSync symlink crash | P1 | **No** — GAIA has no
symlinked sensitive files (verified `CLAUDE.md`, `.claude/`) |
| [#1206](anthropics/claude-code-action#1206)
HTTP 401 app-token exchange | P2 | **No** — we use `GITHUB_TOKEN`, not a
GitHub App installation |
| [#1210](anthropics/claude-code-action#1210)
Git credentials overwritten | P2 | Low risk — we only read |
| [#1218](anthropics/claude-code-action#1218)
Fork PR fetch-by-name | P2 | Fixed in v1.0.96 |
| [#1222](anthropics/claude-code-action#1222)
`gh` CLI unavailable on default-branch PRs | P2 | **Maybe** — GAIA PRs
target `main`. Mitigated by our pre-generated `pr-diff.txt` /
`pr-files.txt` |
| [#1223](anthropics/claude-code-action#1223)
`--model` ignored in tag mode | — | **Would affect us** — worked around
by running all jobs in automation mode |
| [#1225](anthropics/claude-code-action#1225)
Opus 4.7 broken on v1.0.97 | P2 | Fixed in v1.0.98 |
| [#1226](anthropics/claude-code-action#1226)
`execution_file` not written on max_turns | P2 | Minor — doesn't affect
user-visible behavior |

## Rollback

Single workflow file, isolated change. `git revert` restores `@beta` —
which restores the silent-failure state, not a known-good state. If a
real merge-time regression appears, fix forward.
@kovtcharov
Copy link
Copy Markdown
Collaborator

@claude can you review this PR?

@github-actions
Copy link
Copy Markdown
Contributor

PR review — #795

CI status: all green. All 50+ checks on the latest commit (8f16d8c) are SUCCESS — there are no failing tests in CI. The earlier "why are tests failing?" was triggered when the pipeline was still running.

The real signal for reviewers in this thread is @itomek's integration report from running the actual NSIS installer end-to-end — CI passing does not mean the shipped installer works for end users.

What this PR does well 🟢

  • Zip-bomb defense is correct: per-file cap and aggregate streaming byte counter during extraction (export_import.py:314-346) — a many-small-files bomb can't slip through.
  • Three-guard endpoint defense (routers/agents.py:114-159): localhost + X-Gaia-UI CSRF header + tunnel-inactive. All three are independently tested.
  • Atomic write protocols: export uses tempfile → os.replace on the same filesystem; import stages each agent in a tempdir then atomically moves with backup-and-restore on failure; seeder writes sentinel last so a partial copy never looks complete (agent-seeder.cjs:176-191).
  • CodeQL py/stack-trace-exposure fix (export_import.py:373-386) is done at the source — os.replace exception text logged server-side, only a generic message flows into HTTP response. Clean fix.
  • Test coverage is broad: 14 export/import pytests + 6 router security-guard tests + 14 Jest seeder tests.

Issues worth addressing

🟡 Integration findings from @itomek are still open

@itomek's report found 6 issues on the actual installer. Most are real blockers regardless of green CI:

# Severity Issue Status
1 🔴 BLOCKER Installer pins amd-gaia[ui]==0.17.2 from PyPI — predates this PR, so none of the new Python code ships Unaddressed
2 🟡 HIGH Export All fails silently in UI on non-2xx responses Unaddressed
3 🟡 HIGH NSIS silent install (/S) crashes in System.dll Out of scope for this PR but blocks the playbook's unattended-deploy story
4 🟡 HIGH gaia init swallows MSI 1618 (race); reports state: ready on failure Out of scope but touches the installer flow this PR showcases
5 🟢 MEDIUM Seeder is a no-op — build/bundled-agents/ is empty in the built artifact electron-builder.yml entry exists but no source bundle exists in-repo → playbook's "install → see Zoo Agent" flow won't work out of the box
6 🟢 LOW Route-ordering concern (see below) Mostly fine, one small gap

#1 is the big one. Every unit test in this PR runs against source, so none of them catch that the shipped binary doesn't include the new code. Consider a CI step that posts POST /api/agents/export against the installed backend (not a TestClient in source tree) and asserts it isn't 405.

#5 is a gap between the docs and the shipped artifact. The playbook at docs/playbooks/custom-installer/index.mdx promises a preloaded Zoo Agent experience, but without a checked-in example under build/bundled-agents/, or a CI assertion that the directory is populated, the playbook works only for readers who first follow every step to create the staging dir.

🟡 PR description has drifted from code

  • Body says gaia agents export / gaia agents import (plural). Actual CLI registered at cli.py:2528 is gaia agent (singular). Docs also use singular. Please update the PR body to match.
  • Body still advertises "Custom installer guide" but commit 8f16d8c reverted that file — only the playbook remains. Either rename the PR to feat(installer): agent export/import + first-launch seeder, or note in the body that the guide moved to the playbook.

🟢 Reserved agent IDs (itomek's #6, partial)

_AGENT_ID_RE in export_import.py:48 accepts export and import as valid IDs. Combined with the GET /api/agents/{agent_id:path} wildcard, GET /api/agents/export resolves to a 404 rather than a 405 — a user-created agent named export would silently shadow the endpoint's URL space in GET requests. Suggest adding export/import to the reserved-name set and a small router test that asserts they're rejected.

🟢 Silent except Exception in hot-register

routers/agents.py:215 catches Exception during live_registry.register_from_dir(...). It does log at warning, so the error isn't fully swallowed, but per CLAUDE.md "fail loudly" it should catch a narrower exception type — or if broad catch is intentional here (best-effort per-agent, don't take down the whole import), add a short comment to that effect.

Suggested next steps

  1. Update PR description (agent not agents; drop the "Custom installer guide" claim).
  2. Decide how to handle @itomek's Update installer and workflows/actions for CI/CD #1 — at minimum, document that the PyPI pin needs a bump in the same cycle as this merge, or switch the installer to install from a locally-built wheel.
  3. Ship a minimal example under build/bundled-agents/ so the playbook's Zoo Agent flow actually works on a fresh install, plus a CI assertion that the dir is non-empty in the installer job.
  4. Small, low-risk: add export/import to reserved agent IDs with a matching test.

Security + atomicity work is solid. The gaps are mostly delivery/integration, not the core new code.

cc @kovtcharov-amd @itomek

@itomek
Copy link
Copy Markdown
Collaborator Author

itomek commented Apr 17, 2026

@itomek — thank you for the thorough Windows NSIS walkthrough. All six findings were read. Here's the disposition:

Addressed in commit 853f6a0:

  • Use public Lemonade hybrid installer #2 (Export UI silent failure)CustomAgentsSection.tsx now adds an unconditional console.error in both export and import catches, and the .danger-warning banner auto-scrolls into view when status.kind === 'error'.
  • Update Documentation #5 (Bundled agents dir empty)zoo-agent is now committed into build/bundled-agents/ (gitignore unblocked) so any fresh clone that runs npm run package:* will have a real demo agent seeded on first launch. The seeder also logs WARN (not INFO) when the dir is missing in a packaged build.
  • Prevent Users from Installing Hybrid mode on Unsupported Systems #6 (Reserved-word route coverage)TestRouteShadowing added to test_agents_router.py; assertions verify /export and /import resolve before the {agent_id:path} wildcard.

Out of scope for this PR:

@github-actions
Copy link
Copy Markdown
Contributor

Summary

Well-executed security-first feature that delivers three tightly-integrated pieces: a gaia agent export/import flow, FastAPI endpoints with three layered guards (localhost + X-Gaia-UI + tunnel-inactive), and a crash-safe Electron first-launch seeder. Zip handling is defence-in-depth (streaming byte caps, per-entry symlink/absolute/traversal checks re-validated during extract, atomic temp-file staging with rollback). Test coverage is thorough — 14 Python bundle tests, 6 router guard tests, 2 route-shadowing tests, and 10+ Jest seeder tests — and the playbook MDX doubles as product documentation and architecture reference. The single most important thing to address before merge: the exporter walks rglob("*") without filtering __pycache__/ or hidden dev artefacts, which will bloat bundles and defeats the "review before sharing" warning for anyone whose agent has been run locally.

Issues Found

🟡 Important

1. Export includes __pycache__/, .pyc, and any other dev artefacts in agent dirs (src/gaia/installer/export_import.py:173)

agent_dir.rglob("*") sweeps everything. A user who ran their agent once will ship compiled bytecode; a user with a .env at ~/.gaia/agents/my-agent/.env ships their API keys. The .gitignore block in the repo blocks __pycache__ for the staged zoo-agent, but end-user ~/.gaia/agents/ has no such protection. This also undercuts the stated 500 MB cap — a heavy .venv/ accidentally left in an agent dir will trip the import limit for the recipient.

Minimal filter aligned with what the seeder would plausibly copy:

    # Skip common dev artefacts so bundles stay small and don't leak
    # bytecode / editor state / dotfiles the user didn't intend to ship.
    _EXPORT_SKIP_DIRS = {"__pycache__", ".git", ".venv", "venv", "node_modules", ".pytest_cache", ".mypy_cache"}
    _EXPORT_SKIP_SUFFIXES = {".pyc", ".pyo"}

    try:
        with zipfile.ZipFile(tmp_path, "w", compression=zipfile.ZIP_DEFLATED) as zf:
            zf.writestr(BUNDLE_JSON_NAME, json.dumps(manifest, indent=2))
            for agent_dir in agent_dirs:
                for file_path in sorted(agent_dir.rglob("*")):
                    if not file_path.is_file():
                        continue
                    rel = file_path.relative_to(agent_dir)
                    if any(part in _EXPORT_SKIP_DIRS for part in rel.parts):
                        continue
                    if file_path.suffix in _EXPORT_SKIP_SUFFIXES:
                        continue
                    arcname = f"{agent_dir.name}/{rel.as_posix()}"
                    zf.write(file_path, arcname=arcname)
        os.replace(tmp_path, output_path)

(Hoist the constants to module level — shown inline here for a one-shot diff.)

2. Doc/registry drift: YAML-manifest agents still work, but docs/guides/custom-agent.mdx now says only Python modules are supported (docs/guides/custom-agent.mdx:13, docs/guides/custom-agent.mdx:54)

src/gaia/agents/registry.py:211 still loads agent.yaml, _is_custom_agent_dir at src/gaia/installer/export_import.py:63 still treats YAML-only dirs as valid custom agents, and the YAML-registry tests in tests/ presumably still pass — but the guide was rewritten as if YAML is gone. Either:

  • (a) Keep YAML working and restore a short "YAML manifest (legacy/declarative)" section so users with existing YAML agents don't think they're broken, or
  • (b) Actually deprecate YAML: tighten _is_custom_agent_dir to agent.py only, add a one-release warning log when YAML is discovered, and note the removal plan in docs/guides/custom-agent.mdx.

The current state — YAML loads silently, docs pretend it doesn't exist, export bundles it anyway — is the worst of both worlds. Discussion-only; no concrete diff since this is a product decision for @kovtcharov-amd.

3. PR description references paths that don't exist in the diff

  • "Custom installer guide (docs/guides/custom-installer.mdx)" — the guide is actually at docs/playbooks/custom-installer/index.mdx.
  • "gaia agents export and gaia agents import" — the CLI subcommand is singular (gaia agent, see src/gaia/cli.py:2526, docs/reference/cli.mdx:353).

Not a code issue, but worth fixing the PR body before merge so the release-notes generator doesn't pick up wrong paths.

🟢 Minor

4. Redundant double-logging on every failed export/import branch (src/gaia/apps/webui/src/components/CustomAgentsSection.tsx:1261-1264, :1350-1354)

Both console.error(...) and log.api.error(...) fire for the same error. log.api.error already writes to console in addition to the log buffer. Pick one — log.api.error is the project convention (see how refreshAgents at line 1219 uses only log.api.warn).

        } catch (err) {
            const message = err instanceof Error ? err.message : String(err);
            log.api.error('Agent export failed', err);
            flashStatus({ kind: 'error', message: `Export failed: ${message}` });
        }

(And the parallel change at line 1350 for import.)

5. Two sequential window.confirm() dialogs before the import actually happens (src/gaia/apps/webui/src/components/CustomAgentsSection.tsx:1296, and the existing pattern at :1231)

Each window.confirm is a modal browser dialog that blocks the renderer. The Settings modal already uses a proper styled modal idiom elsewhere — consider replacing with an in-app confirm component for consistency. Not blocking, but worth a follow-up issue; native confirm() dialogs look jarring against the rest of the GAIA UI.

6. handle_agent_import CLI error path collapses specific errors into a generic "Error:" (src/gaia/cli.py:5734)

The broad except Exception after the tuple-caught (BadZipFile, KeyError, JSONDecodeError, UnicodeDecodeError) catches ValueError (e.g. the 1 MB manifest cap) and emits Error: {exc} with no context. Minor, since the message itself is user-facing — but print(f"Error: invalid bundle: {exc}", ...) (matching the tuple branch) would read more consistently. Consider adding ValueError to the specific-tuple catch.

7. Hardcoded backend port in API_BASE (src/gaia/apps/webui/src/components/CustomAgentsSection.tsx:1177)

const API_BASE = window.location.protocol === 'file:'
    ? 'http://localhost:4200/api'
    : '/api';

This duplicates the same constant from services/api.ts — grep for localhost:4200 shows this is a convention, so not blocking, but a BACKEND_BASE_URL export from services/api.ts would eliminate one more place to update if the port ever moves.

8. readBundleAgentIds reimplements a zip central-directory parser in ~80 lines (src/gaia/apps/webui/src/components/CustomAgentsSection.tsx:1454-1531)

It works and is annotated well, but browser bundles usually pull in fflate (~8 KB gzipped) or jszip for this. Since this is a best-effort pre-read that already falls back to "unable to read" on any parse error, the cost/risk tradeoff is a fair call — flagging for awareness, not as a blocker.

Strengths

  • Security layering is textbook defence-in-depth. Pre-validation of zip entries (size/count/symlink/absolute/traversal) PLUS streaming byte counters during extraction that re-check limits per chunk, PLUS a defensive is_relative_to(staging_root) during extract that doubles up on the pre-check. Agent-ID regex mirrors the existing AgentManifest.validate_id so there's no semantic drift. Three endpoint guards (_require_localhost + _require_ui_header + _require_tunnel_inactive) are each independently necessary and composed cleanly via Depends([…]). Tunnel-inactive guard specifically blocks the data-exfil path that ngrok would otherwise open — good thinking.
  • Atomicity across three write paths is consistent. Export uses tempfile.mkstemp + os.replace for crash-safety. Import stages every agent under a temp dir under agents_root (same filesystem → rename is atomic), moves each dir with rollback-via-backup on failure. Seeder uses <id>.partial/renameSync(<id>) → write .seeded sentinel last. Same pattern, three layers — easy to reason about.
  • Route-shadowing test (tests/unit/chat/ui/test_agents_router.py::TestRouteShadowing) is exactly the right test to have. Catches the footgun where GET /{agent_id:path} could eat the named /export / /import routes — discriminates 403/404 vs 405 to prove resolution order without needing a happy-path fixture.
  • Playbook is unusually useful — it's a product doc, an architecture doc, AND a troubleshooting runbook, with per-OS tabs, .seeded sentinel recovery steps, and a clear "when to use Path A vs Path B" table at the end.

Verdict

Approve with suggestions.

Issue 🟡1 (export filter) and 🟡2 (YAML docs drift) should be addressed — the first in this PR, the second either here or in an immediate follow-up. Everything else is minor polish. The core design is sound, the tests are comprehensive, and the security posture is rightly paranoid.

kovtcharov added a commit that referenced this pull request Apr 17, 2026
PR #795 (large installer + agent export-import feature, many changed
files) exceeded the 20-turn budget on pr-review before Claude could
post its review comment. Run 24586335693 failed with
`error_max_turns`, visible now that #797 removed the continue-on-error
mask.

issue-handler was already at 50 from #797. Matching pr-review and
pr-comment for consistency — same failure mode, same fix. release-notes
stays at 30 since release diffs are bounded to a tag range.
github-merge-queue bot pushed a commit that referenced this pull request Apr 17, 2026
## Summary

PR #795 ([run
24586335693](https://github.com/amd/gaia/actions/runs/24586335693))
exceeded pr-review's `--max-turns 20` budget and failed with
`error_max_turns` — no review comment posted. That failure is visible
(not silently swallowed) thanks to #797 removing the `continue-on-error`
mask. The fix is the same bump I applied to `issue-handler` during the
v1 migration: 20 → 50.

Matching `pr-comment` at the same time for consistency — same failure
mode would apply on a large-diff PR conversation. `release-notes` stays
at 30 since release diffs are bounded to a tag-to-tag range.

## Test plan

- [ ] After merge, re-run `pr-review` on PR #795 (close+reopen or push
an empty commit) — confirm Claude completes the review within 50 turns
and posts the comment
- [ ] Spot-check next 2-3 post-merge PRs don't regress to failures
itomek added 2 commits April 17, 2026 18:00
…aunch seeder

Adds end-to-end support for shipping a GAIA installer with a custom agent
pre-loaded, and for transferring agents between machines via zip bundles.

Changes:
- `src/gaia/installer/export_import.py` — new module: zip-based custom agent
  export/import with zip-bomb defences (entry count, per-file and total size
  limits), path-traversal and symlink rejection, atomic write/overwrite
- `src/gaia/apps/webui/services/agent-seeder.cjs` — first-launch bundled-agent
  seeder: copies `<resources>/agents/<id>` into `~/.gaia/agents/<id>` with an
  atomic partial→rename→sentinel protocol; idempotent across re-launches
- `src/gaia/apps/webui/electron-builder.yml` — `extraResources` entry to bundle
  `build/bundled-agents/` into the installer as `<resources>/agents/`
- `src/gaia/apps/webui/main.cjs` — call `seedBundledAgents()` on app startup
  before the Python backend starts
- `src/gaia/cli.py` — `gaia agent export` and `gaia agent import` subcommands
  with interactive trust gate and `--yes` flag for non-TTY use
- `src/gaia/ui/routers/agents.py` — `POST /api/agents/export` and
  `POST /api/agents/import` endpoints with localhost-only, CSRF-header, and
  tunnel-inactive guards; hot-registers imported agents into the live registry
- `src/gaia/apps/webui/src/components/CustomAgentsSection.tsx` — Settings panel
  section for export/import with inline ZIP pre-read for trust modal
- `src/gaia/apps/webui/src/components/SettingsModal.tsx` — wires in
  CustomAgentsSection
- `docs/guides/custom-agent.mdx` — rewritten around Python agents; removed
  YAML-manifest section, replaced all examples with Python equivalents
- `docs/guides/custom-installer.mdx` — new high-level guide (when to build a
  custom installer and pointer to the playbook)
- `docs/playbooks/custom-installer/index.mdx` — new end-to-end playbook: Path A
  (branded installer with Zoo Agent, 3-OS tabs) and Path B (export/import flow)
- `docs/reference/cli.mdx` — documents `gaia agent export` and `gaia agent import`
- `docs/deployment/ui.mdx` — adds Custom Installer card to the CardGroup
- `docs/docs.json` — registers new guide and playbook pages in nav
- `util/check_doc_citations.py` — new CI utility: verifies that path:NNN citations
  in docs resolve to the expected symbol at that line
- `.github/workflows/check_doc_links.yml` — adds citation-checker step and
  symbol-drift path triggers
- `tests/unit/test_export_import.py` — 14 pytest cases (round-trip, zip-slip,
  symlink, absolute path, oversized, too many entries, invalid IDs, atomicity,
  overwrite, missing manifest, wrong version)
- `tests/electron/agent-seeder.test.cjs` — 12 Jest cases (seed, idempotency,
  sentinel semantics, user-data preservation, partial-copy recovery, all 3
  platform paths)

Verified on macOS (arm64 DMG), Ubuntu (AppImage), and Windows (NSIS exe).

Closes #776
…ndency

TestClient hardcodes scope["client"] = ("testclient", 50000); TunnelAuthMiddleware
saw a non-localhost host with an active tunnel and returned 401 before the
_require_tunnel_inactive FastAPI dependency could fire its 503.  Patching
_LOCAL_HOSTS to include "testclient" in those two tests lets the middleware
pass through so the dependency under test is actually exercised.
itomek added 6 commits April 17, 2026 18:00
…review)

- export_import.py: aggregate streaming byte counter catches multi-entry
  zip-bombs; replaced iter-lambda with explicit while loop (fixes W0640)
- routers/agents.py: log silenced OSError on temp-file cleanup; add
  requires_restart flag to import response; move os import to module top
- cli.py: 1 MB hard cap on bundle.json before pre-read to prevent OOM
- CustomAgentsSection.tsx: 100 MB size guard before arrayBuffer(); surface
  requires_restart warning in the import success message
- agent-seeder.cjs: remove redundant existsSync before rmSync (force:true)
- test_export_import.py: drop unused import gaia.installer.export_import
- check_doc_citations.py: fix off-by-one anchor line numbers (125/272)
- test_electron_chat_app.js: update stale assertions that expected
  electron-forge (we use electron-builder) and <title>GAIA Agent UI</title>
  (now just <title>GAIA</title>); fix uploadDocumentByPath → uploadDocumentBlob
- routers/agents.py: convert errors from flat strings to {id, error} objects
  so the frontend can display them per-agent; resolves CodeQL information-
  exposure advisory (exception data no longer flows as a raw string)
- rag/sdk.py: remove redundant inner 'import json' at lines 963 and 1113
  (W0404 reimport; json is already imported at module top)
test_electron_chat_installer.js had stale assertions for electron-forge
(no longer used): @electron-forge/cli devDependency, forge makers config,
and scripts.make. Updated to check electron-builder.yml, electron-builder
devDependency, and platform-specific package scripts. Also fixed bin
path check to use the actual bin field value (gaia-ui.cjs, not .mjs).
CodeQL alert #251 (py/stack-trace-exposure, CWE-209): str(exc) from an
os.replace failure flowed from export_import.py into the /api/agents/import
HTTP response via ImportResult.errors, potentially exposing absolute file
paths and OS-level details to the caller.

Fix at the source: log the full exception server-side at WARNING and
append only a stable generic message to result.errors. The router's
existing {id, error} structuring continues to work unchanged; it now
splits a bounded message instead of raw OS-exception text.
…citation-checker

Per @itomek review on PR #795:
- util/check_doc_citations.py: out of scope, removed (was in prior commit)
- .github/workflows/check_doc_links.yml: revert to origin/main
- docs/guides/custom-installer.mdx: removed in favor of the playbook at
  docs/playbooks/custom-installer/index.mdx (was in prior commit)

Updated internal links so the navigation and cross-references resolve:
- docs/docs.json: drop guide nav entry (playbook entry stays)
- docs/deployment/ui.mdx: Card href → /playbooks/custom-installer/index
- docs/guides/custom-agent.mdx: same Card href swap
- docs/playbooks/custom-installer/index.mdx: rewrite intro paragraph that
  self-linked to the now-removed guide
…e tests

- Add zoo-agent to build/bundled-agents/ staging dir so the custom-installer
  playbook has a working example seeded on first launch; extend .gitignore
  to un-ignore this specific path while keeping the rest of build/ excluded
- Fix playbook: remove _TOOL_REGISTRY import and clear() call from the
  ZooAgent example (_register_tools must be pass, not clear(), to avoid
  wiping the process-wide tool registry)
- agent-seeder.cjs: log at WARN (not INFO) when bundled-agents dir is
  missing inside a packaged Electron app; dev mode stays at INFO
- CustomAgentsSection.tsx: add console.error in both export and import
  catches; scroll error banner into view when status.kind === 'error'
- tests: add Jest structure test for CustomAgentsSection error paths;
  add TestRouteShadowing to confirm /export and /import resolve before
  the {agent_id:path} wildcard
@itomek itomek force-pushed the 776-docs-guide-for-building-a-custom-installer-with-a-custom-agent branch from 853f6a0 to 47bf98a Compare April 17, 2026 22:00
@github-actions
Copy link
Copy Markdown
Contributor

Summary

Solid, security-conscious implementation of the agent export/import round-trip plus a first-launch bundled-agent seeder for installer builds. The zip hardening (per-file + aggregate streaming caps, symlink reject, path-traversal double-check, agent-id regex, atomic staging+backup), the three-guard HTTP surface (localhost + X-Gaia-UI + tunnel-inactive), and the crash-safe <id>.partial → rename protocol in the seeder are all well thought through and well tested.

Two things worth fixing before merge: (1) the bundled zoo-agent/agent.py diverges from the pattern every other agent uses — it will silently inherit global tool state from any agent loaded earlier; (2) the custom-agent guide now documents only Python agents, but AgentRegistry still loads YAML manifests and validates them through AgentManifest (Pydantic) — users on that code path now have no reference docs.


Issues Found

🟡 Important

1. Bundled Zoo Agent's _register_tools breaks the global-registry convention (src/gaia/apps/webui/build/bundled-agents/zoo-agent/agent.py:28)

_TOOL_REGISTRY in src/gaia/agents/base/tools.py:16 is module-global dict state. Every other agent that owns its tool surface (src/gaia/agents/builder/agent.py:157, the builder templates, and the YAML-manifest loader at src/gaia/agents/registry.py:393) clears the registry first so the agent doesn't inherit tools registered by a previously loaded agent in the same process. The bundled Zoo Agent uses pass, which means if another agent was imported first and registered tools (e.g. during registry discovery of multiple custom agents on startup), the Zoo Agent will be handed those tools when its instance spins up.

The guide example in docs/guides/custom-agent.mdx:253 correctly uses _TOOL_REGISTRY.clear(). The bundled code and the guide disagree with each other on this pattern, and the bundled version is the buggy one.

from gaia.agents.base.agent import Agent
from gaia.agents.base.console import AgentConsole
from gaia.agents.base.tools import _TOOL_REGISTRY


class ZooAgent(Agent):
    AGENT_ID = "zoo-agent"
    AGENT_NAME = "Zoo Agent"
    AGENT_DESCRIPTION = "A zookeeper who loves animals"
    CONVERSATION_STARTERS = [
        "Hello! What's happening at the zoo today?",
        "Tell me a fun fact about one of your animals.",
    ]

    def _get_system_prompt(self) -> str:
        return (
            "You are a funny and enthusiastic zookeeper! You work at the world's "
            "best zoo and every response you give includes a fun fact or a playful "
            "reference to one of your beloved zoo animals."
        )

    def _create_console(self) -> AgentConsole:
        return AgentConsole()

    def _register_tools(self) -> None:
        _TOOL_REGISTRY.clear()

Same fix also brings docs/playbooks/custom-installer/index.mdx:67 (the inline example) in line. Both copies of the Zoo Agent should match.

2. Guide removes YAML manifest docs while the feature is still live (docs/guides/custom-agent.mdx)

The rewrite removes the entire "Manual Creation: YAML Manifest" section (the full-manifest reference, tool table, Zoo YAML example, research-agent YAML example, validation-error accordion). But AgentRegistry._load_manifest_agent (src/gaia/agents/registry.py:311), AgentManifest (Pydantic schema), and KNOWN_TOOLS all still exist and are the documented preferred path in CLAUDE.md for agents that only need built-in mixins:

New agents are preferably registered via YAML manifests validated by Pydantic in src/gaia/agents/registry.py

Existing user agents in ~/.gaia/agents/<id>/agent.yaml will keep working, but a user who tries to author one now has no reference for the tools:, models:, mcp_servers:, conversation_starters: schema. Two acceptable paths:

  • Keep the YAML section as a shorter reference (tool names, minimal manifest, full example), labelled as "Quickest path for zero-code agents."
  • Or formally deprecate YAML: add a deprecation note in the guide, wire a DeprecationWarning into _load_manifest_agent, and leave the removal for a follow-up release so existing authored agents aren't orphaned.

Either is fine, but silently deleting the docs while the code path stays active is the worst of both worlds.

3. pr-files.txt and PR body disagree with the shipped CLI on the command name

The PR description calls out gaia agents export (plural) in the Summary; the actual CLI registered in src/gaia/cli.py:2527 is gaia agent export (singular), and that's what docs/reference/cli.mdx:346 and docs/playbooks/custom-installer/index.mdx:613 document. Code and docs agree — only the PR description is out of date. Update the description before merge so the changelog captures the right name.

🟢 Minor

4. copyDirRecursive has two different symlink policies depending on Node version (src/gaia/apps/webui/services/agent-seeder.cjs:129-152)

The fs.cpSync branch passes dereference: true, which follows symlinks and copies their targets. The hand-rolled fallback skips symlinks entirely (log("WARN", ...)). Both are defensible individually, but they produce different trees for the same input bundle. For a seeder whose stated invariant is "prevent out-of-tree references in ~/.gaia/agents/<id>/," skipping is the safer default everywhere. Since Electron 40 always has cpSync, you can either:

  • Replace dereference: true with an explicit pre-walk that rejects/skips symlinks before calling cpSync, or
  • Drop the fallback branch entirely (the code comment already says it shouldn't normally hit).

Not a blocker, but the inconsistency will surprise someone eventually.

5. Broad except Exception in the CLI import/export handlers (src/gaia/cli.py:5688, 5708, 5732, 5745)

Four except Exception as exc: # noqa: BLE001 blocks, each catching everything and printing f"Error: {exc}" before sys.exit(1). This is fail-loud enough (the user does see the error), but CLAUDE.md "No Silent Fallbacks" pushes for named exceptions so the traceback hierarchy stays meaningful. Every caller here throws either ValueError or zipfile.BadZipFile (already caught), plus OSError for filesystem issues. A single except (OSError, RuntimeError) after the ValueError branch would cover the realistic failure modes without the catch-all.

6. seeder.log grows unbounded (src/gaia/apps/webui/services/agent-seeder.cjs:99-107)

Every launch appends a line, even when seeding is a no-op ("already seeded — sentinel present"). Over the lifetime of an installed app that's one INFO line per agent per launch, forever. Consider either:

  • Truncating seeder.log on each run (it's diagnostic, not audit), or
  • Only logging the "seeded complete" summary at INFO and dropping the per-agent "skipping — already seeded" to DEBUG (dropped from the file).

7. .gitignore build/bundled-agents/ re-enable is fragile (.gitignore:8-14)

build/
# Allow committed demo agents in the installer staging dir
!src/gaia/apps/webui/build/
!src/gaia/apps/webui/build/bundled-agents/
!src/gaia/apps/webui/build/bundled-agents/**

Re-enabling a subpath of an ignored directory works, but src/gaia/apps/webui/build/ is also the npm build-output root (see electron-builder.yml's extraResources entries for dist/ under the same parent). A future npm run clean or a webpack config change that empties build/ will wipe the checked-in Zoo Agent. Consider moving bundled agents out of the build/ tree entirely — e.g., src/gaia/apps/webui/bundled-agents/ — and have electron-builder.yml extraResources point at that stable location. The seeder invariant (source = <resourcesPath>/agents) is unaffected.

8. _validate_zip_entries trusts header file_size (src/gaia/installer/export_import.py:1970)

Not a real hole — the extraction loop at src/gaia/installer/export_import.py:2089-2099 re-checks both bytes_written per file and total_written across the whole archive against the same caps, so a header-spoofed size gets caught. Worth a one-line comment above _validate_zip_entries noting it's a fast-reject pass and the extract loop is the authoritative enforcement. Future readers will otherwise wonder why the size check is duplicated.

9. CustomAgentsSection.tsx uses window.confirm for the trust gate (src/gaia/apps/webui/src/components/CustomAgentsSection.tsx:1232, 1296)

Native confirm() can't show an agent-ID list in a styled, scrollable region — if a bundle declares 20 agents the dialog truncates or overflows depending on OS. A proper modal built from the existing settings-modal primitives would read the full list, support per-agent opt-out, and match the rest of the UI. Not blocking for shipping, but flag it for the design backlog.


Strengths

  • Security posture on the Python side is excellent. Aggregate + per-file zip-bomb caps enforced twice (fast reject on headers, authoritative counter during extraction), path-traversal double-check with is_relative_to, symlink bit check via external_attr >> 16, reserved-Windows-name list, agent-id regex reused from AgentManifest.validate_id, atomic staging + backup+restore on per-agent move failure. The error text change at src/gaia/installer/export_import.py:2132 that keeps OS paths out of HTTP responses while still logging them server-side is exactly the right pattern for CodeQL py/stack-trace-exposure.
  • Three-guard HTTP surface is the right model. _require_localhost + _require_ui_header + _require_tunnel_inactive compose cleanly as FastAPI dependencies, each fails loud with a specific status (403/403/503), and TestExportImportSecurityGuards in tests/unit/chat/ui/test_agents_router.py proves every combination. TestRouteShadowing is a nice touch — catches the {agent_id:path} wildcard stealing /export if someone ever reorders the router.
  • Seeder crash-safety is carefully designed. <id>.partial sibling → renameSync.seeded sentinel last means a crash mid-copy leaves no state that future runs misread as either "user-owned" or "complete." The error-isolation loop (one failing agent doesn't block the others) plus the rollback-on-rename-failure branch in seedOneAgent covers the realistic failure modes. Tests at tests/electron/agent-seeder.test.cjs exercise every one of those branches on a real tmpdir sandbox.

Verdict

Request changes — Issue #1 (bundled Zoo Agent missing _TOOL_REGISTRY.clear()) is a small but real correctness bug that ships as the reference implementation users copy into their own bundles. Issue #2 (YAML guide deletion while the code path is live) leaves CLAUDE.md's stated preferred authoring path undocumented. Both are quick fixes. Everything else is minor and can follow in a separate PR.

Once #1 and #2 are addressed this is ready to merge.

@itomek
Copy link
Copy Markdown
Collaborator Author

itomek commented Apr 17, 2026

Integration test: Windows 11, AMD Ryzen AI MAX+ 395 (Strix Halo), Python 3.12.12

Tested against PR source installed as editable package into ~/.gaia/venv, frontend from the PR-built NSIS installer.

✅ First-launch agent seeder

  • agent-seeder.cjs ran on every GAIA launch
  • Correctly detected zoo-agent/ as user-owned (directory existed without .seeded sentinel), logged WARN, skipped — expected per design
  • zoo-agent bundled correctly at resources/agents/zoo-agent/agent.py in the installed app

✅ Export All (via UI)

  • Clicked Export All → trust dialog: "Exported bundle contains your agent source files as-is. Any API keys, tokens, or credentials in agent.py will be included in the bundle. Review before sharing. Continue?"
  • Clicked OK → native file-save dialog opened with gaia-agents-export.zip pre-filled in Downloads
  • Zip saved (3,720 bytes): bundle.json + bob-the-zookeeper/agent.py

✅ Import (via UI)

  • Clicked Import → file picker → selected zip → trust dialog: "Importing this bundle will run third-party Python code on your machine when the agent is selected. Only import bundles from sources you trust. Agents to install: bob-the-zookeeper"
  • Clicked OK → status: "Installed 1 agent(s): bob-the-zookeeper (replaced: bob-the-zookeeper) — restart required for replaced agents to take full effect"

✅ Error banner (backend down)

  • Killed backend → Export All → trust OK → inline error: "⚠ Export failed: Failed to fetch"

🐛 Bug: PermissionError in _is_custom_agent_dir on Python 3.12.12 / Windows

export_import.py lines 93-97:

def _is_custom_agent_dir(path: Path) -> bool:
    return path.is_dir() and (
        (path / "agent.py").is_file() or (path / "agent.yaml").is_file()
    )

On Python 3.12.12 (uv-managed), Path.is_file() raises PermissionError: [WinError 5] Access is denied instead of returning False when the path is access-denied. My ~/.gaia/agents/ contained a link/ subdirectory with restricted NTFS permissions, causing every Export attempt to return 500 Internal Server Error.

Fix applied locally before testing could proceed:

def _is_custom_agent_dir(path: Path) -> bool:
    """A directory qualifies as a custom agent if it holds agent.py or agent.yaml."""
    try:
        return path.is_dir() and (
            (path / "agent.py").is_file() or (path / "agent.yaml").is_file()
        )
    except OSError:
        return False

Path.is_file() behaviour on permission-denied paths changed in Python 3.12. Recommend merging this fix before shipping.

@itomek
Copy link
Copy Markdown
Collaborator Author

itomek commented Apr 20, 2026

Follow-up: seeder happy path verified (fresh ~/.gaia/agents/ directory)

To test the seeder's copy path (not just the skip path), I temporarily moved ~/.gaia/agents/ aside, created a fresh empty one, and relaunched GAIA.

Seeder log on relaunch:

[INFO] Seeded "zoo-agent" from ...resources/agents/zoo-agent to ~/.gaia/agents/zoo-agent
[INFO] Seeding complete — seeded=1 skipped=0 errors=0

Result:

  • ~/.gaia/agents/zoo-agent/ created with agent.py
  • .seeded sentinel written with seededAt timestamp and source path ✓
  • Zoo Agent appeared in the agent selector dropdown alongside Chat Agent ✓

Both seeder paths now confirmed:

  • Fresh install (no prior zoo-agent/): seeder copies, writes .seeded, agent visible in UI ✓
  • Existing directory without .seeded (user-owned data): seeder skips with WARN ✓

@itomek itomek added this pull request to the merge queue Apr 20, 2026
Merged via the queue into main with commit 880ad60 Apr 20, 2026
54 checks passed
@itomek itomek deleted the 776-docs-guide-for-building-a-custom-installer-with-a-custom-agent branch April 20, 2026 13:21
@itomek
Copy link
Copy Markdown
Collaborator Author

itomek commented Apr 20, 2026

Integration Test Results — PR #795 (Windows 11 / AMD Ryzen AI MAX+ 395)

Test date: 2026-04-20
Tester: @itomek (automated via Claude Cowork)
Platform: Windows 11, AMD Ryzen AI MAX+ 395 w/ Radeon 8060S, 102 GB RAM
Installer built with: npm run package:winelectron-builder --win --config electron-builder.yml
Installer: gaia-agent-ui-0.17.2-x64-setup.exe (297.4 MB installed)


✅ NSIS Installer Wizard (fresh per-user install)

Walked through all four wizard pages:

  1. License Agreement — MIT License, AMD copyright displayed correctly
  2. Installation Options — "Only for me (tomas)" pre-selected; shows "Fresh install for current user only" — no UAC elevation required ✅
  3. Choose Install LocationC:\Users\tomas\AppData\Local\Programs\gaia-desktop, 297.4 MB required ✅
  4. Completing GAIA Setup — splash screen with GAIA Agent UI branding and AMD RYZEN AI logo; "Run GAIA" checkbox ticked; Finish launches the app ✅

✅ First-Launch Seeder — All Four Paths Tested

Scenario seeder.log result
No resources/agents/ dir (old install) No bundled agents directory — nothing to seed
Agent present, no .seeded sentinel Skipping "zoo-agent" — target exists without .seeded sentinel (treating as user-owned data)
Agent absent (fresh user dir) Seeded "zoo-agent" from resources/agents/zoo-agent to ~/.gaia/agents/zoo-agent
Agent present + .seeded sentinel (reinstall) Skipping "zoo-agent" — already seeded (sentinel present)

The sentinel-based logic correctly distinguishes user-owned data from seeded data across all paths.


✅ Export/Import Round-Trip

Settings UI → Custom Agents section:

  • Lists bob-the-zookeeper (custom_python) and zoo-agent (custom_python) ✅

Export All:

  • Security warning shown: "Any API keys, tokens, or credentials in agent.py will be included. Review before sharing." ✅
  • Save-as dialog pre-fills gaia-agents-export.zip
  • bundle.json content verified: format_version: 1, gaia_version: 0.17.2, agent_ids: ["bob-the-zookeeper", "zoo-agent"]

Import:

  • File picker filtered to *.zip
  • Pre-import security warning: "Importing this bundle will run third-party Python code. Only import bundles from sources you trust." Lists agents to install ✅
  • Both agents restored to ~/.gaia/agents/ from zip ✅
  • Settings UI reflects imported agents immediately ✅

🐛 Bug Found & Fixed: PermissionError in _is_custom_agent_dir (Windows / Python 3.12.x)

File: src/gaia/installer/export_import.py, _is_custom_agent_dir()

On Windows with Python 3.12.12, Path.is_file() raises PermissionError: [WinError 5] Access is denied instead of returning False for NTFS paths with restricted ACLs. Any ~/.gaia/agents/ subdirectory with restricted permissions causes POST /api/agents/export to return HTTP 500.

Fix applied:

def _is_custom_agent_dir(path: Path) -> bool:
    try:
        return path.is_dir() and (
            (path / "agent.py").is_file() or (path / "agent.yaml").is_file()
        )
    except OSError:
        return False

See earlier comment for full details.


⚠️ Doc Gap: build-ui-installer.ps1 not updated for Phase C (electron-builder)

installer/scripts/build-ui-installer.ps1 still calls electron-forge package and electron-forge make (steps 4 and 5). Since Phase C (this PR) migrates to electron-builder, the script would fail if run as documented. The script should be updated to call npm run package:win instead, or the README note should be promoted to a clear action item.

The installer README does include a note flagging this, but the script itself still contains the old commands and will fail with electron-forge: command not found.


Overall: PASS (with one bug fix needed and one doc gap to address)

github-merge-queue bot pushed a commit that referenced this pull request Apr 20, 2026
# GAIA v0.17.3 Release Notes

GAIA v0.17.3 is an extensibility and resilience release. You can now
package your own agents into a custom GAIA installer and seed them on
first launch, point GAIA at alternative OpenAI-compatible inference
servers from the C++ library (Ollama, for example), and start from three
new reference agents (weather, RAG Q&A, HTML mockup) that execute
against real Lemonade hardware in CI. It also hardens the RAG cache
against an insecure-deserialization class of bug (CWE-502) — all users
should upgrade.

**Why upgrade:**
- **Ship your own GAIA** — Export and import agents between machines,
follow a new guide to produce a custom installer that seeds your agents
on first launch, and on Windows install everything in one step because
the installer now includes the Lemonade Server MSI.
- **Work with alternative inference backends** — The C++ library now
preserves OpenAI-compatible `/v1` base URLs instead of rewriting them to
`/api/v1`, so servers that expose the standard `/v1` path (Ollama, for
example) work out of the box.
- **Start from a working example** — Three new reference agents (weather
via MCP, RAG document Q&A, HTML landing-page generator) with integration
tests that actually execute against Lemonade on a Strix CI runner.
- **Safer RAG cache** — Replaces `pickle` deserialization with JSON +
HMAC-SHA256 (CWE-502). Unsigned or tampered caches are rejected and
transparently rebuilt on the next query.
- **Better document handling** — Encrypted or corrupted PDFs now produce
distinct, actionable errors (`EncryptedPDFError`, `CorruptedPDFError`)
instead of generic failures, and the RAG index is hardened for
concurrent queries.

---

## What's New

### Custom Installers and Agent Portability

You can now package a custom GAIA installer that ships with your own
agents pre-loaded, and move agents between machines with export/import
(PR #795). On Windows, the official installer now includes the Lemonade
Server MSI and runs it during install, so a fresh machine has the
complete local-LLM stack after a single download (PR #781).

**What you can do:**
- Export an agent from `~/.gaia/agents/` to a portable bundle with `gaia
agents export` and import it on another machine with `gaia agents
import`
- Follow the new custom-installer playbook at
[`docs/playbooks/custom-installer/index.mdx`](/playbooks/custom-installer)
to distribute GAIA with your agents pre-loaded — useful for workshops,
team deployments, and internal tooling
- On Windows, the installer now includes Lemonade Server — no separate
download for a complete first-run experience

**Under the hood:**
- `gaia agents export` / `gaia agents import` CLI commands round-trip
agents between machines as portable bundles
- First-launch agent seeder
(`src/gaia/apps/webui/services/agent-seeder.cjs`) copies
`<resourcesPath>/agents/<id>/` into `~/.gaia/agents/<id>/` the first
time the app starts
- Windows NSIS installer embeds `lemonade-server-minimal.msi` into
`$PLUGINSDIR` and runs it via `msiexec /i ... /qn /norestart` during
install (auto-cleaned on exit)

---

### Broader Backend Compatibility in the C++ Library

The C++ library now preserves OpenAI-compatible `/v1` base URLs (PR
#773) instead of rewriting them to `/api/v1`. That means inference
servers that expose the standard OpenAI `/v1` path — for example, Ollama
at `http://localhost:11434/v1` — work out of the box without needing a
special adapter.

---

### Reference Agents and Real-Hardware Integration Tests

Three new example agents and a Strix-runner CI workflow land together
(PR #340).

**What you can do:**
- Copy `examples/weather_agent.py`, `examples/rag_doc_agent.py`, or
`examples/product_mockup_agent.py` as a starting point for your own
agents
- Run the new integration tests locally against Lemonade to validate
agents end-to-end, not just structurally

**Under the hood:**
- `tests/integration/test_example_agents.py` executes agents and
validates responses with a 5-minute-per-test timeout
- `.github/workflows/test_examples.yml` runs on the self-hosted Strix
runner (`stx` label) with Lemonade serving `Qwen3-4B-Instruct-2507-GGUF`
- Docs homepage refreshed with a technical value prop ("Agent SDK for
AMD Ryzen AI") and MCP / CUA added to the capabilities list

---

### Smarter PDF Handling in RAG

Encrypted and corrupted PDFs now surface as distinct, actionable errors
(`EncryptedPDFError`, `CorruptedPDFError`, `EmptyPDFError`) instead of
generic failures or silent 0-chunk indexes (PR #784, closes #451).
Encrypted PDFs are detected before extraction; corrupted PDFs are caught
during extraction with a clear message. Combined with the
indexing-failure surfacing in PR #723, you get a visible indexing-failed
status the moment a document fails — and the RAG index itself is now
thread-safe under concurrent queries (PR #746).

---

## Security

### RAG Cache Deserialization Replaced with JSON + HMAC

Fixes an insecure-deserialization issue in the RAG cache (CWE-502, PR
#768). Previously, cached document indexes were serialized with Python
`pickle`; if an attacker could write to `~/.gaia/` — via a shared drive,
a sync conflict, or a malicious extension — loading that cache could
execute arbitrary code.

v0.17.3 replaces `pickle` with signed JSON: caches are now serialized as
JSON and authenticated with HMAC-SHA256 using a per-install key stored
at `~/.gaia/cache/hmac.key`. Unsigned or tampered caches are rejected
and transparently rebuilt on the next query. Old `.pkl` caches from
previous GAIA versions are ignored and re-indexed the next time you
query a document.

**You should upgrade if you** share `~/.gaia/` across machines (Dropbox,
iCloud, network home directories), run GAIA in a multi-user environment,
or have ever imported RAG caches from another source.

---

## Bug Fixes

- **Ask Agent attaches files before sending to chat** (PR #725) —
Dropped files are indexed into RAG and attached to the active session
before the prompt is consumed, so the model sees the document on the
first turn instead of the second.
- **Document indexing failures are surfaced** (PR #723) — A document
that produces 0 chunks now raises `RuntimeError` in the SDK and surfaces
as `indexing_status: failed` in the UI, instead of looking like a silent
success. Covers RAG SDK, background indexing, and re-index paths.
- **Encrypted or corrupted PDFs produce actionable errors** (PR #784,
closes #451) — RAG now raises distinct `EncryptedPDFError` and
`CorruptedPDFError` exceptions instead of generic failures, so you see
exactly what went wrong.
- **RAG index thread safety hardened** (PR #746) — Adds `RLock`
protection around index mutation paths and rebuilds chunk/index state
atomically before publishing it, so concurrent queries read consistent
snapshots and failed rebuilds no longer leak partial state.
- **MCP JSON-RPC handler guards against non-dict bodies** (PR #803) — A
malformed JSON-RPC payload (array, string, null) now returns HTTP 400
`Invalid Request: expected JSON object` instead of an HTTP 500 from a
`TypeError`.
- **File-search count aligned with accessible results** (PR #754) — The
returned count now matches the number of files the tool actually
surfaces, instead of a pre-filter total that over-reported results the
caller could not access.
- **Tracked block cursor replaces misplaced decorative cursor** (PR
#727) — Fixes the mis-positioned blinking cursor in the chat input box,
which now tracks the actual caret position via a mirror-div technique.
- **Ad-hoc sign the macOS app bundle instead of skipping code signing**
(PR #765) — The `.app` bundle inside the DMG now carries an ad-hoc
signature, so Gatekeeper presents a single "Open Anyway" bypass in
System Settings instead of the unrecoverable "is damaged" error. Full
Apple Developer ID signing is still being finalized.

---

## Release & CI

- **Publish workflow: single approval gate, no legacy Electron apps**
(PR #758) — Removed the legacy jira and example standalone Electron apps
from the publish pipeline; a single `publish` environment gate governs
PyPI, npm, and installer publishing.
- **Claude CI modernization** (PR #797, PR #799, PR #783) — Migrated all
four `claude-code-action` call sites to `v1.0.99` (pinned by SHA, fixes
an issue-handler hang), bumped `--max-turns` from 20 to 50 on both
`pr-review` and `pr-comment` for deeper analysis, upgraded to Opus 4.7,
standardized 23 subagent definitions with explicit when-to-use sections
and tool allowlists, and added agent-builder tooling (manifest schema,
`lint.py --agents`, BuilderAgent mixins).

---

## Docs

- **Roadmap overhaul** (PR #710) — Milestone-aligned plans with
voice-first as P0 and 9 new plan documents for upcoming initiatives.
- **Plan: email triage agent** (PR #796) — Specification for an upcoming
email triage agent.
- **Docs/source drift resolved** (PR #794) — Fixed broken SDK examples
across 15 docs, rewrote 5 spec files against the current source
(including two that documented entire APIs that don't exist in code),
added 20+ missing CLI flags to the CLI reference, and removed 2
already-shipped plan documents (installer, mcp-client).
- **FAQ: data-privacy answer clarified for external LLM providers** (PR
#798) — Sharper guidance on what leaves your machine when you point GAIA
at Claude or OpenAI.

---

## Full Changelog

**21 commits** since v0.17.2:

- `6d3f3f71` — fix: replace misplaced decorative cursor with tracked
terminal block cursor (#727)
- `874cf2a3` — fix: Ask Agent indexes and attaches files before sending
to chat (#725)
- `4fa121e2` — fix: surface document indexing failures instead of silent
0-chunk success (#723)
- `34b1d06e` — fix(ci): ad-hoc sign macOS DMG instead of skipping code
signing (#765)
- `7188b83c` — Roadmap overhaul: milestone-aligned plans with
voice-first P0 and 9 new plan documents (#710)
- `1beddac5` — cpp: support Ollama-compatible /v1 endpoints (#773)
- `cf9ac995` — fix: harden rag index thread safety (#746)
- `1c55c31b` — fix(ci): remove legacy electron apps from publish, single
approval gate (#758)
- `52946a7a` — feat(installer): bundle Lemonade Server MSI into Windows
installer (#774) (#781)
- `e96b3686` — ci(claude): review infra + conventions + subagent
overhaul + agent-builder tooling (#783)
- `058674b5` — fix(rag): detect encrypted and corrupted PDFs with
actionable errors (#451) (#784)
- `7bcb5d51` — fix: replace insecure pickle deserialization with JSON +
HMAC in RAG cache (CWE-502) (#768)
- `a5167e5f` — fix: keep file-search count aligned with accessible
results (#754)
- `da5ba458` — ci(claude): migrate to claude-code-action v1.0.99 + fix
issue-handler hang (#797)
- `03f546b9` — ci(claude): bump pr-review and pr-comment --max-turns 20
-> 50 (#799)
- `4119d564` — docs(faq): clarify data-privacy answer re: external LLM
providers (#798)
- `0cfbcf41` — Add example agents and integration test workflow (#340)
- `c4bd15fb` — docs: fix drift between docs and source (docs review pass
1 + 2) (#794)
- `407ed5b8` — docs(plans): add email triage agent spec (#796)
- `06fb04a4` — fix(mcp): guard JSON-RPC handler against non-dict body
(#803)
- `880ad603` — feat(installer): custom installer guide, agent
export/import, first-launch seeder (#795)

Full Changelog:
[v0.17.2...v0.17.3](v0.17.2...v0.17.3)

---

## Release checklist
- [x] `util/validate_release_notes.py docs/releases/v0.17.3.mdx --tag
v0.17.3` passes
- [x] `src/gaia/version.py` → `0.17.3`
- [x] `src/gaia/apps/webui/package.json` → `0.17.3`
- [x] Navbar label in `docs/docs.json` → `v0.17.3 · Lemonade 10.0.0`
- [x] All 21 PRs in the range (v0.17.2..HEAD) are represented in the
notes
- [ ] Review from @kovtcharov-amd addressed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cli CLI changes devops DevOps/infrastructure changes documentation Documentation changes performance Performance-critical changes rag RAG system changes tests Test changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Docs: Guide for building a custom installer with a custom agent

3 participants