Skip to content

feat(email): fast dev-iteration loop for the email agent SDK#2083

Merged
kovtcharov merged 8 commits into
mainfrom
feat/email-agent-dev-iteration
Jul 17, 2026
Merged

feat(email): fast dev-iteration loop for the email agent SDK#2083
kovtcharov merged 8 commits into
mainfrom
feat/email-agent-dev-iteration

Conversation

@kovtcharov-amd

Copy link
Copy Markdown
Collaborator

Why this matters

Before: an app integrator who found a bug in the email agent was stuck — we ship the agent as a frozen, opaque binary, so their only recourse was to file an issue and wait days for a new binary + npm release. After: they run the agent's Python source (which serves a contract byte-for-byte identical to the frozen binary — the binary is that source frozen) and drive it with the same shipped TS client. Edit Python, it auto-reloads, re-test in seconds. Dev and prod differ only in where the client points.

No wire change: SCHEMA_VERSION stays 2.3, checkVersion still guards drift, existing integrations are untouched, and no eval is required (transport/packaging only — no prompt/tool/model/error-classification change).

What's in it

  • connectSidecar({ baseUrl, authToken? }) (npm) — attach mode: health- and version-checks a server you already run and returns a bound client. Spawns nothing, no child, nothing to shutdown(). The client half of the dev loop.
  • gaia-agent-email serve --reload (Python) — a new console script that runs the source agent with hot reload (token-off for local dev). The full sidecar app wiring now lives in an importable gaia_agent_email.server; packaging/server.py is a thin re-export, so the freeze entry, uvicorn server:app, and the by-path caller-auth test are unchanged.
  • agent-email dev (npm CLI) — one command to launch the source server and print the base URL to attach.

Test plan

  • cd hub/agents/npm/agent-email && npm run build && npm test — 78 pass (new connect.test.ts + dev launcher tests)
  • pytest hub/agents/python/email/tests/{test_server_cli,test_caller_auth,test_capability_matrix}.py — 35 pass
  • python -m gaia_agent_email.export_openapi --check — no contract drift
  • black/isort clean on touched Python
  • Smoke: booted the source server; /health, /version (apiVersion 2.3), /openapi.json all serve the frozen-binary contract
  • Reviewer golden path: pip install -e hub/agents/python/emailgaia-agent-email serve --reloadconnectSidecar({ baseUrl })triage, then edit a triage heuristic and confirm reload serves the change

Notes for the reviewer

  • The Python package version stays 0.4.0 (npm bumped to 0.5.0); the new console script can fold into the next Python release bump. The npm dev command's default launcher (gaia-agent-email serve) needs the Python package built from this commit — a dev-only convenience; connectSidecar itself has no such coupling.

… attach mode

Integrators who hit a bug in the frozen email sidecar had no way to fix it:
the binary is opaque, so the only recourse was to file an issue and wait for a
new binary + npm release — a multi-day loop. This adds a seconds-long loop that
runs the agent's Python source (which serves a contract identical to the frozen
binary) and drives it with the same shipped TS client. Dev and prod now differ
only in where the client points.

Python: move the full sidecar app wiring into an importable
`gaia_agent_email.server` (single source of truth) and reduce `packaging/server.py`
to a thin re-export so the freeze entry, `uvicorn server:app`, and the by-path
caller-auth test are unchanged. Add a `gaia-agent-email` console script with a
`serve` command supporting `--reload`/`--dev` for the hot-reload dev loop.

npm: add `connectSidecar({ baseUrl })` (attach mode — health + version check
against a server you already run, spawns nothing) and an `agent-email dev` CLI
that launches the source server. No wire change: SCHEMA_VERSION stays 2.3.
@github-actions github-actions Bot added documentation Documentation changes agent::email Email agent changes labels Jul 14, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Verdict: Approve with suggestions.

This PR adds a fast local dev loop for the email agent: a new importable gaia_agent_email.server (with the frozen packaging/server.py reduced to a thin re-export), a gaia-agent-email serve --reload console script, an agent-email dev npm CLI, and a connectSidecar() attach-mode client. No wire change (SCHEMA_VERSION stays 2.3), so existing integrations are untouched — the eval skip is justified. The refactor keeps the freeze entry and by-path caller-auth test working, and a new parity test guards against source/frozen drift. Clean, well-tested, and the five doc surfaces (README/SPEC/SKILL/CHANGELOG + the guide) are all in sync.

The one thing worth fixing before merge: when agent-email dev fails to launch the Python server (e.g. wrong --python path, package not installed), the CLI prints the actionable hint immediately but then hangs for a full 60 seconds before exiting — because the health-poll only stops on its own timeout, not on the spawn error. That reads as "is it working or stuck?" and undercuts the tool's whole purpose. Everything else is minor.

🔍 Technical details

🟡 Important

60s hang after an immediate launch failure (src/cli.ts:363-389). child.on("error") sets launchError and prints the hint on the next tick, but connectSidecar({ healthTimeoutMs: 60_000 }) keeps polling /health against a port nothing is listening on until its own deadline (waitForHealth loops until Date.now() >= deadline, lifecycle.ts:285). On ENOENT the child never emits exit, so nothing short-circuits the wait — the user sees the error, then a 60s silent hang before exit code 1. Consider racing the launch error against the health wait so a spawn failure aborts the poll immediately, e.g.:

  const baseUrl = `http://${host}:${port}`;
  const launchFailed = new Promise<never>((_, reject) => {
    child.once("error", (e) => reject(e));
  });
  try {
    const dev = await Promise.race([
      connectSidecar({ baseUrl, healthTimeoutMs: 60_000 }),
      launchFailed,
    ]);

(then keep the existing catch that calls killDevTree and returns 1 when launchError is set). This turns a 60s hang into an instant, clear failure.

🟢 Minor

killDevTree doc comment says SIGKILL, code sends SIGTERM (src/cli.ts:308). The docstring reads "SIGKILL the launched dev-server process tree" but the POSIX path uses process.kill(-child.pid, "SIGTERM"). Harmless, but the comment misleads a future reader debugging shutdown.

/** Terminate the launched dev-server process tree (uvicorn --reload forks a child). */

"caller-token off for local dev" comment slightly overstates --reload (README.md:114, email-integration.mdx:24). --reload doesn't disable the token; it's off only when GAIA_EMAIL_SIDECAR_TOKEN is unset (the prose two paragraphs down states this correctly). The inline comment reads as if --reload itself turns auth off. Minor wording only — non-blocking.

Strengths

  • Source/frozen parity is enforced, not just asserted in prose. test_server_cli.py::test_shim_serves_identical_routes_to_in_package_builder compares route sets and checks shim.build_app is server.build_app, so the re-export can't silently fork from the frozen contract.
  • Clean refactor discipline. packaging/server.py collapses to a re-export while preserving the freeze entry, uvicorn server:app --app-dir, and the by-path test_caller_auth loader — the tricky no-__init__.py constraint is respected and documented.
  • connectSidecar is well-covered (test/connect.test.ts): auth-token forwarding, version-mismatch, verifyVersion: false bypass, and the missing-baseUrl TypeError are all tested.
  • Doc-sync done right — the same claim lands in README, SPEC, SKILL, CHANGELOG, and the guide together, exactly as CLAUDE.md requires.

Ovtcharov added 3 commits July 14, 2026 12:21
Review follow-ups on the fast-iteration loop:

- `agent-email dev` now races readiness against an early child exit/spawn error,
  so a bad launcher (typo, missing package, taken port) fails immediately with an
  actionable ENOENT hint instead of waiting out the full health timeout.
- Rewrote the "Fast local iteration" guide (email-integration.mdx + README) as a
  scannable numbered loop — the doc a developer iterating on the agent actually
  wants: one-line why, three copy-paste steps, one line back to production.
- Doc accuracy: the sidecar app wiring + inline probes now live in
  `gaia_agent_email.server`, so the caller-auth docstrings and the capability
  matrix footnote point there (regenerated CAPABILITY_MATRIX.md) instead of
  `packaging/server.py`, which is now a thin re-export.
…ntract

Second review pass, hardening edge cases:

- `connectSidecar` now takes an `AbortSignal`, and `agent-email dev` uses it: when
  the launched server dies before it's ready, the health poll is cancelled instead
  of probing for the full 60s in the background (masked by process.exit in the CLI,
  but a real leak for programmatic callers). The losing race branch's rejection is
  pre-caught so it's never unhandled.
- `serve --reload`/`--dev` now fails loud in a frozen binary (no source to watch,
  and uvicorn's reloader would re-exec the exe) instead of misbehaving.
- Locked the frozen-binary invocation contract with a test: `main(["--host", H,
  "--port", P])` (no `serve` token, exactly how spawnSidecar and smoke_test.py
  launch it) normalizes to a serve run of the pre-built app.
- Added a fast-fail test for `agent-email dev` (returns 1 well under the health
  timeout when the launcher can't be spawned).
A developer iterating on triage would otherwise be confused when the source
server starts fine (health is liveness-only) but live triage returns 502 until a
local Lemonade Server is up. One bullet, so they know to start Lemonade for LLM
paths and can iterate on non-LLM code before then.
@github-actions

Copy link
Copy Markdown
Contributor

🟡 The Python package version wasn't bumped to match the npm package bump, which will break the release CI.

The npm package went from 0.4.00.5.0 in package.json, but gaia_agent_email/version.py still says AGENT_VERSION = "0.4.0". The release workflow runs stamp_version.py --check, which validates that every version reference (including hub/agents/npm/agent-email/package.json) matches AGENT_VERSION — that check will fail and block the release. Independently, the Python package also gained a new public entry point (gaia-agent-email console script) that warrants the bump on its own: users with 0.4.0 installed won't know to upgrade to pick up the new command.

Fix: bump AGENT_VERSION in version.py to "0.5.0", then run stamp_version.py to propagate it to pyproject.toml and the other targets the script manages.

🔍 Technical details
  • hub/agents/python/email/gaia_agent_email/version.py:31AGENT_VERSION = "0.4.0" (needs "0.5.0")
  • hub/agents/python/email/pyproject.toml:7version = "0.4.0" (will be fixed by stamp_version.py)
  • hub/agents/python/email/packaging/stamp_version.py:113–116 — checks npm package.json against AGENT_VERSION; a mismatch exits non-zero

The correct flow: update version.py first, then run python hub/agents/python/email/packaging/stamp_version.py to stamp pyproject.toml and the npm package.json badge/lock targets in sync.

Closes a hidden CI gap. The sidecar-app contract tests
(tests/unit/test_email_sidecar_devmode_app.py + _server_wiring.py) verify that
`uvicorn server:app --app-dir <email>/packaging` resolves and that the live
sidecar routes correctly — the exact contract the packaging/server.py shim +
in-package build_app must uphold. But they live at tests/unit/ top level, which
only test_unit.yml runs, and that workflow triggers on src/**/tests/** — NOT
hub/agents/python/email/**. So a PR that refactors only the email package (like
the dev-iteration server move) could break Agent UI dev mode with all its own
checks green.

Wire both guards into test_email_agent_unit.yml, which already triggers on the
email package and installs core + the email package editable — so an
email-package change now runs the dev-mode contract guard it can break.
@github-actions github-actions Bot added the devops DevOps/infrastructure changes label Jul 14, 2026
Resolved README.md conflict by taking main's rewritten lean overview — main
restructured the README to delegate detail to SPEC.md, and my "Fast local
iteration" content already lives in SPEC.md + docs/guides/email-integration.mdx,
matching that structure.

Merge notes:
- CHANGELOG: folded my dev-iteration entry into main's single 0.5.0 section (was
  duplicated), in main's plain-language voice; dropped the "SCHEMA_VERSION stays
  2.3" claim since main's 0.5.0 also lands the 2.4 query endpoint.
- Versions now align at 0.5.0 (main bumped py + npm; my [project.scripts] intact).
- Verified the merged sidecar (my in-package build_app) still serves main's new
  /v1/email/query (2.4) — it's included into api_routes.router, which build_app
  mounts. Regenerated CAPABILITY_MATRIX.md.
- npm build + 79 tests pass; hub email suite 577 pass (1 pre-existing env-only
  version-metadata failure, green in CI); Agent UI dev-mode guards pass.

@kovtcharov kovtcharov left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept and execution both look right: dev-source parity through the same shipped TS client is exactly the integrator story #1653/#1896 wanted, the fail-loud resolveBinaryPath guidance is good, and the npm suite passes on the branch (79 tests, verified locally). Doc surfaces updated together per the multi-surface rule.

One sequencing flag before merge: this branch predates both #2102 (the npm /query client — lifecycle.ts/index.ts/package-lock overlap) and #2060 (which moves the whole npm package to hub/agents/email/npm/). Merging order matters: rebase onto current main first (expect conflicts in lifecycle.ts/index.ts with #2102's additions — resolve by keeping both features), and coordinate with #2060 so whichever lands second re-runs its path/rename resolution. After the rebase the suite should be at 101+79-overlap tests, not 79 — worth confirming the /query tests still pass alongside the dev-loop ones.

@itomek

itomek commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

Parking this until #2142 (V2-6: AgentSidecarManager + dev-mode registration) is picked up in the milestone-55 run — the serve --reload / importable gaia_agent_email.server half partially implements that issue's dev-mode scope, so it will be finished and rebased then rather than redone. Needs a rebase over #2102 and coordination with #2060's tree move before merge.

kovtcharov
kovtcharov previously approved these changes Jul 17, 2026
test_shim_serves_identical_routes_to_in_package_builder failed on CI because
its Starlette represents include_router-mounted routes as _IncludedRouter
objects without a .path attribute — every /v1/email/* path read as None, so
'/v1/email/triage' was never in the introspected set even though the route is
mounted (the same run's --print-openapi test proves it serves). Locally a newer
Starlette exposes .path, masking the failure.

Assert parity the version-robust way the sibling dev-mode tests already do:
builder identity (the shim re-exports build_app), OpenAPI-contract equality
between both entry points, and TestClient reachability probes (!= 404) for the
canonical surfaces.
@kovtcharov
kovtcharov added this pull request to the merge queue Jul 17, 2026
Merged via the queue into main with commit ba58e31 Jul 17, 2026
33 of 34 checks passed
@kovtcharov
kovtcharov deleted the feat/email-agent-dev-iteration branch July 17, 2026 23:30
@itomek itomek mentioned this pull request Jul 22, 2026
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent::email Email agent changes devops DevOps/infrastructure changes documentation Documentation changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants