Skip to content

[feat] /invoke absorbs /messages + align agent env var names#4868

Merged
ardaerzin merged 34 commits into
big-agentsfrom
chore-agent-env-flag-naming
Jun 26, 2026
Merged

[feat] /invoke absorbs /messages + align agent env var names#4868
ardaerzin merged 34 commits into
big-agentsfrom
chore-agent-env-flag-naming

Conversation

@junaway

@junaway junaway commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Context

The agent playground talked to a separate /messages endpoint for its streamed chat, while every other workflow used /invoke. Two endpoints meant two code paths to keep in sync, and /messages carried session/stream/format logic that /invoke lacked. This branch makes /invoke the single agent-capable endpoint so the frontend can drop /messages entirely, and folds in the earlier agent env-var naming cleanup this work sits on.

Changes

/invoke absorbs /messages via three negotiations.

  1. Transport. Accept is HTTP sugar over request.flags.stream: text/event-stream negotiates a stream, application/json a batch. A streaming handler aggregates to a batch when a batch is requested instead of returning 406.

  2. Message format. x-ag-messages-format: vercel projects the Vercel UI Message Stream on the way out and converts inbound UIMessage[] to canonical agenta messages on the way in. HTTP-only; in code, messages are always agenta.

  3. Session. session_id is resolved (echoed, or minted as bare uuid4().hex) once in the normalizer, written onto the request and the trace context, and echoed on every response through the body field, x-ag-session-id, and baggage ag.session.id. Trace/span/session propagation is now symmetric, inbound and outbound.

The agent handler reads request.flags.stream and request.session_id off the request instead of receiving them as injected params. The frontend agent lane (agentRequest.ts) now posts to /invoke directly with the vercel format header and data.inputs.messages, instead of rewriting the URL to /messages.

Frontend request body, before:

{ session_id, references, data: { messages, parameters } }

After:

{ session_id, references, data: { inputs: { messages }, parameters } }
headers: { Accept: text/event-stream, x-ag-messages-format: vercel }

Fixes surfaced while wiring this (verified in a running stack).

  • An errored handler returns a batch error response even when a stream was requested. The route now surfaces it as JSON with the real status code instead of a 406 that masked the actual error.
  • store_session passed a namespace kwarg that a raw OTel NonRecordingSpan rejects, so it threw on every agent request and session.id never landed on the span. It now falls back to a flat session.id key.
  • The session_id read was coupled to the trace/span link parse inside one suppress() block, so a malformed link dropped the session id from the response. The two are now decoupled.

Also in this branch. The earlier agent env-var naming pass (fa1da68): rename the agent feature's own AGENTA_* vars to one scheme, and add AGENTA_AGENT_RUNNER_HOST to the gh compose variants.

Tests / notes

  • SDK unit suite green (1026 tests), including a new matrix for the four handler shapes (sync/async x fn/generator), the three negotiations, session-id propagation, and the error-batch-on-stream path.
  • Agent service handler tests updated for the new request-based signature.
  • Frontend agentRequest unit tests updated to the new envelope and headers (123 playground tests green).
  • Stack-verified end to end: a real agent run over /invoke streams when credentialed, and surfaces the runner's real error (not a 406) when not.
  • Follow-up not in this PR: hard removal of the /messages route, the vercel/routing.py adapter, and the is_agent gate, once this is merged and the swap is proven in production.

What to QA

  • Open the agent playground for an agent workflow and send a message. The conversation streams token by token (now through /invoke, not /messages).
  • Send a multi-turn conversation. Prior turns replay correctly and the session id stays stable across turns.
  • Regression: trigger a run with a misconfigured model or missing provider key. The panel shows the real error message, not a generic 406 / empty stream.

🤖 Generated with Claude Code

jp-agenta and others added 26 commits June 19, 2026 11:43
Inbound dual of webhooks: turn external provider events into Agenta workflow runs. Adds a shared routerless connections domain (core/gateway/connections), a triggers domain (event catalog, subscriptions, deliveries), a global Composio ingress endpoint with HMAC verification + async dispatch worker, and the web UI for catalog browse and subscription/delivery management. Includes design docs and unit/acceptance tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
get_default_workspace_id no longer prefers owner-role (multi-org: an invitee owns their own empty personal workspace). Assert oldest membership wins regardless of role.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…g in tests

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Normalize the inbound provider envelope in the dispatcher into a stable
context (event.attributes + synthetic trigger_id/trigger_type/timestamp/
created_at), parallel to webhooks' event context. Resolve and complete the
bound workflow reference on subscription create/edit (the /deploy pattern) so
a variant id is resolved to a runnable revision. Align the drawer's mapping
suggestions + live preview to the same normalized shape.

Update trigger tests to the new shape and always-verify ingress; gate the
create-roundtrip acceptance tests on an ACTIVE connected account.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Resolve the bound reference via the canonical
WorkflowsService.retrieve_workflow_revision (handles application/evaluator/
workflow + environment families) and rebuild the completed family with
build_retrieval_info, so invoke_workflow finds the service uri. Raise
TriggerReferenceInvalid when it cannot resolve. Skip soft-deleted
subscriptions in the ti_* resolver. FE: scope the picker to application
workflows and send the reference family by its true kind.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ggers, schedules, webhooks

Add trigger schedules: a cron-expression analogue to trigger subscriptions that
fires the same dispatch path on each matching minute tick. Promote ti_id to a
top-level indexed column, type subscription/schedule/webhook flags, and add
/start and /stop play/pause routes (is_active) across all three domains, with
is_valid reserved for third-party connection sync on subscriptions.

- core/triggers: TriggerSchedule* DTOs, schedule CRUD + croniter validation,
  refresh_schedules fire gate, entity-agnostic dispatcher
- db: fold schedules + ti_id column + generalized deliveries into oss000000003;
  data-only oss000000004 backfills webhook flags.is_active
- cron: crons/triggers.{sh,txt} wired into oss+ee compose and all oss/ee x dev/gh Dockerfiles
- web: schedule drawer + list section, ActiveToggle, schedule atoms/hooks
- tests: schedule + webhook play/pause acceptance, dispatcher is_valid unit

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…timeouts, signature diagnostics

- Mandatory project_id in connection provider-id DAO/service/router; OAuth
  callback fails closed when project_id is unresolved. Remove dead unscoped
  ti_id lookup; document the inbound-event resolve as the one sanctioned
  cross-project read.
- Replace Dict[str,Any]/tuple/type:ignore in unreleased gateway code with DTOs:
  ConnectionStatusResponse/ConnectionRefreshResponse; CatalogIntegrationsPage
  and the tools/triggers page DTOs (integrations, actions, events).
- verify_signature: byte-exact HMAC input; compute hex + base64 and accept
  either, logging which matched (diagnostic-first until a real event confirms).
- delete_subscription: narrow bare except to AdapterError; unexpected errors
  surface to @intercept_exceptions.
- 5s asyncio.wait_for on all PR .kiq() enqueue sites (ingress -> 503).
- Guard composio 409 webhook-secret retry; explicit missing-key in catalog
  registry; web projectScopedParams scope-wins; catalog hooks isError guard.
- Manual test: drop public webhook.site default URL.
- docs: gateway-triggers findings.md

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Dispatcher: split into dispatch_subscription/dispatch_schedule sharing _run;
  add is_valid gate (invalid sub -> 409 failed delivery, never invokes);
  _write_delivery takes explicit subscription_id/schedule_id (no union).
- refresh_schedules returns failures==0 instead of always True.
- Migrations (in-place, unreleased; needs nuke): oss000000004 backfill only
  sets is_active where missing; oss000000003 adds schedule-delivery ordering
  index; rename column ti_id -> trigger_id.
- Centralize provider enablement: _sync_provider_enabled = is_active and
  is_valid, used by edit/start/stop/refresh/revoke.
- TriggerScheduleInvalid gains schedule/reason structured context.
- Webhook edit stays full-PUT; test_subscription builder carries flags.
- crons: 00-minute guard + EE-style timeout/error decode in triggers.sh and
  queries.sh.
- Naming: ti_id -> trigger_id across our code (DTO/DB/DAO/service/web);
  event context exposes event_id + event_type (drops trigger_id). Provider
  wire contract (composio metadata.trigger_id/nano_id/id) left verbatim.
- Tests + AGENTS.md doc fix.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…nature to hex

The dispatcher refactor added a required triggers_dao kwarg to TriggersWorker
but worker_triggers.py was missed, crash-looping the worker (ingress 202 but
nothing drained the queue). Live events also confirmed Composio signs lowercase
hex, so the diagnostic dual hex/base64 accept is collapsed to hex-only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…39 os.getenv

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…grade, F37 skip, F40 dup-flags

- F5: make triggers_dao/connections_service/workflows_service required (drop Optional)
- F16: extract selector-preview to pure core/selectorPreview.ts + 12 web unit tests
- F17: add schedule cron fire-gate unit tests (_validate_schedule + refresh_schedules)
- F18: try/finally cleanup in six connection/subscription lifecycle tests
- F20: symmetric oss000000004 downgrade (strip only backfill-created flags)
- F37: skip ingress dedup test when webhook secret unresolvable (no 401 flake)
- F40: remove duplicate flags key breaking @agenta/entities tsc build
- F2 verified not fail-open; F13/F24/F39 dispositioned wontfix

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds the generated Python (agenta_client/triggers/*, trigger_* types) and
TypeScript (agenta-api-client triggers resource, Trigger*/Connection*/Catalog*
types) clients for the gateway-triggers surface; renames the tool-scoped
connection/provider/auth enums to the shared Connection*/Catalog* names and
drops the obsolete Tool* variants.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…rsion

global-setup now reads the backend's /auth/discover response and promotes the
'auto' auth mode to the method the deployment actually serves: 'email:password'
(local dev / no SMTP — direct signup, no email verification) vs 'email:otp'
(SMTP/SendGrid enabled). Local dev no longer demands Testmail/OTP wiring.

run-tests excludes the agenta-web-tests workspace from the vitest layer pass so
its test:acceptance script doesn't recurse into the playwright runner. Re-enable
multi-worker by dropping the temporary workers:1 pin.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The gateway-triggers feature runs a dedicated queue consumer
(entrypoints.worker_triggers) alongside the schedule-ticking cron. It was
wired into every docker-compose variant but missing from the Helm chart, so
Helm-deployed clusters ingest+enqueue trigger events (and tick schedules via
the baked crontab) but have nothing consuming the queue — no delivery ever
fires. This is the F38 worker-wiring gap, permanent on Kubernetes.

Adds worker-triggers-deployment.yaml modeled on worker-webhooks (same image,
commonEnv, init containers, NewRelic command branch, pgrep liveness probe),
the agenta.workerTriggers.enabled/.replicas helpers, and the workerTriggers
knob in both values examples. Defaults enabled with replicas:1; toggle off
via workerTriggers.enabled=false. Verified with helm template + helm lint.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…* names

The fern regen renamed the shared gateway enums/DTOs from Tool*-prefixed to the
backend's sub-domain names (ConnectionAuthScheme/ConnectionProviderKind/
ConnectionCreateData/ConnectionStatus) and dropped the Tool* variants, breaking
the @agenta/entities build (TS2694/TS2724) in gatewayTool/core/types.ts. Point
the four package-local Tool* aliases at the new names; the aliases keep the
gatewayTool package's public surface stable. Verified: @agenta/entities builds.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…schema leak

The tools/triggers domains subclass the shared gateway Connection/Catalog DTOs
so their public surface is Tool*/Trigger*-named (per the design comments in each
dtos.py). But the subclasses were empty (`pass`), so every inherited leaf type
still resolved to its gateway name in the OpenAPI schema and leaked into the
generated clients: ConnectionProviderKind/ConnectionStatus/ConnectionCreateData/
ConnectionAuthScheme (via Connection/ConnectionCreate) and CatalogProviderKind/
CatalogAuthScheme (via CatalogProvider/CatalogIntegration). The domains even
redefined Tool/TriggerProviderKind and ToolAuthScheme but never referenced them.

Override the leaked fields so each domain fully shadows the gateway leaves:
- ToolConnection/TriggerConnection: provider_key + status -> domain-named
- Tool/TriggerConnectionCreate: provider_key + data -> domain-named
- add Tool/TriggerConnectionStatus, Tool/TriggerConnectionCreateData subclasses
- add TriggerAuthScheme; point catalog provider.key/integration.auth_schemes at
  the domain enums

Verified against the live /openapi.json: zero bare Connection*/Catalog* names
remain; all connection/catalog schema types are Tool*/Trigger*-prefixed.
Regenerated both clients (gateway leaf types deleted, domain types added) and
re-pointed the gatewayTool/core/types.ts aliases back to the honest Tool* names.
py-run-tests --api (1876) + --sdk (1007) green; web --ui unit+integration green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The /schedules create + list routes were registered without a trailing
slash while the acceptance tests, the web client, and the sibling
/subscriptions/ routes all use one. Behind the preview-env ingress the
307 redirect to the slashless path resolved against the frontend host,
returning a Next.js 404 HTML page (CI: 4 failures on POST /schedules/).

Align /schedules/ to the /subscriptions/ convention; update the two
slashless GET list calls in the acceptance test to match.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Picks up the create + list collection routes moving to /triggers/schedules/
(RPC/item routes unchanged). Python raw_client + TS generated client.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rename the agent feature's own (AGENTA_*) environment variables to a single,
consistent scheme; drop dead/over-broad knobs. No back-compat fallbacks: these
vars are unreleased on this branch. Third-party contracts (SANDBOX_AGENT_*,
PI_*, CLAUDE_*, ANTHROPIC_*, DAYTONA_*, OTEL_SERVICE_NAME, HOME/PATH) are left
untouched.

Agent workflow service (Python):
- AGENTA_AGENT_ENABLE_MCP      -> AGENTA_AGENT_MCP_SERVERS_ENABLED
- AGENTA_AGENT_CAPTURE_CONTENT -> AGENTA_AGENT_CONTENT_CAPTURE_ENABLED

Runner sidecar (TS):
- PORT                              -> AGENTA_AGENT_RUNNER_PORT
- AGENTA_HOST                       -> AGENTA_API_URL (runner derives /otlp/v1/traces)
- AGENTA_AGENT_TOOL_CALL_TIMEOUT_MS -> AGENTA_AGENT_TOOLS_TIMEOUT
- AGENTA_TOOL_BRIDGE_COMMAND        -> AGENTA_AGENT_TOOLS_BRIDGE_COMMAND
- AGENTA_TOOL_RELAY_POLL_MS         -> AGENTA_AGENT_TOOLS_RELAY_POLLING
- AGENTA_TOOL_RELAY_TIMEOUT_MS      -> AGENTA_AGENT_TOOLS_RELAY_TIMEOUT

Python SDK platform connection:
- drop AGENTA_AGENT_TOOLS_API_URL override (base URL already derives from the
  OTLP endpoint, then AGENTA_API_URL)
- drop AGENTA_AGENT_TOOLS_TIMEOUT env knob; fix the backend round-trip budget at
  5s (a single blanket timeout for arbitrary API calls is the wrong knob)

Updates consumers, unit tests, and operator-facing config (docker-compose
oss+ee, helm, railway, env templates, self-host docs). Design-history docs under
docs/design/agent-workflows/ left as record.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bind the sandbox-agent runner on 0.0.0.0 (cross-container) in the gh
local/ssl compose variants, consistent with the env-var naming convention.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Make /invoke the single agent-capable workflow endpoint so the frontend can
drop /messages. Three negotiations on /invoke:

- transport: Accept (text/event-stream | application/json) maps to
  request.flags.stream; a streaming handler aggregates to a batch when a
  batch is requested instead of 406ing.
- message format: x-ag-messages-format: vercel projects the Vercel UI
  Message Stream on the way out and converts UIMessage[] on the way in
  (HTTP-only; in code messages are always agenta).
- session: session_id resolved/minted once in the normalizer onto the
  request + trace context; echoed on every response (field, x-ag-session-id,
  baggage ag.session.id). Full symmetric trace/span/session propagation.

Also fixes surfaced while wiring this:
- error batches are returned as JSON (real status) even when a stream was
  requested, instead of a misleading 406 that masks the error.
- store_session falls back to a flat session.id key on raw OTel spans
  (NonRecordingSpan rejects the namespace kwarg).
- session_id read decoupled from the trace/span link parse so a malformed
  link can't drop it.

The agent handler now reads request.flags.stream and request.session_id off
the request instead of receiving them as params. Frontend agent lane talks to
/invoke directly with the vercel format header and data.inputs.messages.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The QA driver carries a dummy MCP record secret (not a live credential);
allowlist its gitleaks fingerprint so pushes from this branch pass the hook.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings June 26, 2026 00:46
@vercel

vercel Bot commented Jun 26, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
agenta-documentation Ready Ready Preview, Comment Jun 26, 2026 11:25am

Request Review

@coderabbitai

coderabbitai Bot commented Jun 26, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 0dc1b4c8-17be-4a49-9df6-3b51d5486b99

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch chore-agent-env-flag-naming

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@junaway junaway changed the base branch from main to big-agents June 26, 2026 00:47
@junaway junaway changed the title [chore] Fix env vars and invoke [feat] /invoke absorbs /messages + align agent env var names Jun 26, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

/invoke fully covers the agent UI Message Stream (Accept + x-ag-messages-format
negotiation, session handling), so the parallel /messages endpoint is dead.

- Drop the /messages route registration, the `_add_agent_routes` helper, and the
  `is_agent` route gate from the route decorator.
- Reduce vercel/routing.py to the Vercel protocol-header helper that /invoke still
  uses; delete make_messages_endpoint, register_agent_message_routes,
  inject_stream_session_id, and the /messages-only resolve_session_id.
- Infer `is_agent` from `:agent:` in the URI at workflow creation
  (infer_flags_from_data), like is_llm/is_hook, instead of carrying it as a route
  flag. The agent service no longer passes is_agent; the FE already detects agents
  by the builtin URI.
- Delete test_messages_endpoint.py; replace the /messages routing tests with a
  guard asserting /messages is never registered.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
jp-agenta and others added 3 commits June 26, 2026 10:21
Reconcile the upstream agent work with our /invoke-absorbs-/messages
direction. Ours wins on the contract: drop /messages, nest messages
under data.inputs, use flags.stream. Keep upstream's batch/stream
channel toggle (agentChannelModeAtom), pointed at /invoke and driving
the Accept header (negotiation #1); x-ag-messages-format declares the
message format (negotiation #2); history stays optional/default (#3).

- WorkflowRequestData drops the messages and stream fields; reads move
  to data.inputs.messages and flags.stream. Alias kept.
- is_agent inferred from the URI key at workflow creation; route gate
  removed.
- Env vars normalized to our convention: AGENTA_AGENT_SKILLS_LOADED,
  DAYTONA_AUTOSTOP (code, helm, schema, docs, env examples).
- Stale /messages comments in merged-in files updated to /invoke.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings June 26, 2026 10:12
@junaway junaway marked this pull request as ready for review June 26, 2026 10:12
@dosubot dosubot Bot added size:L This PR changes 100-499 lines, ignoring generated files. enhancement New feature or request refactoring A code change that neither fixes a bug nor adds a feature labels Jun 26, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

jp-agenta and others added 4 commits June 26, 2026 13:04
Reconcile the gateway refactor (core/gateway/*) against the base branch's
parallel evolution of the old tools service. Both sides preserved:

- Gateway refactor + triggers + subscription test-mode (this branch).
- Agent-tools resolution, no-auth toolkits, and the SDK agents/tools rewrite
  (base) ported onto the gateway structure.

Resolution highlights:
- No-auth toolkit feature ported into the gateway connections layer
  (has_auth property, _is_no_auth_toolkit, initiate short-circuit, server-owned
  validity flags, refresh no-op).
- De-Agent rename across tool symbols and the resolve endpoint
  (resolve_agent_tools -> resolve_tools); HTTP path /tools/resolve unchanged.
- read_only MCP hint restored on the action-detail path, with a regression
  guard that drives the real adapter.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The mcp_qa_server.mjs hard-coded token is a benign QA fixture (proves the MCP
tool was invoked). It was already allowlisted at its old path; the file moved
to .../projects/qa/scripts/ and re-tripped gitleaks at the new fingerprint.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
[feat] Add triggers (connections, subscriptions, schedules)
@dosubot dosubot Bot added size:XXL This PR changes 1000+ lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Jun 26, 2026
@ardaerzin ardaerzin merged commit b93e88f into big-agents Jun 26, 2026
19 checks passed
@junaway junaway mentioned this pull request Jun 26, 2026
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request refactoring A code change that neither fixes a bug nor adds a feature size:XXL This PR changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants