Skip to content

v1.3.3

Choose a tag to compare

@cryxnet cryxnet released this 11 Oct 19:59
· 31 commits to main since this release

A patch release with fixes found while writing and running the promptise.com guides. Upgrade with pip install -U promptise.

Highlights: the agent runtime's mission judge, max_invocations, health actions, journal API, live instruction updates and MCP client cleanup now work; explicit server credentials (API keys) win over agent identity and forwarded caller tokens; standard RFC 6749 token requests; the debate, deliberate, research and autonomous reasoning patterns end with a real answer; code-action's tool-call budget counts every call and fails loudly, MCP results arrive decoded, and sandbox cleanup takes ~0.1 s; open-mode tools take their declared parameters. Read Changed before upgrading.

Fixed

  • The mission LLM judge always failed -- MissionTracker built its judge with build_agent() without the required servers argument, so every evaluation raised TypeError before any model call and was recorded as achieved=False, confidence=0.5, reasoning="Evaluation failed: build_agent() missing 1 required keyword-only argument: 'servers'". With the default confidence_threshold (0.7) the mission went to paused after its first evaluation, evaluations stopped, and auto_complete could never trigger. The judge is now built with no servers and evaluates the evidence; it is shut down after each evaluation. MissionTracker.evaluate() also accepts a chat model instance as model.
  • MissionConfig.max_invocations was never enforced -- a mission ran past its limit indefinitely. The mission now fails once it reaches max_invocations successful invocations: right after the last one (once its evaluation has run, so a mission completed by that run is not failed), with a mission.failed event whose reason is "max_invocations", and later invocations are skipped, as for timeout_hours.
  • Health on_anomaly was skipped unless the anomaly was less than 5 s old when the run ended, and never applied to a run that raised -- AgentProcess acted only on an anomaly detected in the last 5 seconds, and only after a successful run. A stuck agent keeps calling after the third identical call, so by the end of its run the anomaly was older: no pause, no escalation, no health.anomaly event. A stuck loop that ended in GraphIterationLimitError (or any error) never reached the check, and a high_error_rate anomaly was acted on only if a successful run ended within 5 s. Each anomaly is now acted on once, when the run that produced it ends, whether it succeeded or raised, however long ago it was detected; high_error_rate is acted on when the failure is recorded. A later healthy run no longer re-applies the action for an anomaly that was already handled (it did within the 5 s window). Every new anomaly emits its own health.anomaly event; the action runs once per run, for the latest.
  • GET /api/v1/processes/{name}/journal was always empty for processes with the default process_id -- the endpoint read entries by process_id while the process files them under its name (so they survive a restart), and AgentRuntime.add_process() gives a random UUID process_id. It now reads by name.
  • PATCH /api/v1/processes/{name}/instructions changed nothing until a restart -- it answered "effective_from": "next_invocation" but only set fields read when the agent is built (start(), a restart, an open-mode hot reload). The next invocation now rebuilds the agent with the new instructions before it runs, keeping the connected MCP servers, conversation history and process state; an invocation already running finishes with the old ones. The new AgentProcess.update_instructions(text) does the same from Python. The new instructions replace any an open-mode agent set for itself (rollback() keeps them), and a non-string instructions value is refused with 422 INVALID_CONFIG.
  • AgentProcess never closed its MCP client, so every stop, restart and hot reload leaked the previous connection and its stdio servers -- the process opens its own MCPMultiClient for the servers in its config, and nothing closed it: each stop()/start() cycle, restart-policy restart or open-mode hot reload left the previous stdio server processes running (still stuck in a hung call, if they were). stop() and restarts now close the client after shutting the agent down, and a hot reload closes the replaced agent and its client once the invocation that asked for the reload has finished with them (that run keeps the tools it started with). If a hot reload's rebuild fails, the process keeps its previous agent (it was shut down before the rebuild); a client whose agent fails to build is closed.
  • An agent identity no longer overrides a server's api_key -- build_agent(identity=...) (and AgentProcess) presented the identity's bearer to every server without a bearer_token, including one configured with HTTPServerSpec(api_key=...). The client sent Authorization: Bearer <identity token> next to x-api-key, the server's auth gate checked the bearer first, and the connection failed with 401, so one agent could not use identity-authenticated servers and API-key servers together. An explicit per-server credential now always wins: bearer_token, api_key, or an Authorization / x-api-key header in headers.
  • The built-in token endpoint and MCPClient.fetch_token() speak OAuth 2.0 (RFC 6749) -- enable_token_endpoint() read only a JSON body and answered a standard form-encoded client_credentials request with 400, and fetch_token() posted JSON without grant_type, so it failed against a standard token endpoint. The endpoint now accepts application/x-www-form-urlencoded requests and HTTP Basic client authentication, answers errors with the standard error_description (message is kept), and sends Cache-Control: no-store. JSON bodies are still accepted. fetch_token() sends a form-encoded grant_type=client_credentials request, and retries once as JSON when a built-in endpoint from 1.3.2 or earlier asks for JSON.
  • Tool name collisions: the list matches the routing -- when two servers expose a tool with the same name, MCPMultiClient.list_tools() (and so the agent's tool list) contained every copy, while calls went to the server listed last and the warning said only that version is used. It now returns only that server's copy; list_prompts() likewise. A server whose session is re-opened no longer takes a colliding name over from a server listed after it.
  • promptise audit verify --strict no longer prints "OK ... intact." when it fails -- with only warnings (a truncated last line, say), the summary said OK while the exit code was 1. It now reads FAILED ... no tampering found, but N warning(s) fail the check under --strict, and --json adds strict and passed (the exit code's verdict) next to ok (the chain's verdict).
  • A built-in identity guard implies auth=True -- guards=[RequireAuth()] (or HasRole, HasScope, RequireClientId, the tenant guards) on a tool without auth=True denied every caller, valid credentials or not, because AuthMiddleware authenticates only auth=True tools. Such a guard now implies auth=True, as roles=[...] already did, on tools, resources, prompts and router-level guards. Custom guards are unchanged.
  • Custom auth providers can hand their claims to AuthMiddleware -- the documented ctx.state["jwt_payload"] = claims was ignored (AuthMiddleware read only the built-in providers' private _jwt_payload), so roles, scopes and tenant from a custom provider never reached ctx.client, and roles=[...] denied every call. ctx.state["jwt_payload"] is now the public key for a custom provider's verified claims.
  • MCPClient no longer triggers the MCP SDK's DeprecationWarning -- with mcp 1.30 every HTTP connection warned Use streamable_http_client instead. The client now uses streamable_http_client when the installed SDK has it and falls back to streamablehttp_client on older SDKs (down to the minimum, mcp 1.9).
  • Per-client rate limits that collapse into one shared bucket are reported -- RateLimitMiddleware keys its buckets by ctx.client_id, which is set only when AuthMiddleware runs before it and the tool is authenticated; otherwise every caller silently shared one "global" bucket, and one runaway client throttled all the others. When an AuthMiddleware is installed but cannot identify the caller for the limiter (it runs after the limiter, or the tool is not auth=True), the server now logs a warning naming the tools, once, at startup (and on TestClient's first call). A key_func marks the key as deliberate and is not reported. A limiter without a synchronous consume(key) method now raises TypeError when RateLimitMiddleware is created, instead of failing every call with INTERNAL_ERROR.
  • A forwarded caller token no longer overrides a server's api_key -- forward_caller_token is on by default, so during an invocation whose CallerContext carries a bearer_token, every tool call (and read_resource / get_prompt) to a server configured with HTTPServerSpec(api_key=...) or an x-api-key header went over a session that sent Authorization: Bearer <caller token> next to x-api-key. The server's auth gate checked the bearer first and answered 401, so an API-key server failed every call made for a user with a token: the agent-identity bug above, again. An API key now wins by default and the caller's token is not sent to that server. HTTPServerSpec(forward_caller_token=True) still forwards it, explicitly: both headers are then sent and the server decides which counts (a Promptise server checks the bearer first). The resolved choice is HTTPServerSpec.forwards_caller_token.
  • A .agent manifest dropped forward_caller_token -- manifest_to_process_config() did not carry it to the HTTPServerSpec, so a manifest could not opt a server out of receiving caller tokens. It is carried now, as AgentProcess's own dict normalisation already did.
  • agent_pattern="debate" replied with the judge's verdict, never looped, and had no tools -- its critic and judge answered in plain text (no output_schema), so the critic's severity never sent the proposer back (it had no revise route either), the judge's on_fail never fired, and the run went proposer → critic → judge once and replied with the judge's text ("1) Does it pass ALL criteria? False ..."). build_agent also built it without the agent's tools. Critic and judge now return structured output (DebateCritique, DebateVerdict in promptise.engine.prebuilts): a severity above 0.3 or a failed verdict sends the answer back to the proposer, which sees the critique and verdict and has the agent's tools; a new answer stage writes the reply. Every stage has a budget (critic and judge max_rounds, proposer max_proposer_steps), so a debate always ends with a written answer. Same bug class as the 1.3.0 PEOATR fix.
  • agent_pattern="deliberate" replied with its reflection and never looped -- plan and reflect answered in plain text, so the plan's quality gate and reflect's continue / replan / answer route never fired: every run went think → plan → act → observe → reflect once and replied with the reflection ("1) Overall progress ... 2) Mistakes ..."). Plan and reflect now return structured output (PeoatrPlan, PeoatrReflection), so their routes fire, every stage has a budget (plan 3, act max_act_steps, reflect 4), and a new answer stage writes the reply.
  • agent_pattern="research": the verify step could not fail -- it was a GuardNode with no guards: it passed every time without a model call (the "cross-check the findings" instructions it was given were never used), so the documented loop back to search never happened. Verify is now a model judgement (ValidateNode) that checks the findings against the tool results in the conversation and returns a ResearchVerdict (passes, issues): every claim backed by a tool result, nothing contradicting the sources, the whole question covered. A failed check sends the run back to search, which sees the issues; verify runs at most max_verify_rounds times (default 2), then the run goes to synthesize. Search now has a budget for the whole run (max_search_steps, default 12; it was 5, and a search that used it ended the run without an answer) and hands over to synthesize when it is used.
  • agent_pattern="autonomous" could reply with routing JSON, named tools as steps, and ran one tool round per step -- every routing call appended the full planner prompt and step menu to the conversation as a new system message and the model's JSON choice as an AI message, so the prompts piled up and a run that finished could reply {"next_node": "__done__", ...} or a JSON error object; a chosen tool step ran one model call, so it made one round of tool calls and its model did not see the results unless the agent picked it again; and the model often named a tool ("get_policy" is not available). Now the routing exchange is sent for its one call and never added to the conversation; a chosen step runs its whole tool loop (re-run while its model calls tools, up to its max_iterations; the default reason step allows 10 rounds, max_tool_rounds); a tool name runs the step that has that tool, and an unknown name is reported to the next routing call only; and when the agent finishes without its terminal node having run last, that node runs (synthesize in the default pool, or the new AutonomousNode(answer_node=...); with no terminal node, the node asks the model for the final answer), so the reply is always a written answer. The default pool is built without tools too (it was empty, and the run failed with "No nodes in the pool"). AutonomousNode now streams its steps' model chunks and tool events, so astream_with_tools() returns the answer (its DoneEvent was empty).
  • PromptGraph.peoatr() / .research() passed to build_agent as a graph ran as a node pool -- they were built in the default "autonomous" mode with no edges, so build_agent(agent_pattern=PromptGraph.peoatr(...)) wrapped their nodes in an AutonomousNode instead of running the graph. They are now static graphs, like the other built-in patterns.
  • Code-action: max_tool_calls did not count failed tool calls -- the cap compared it with the number of successful calls, so a program could call a failing tool without limit (331 calls ran under the default cap of 50; a program looping a failing tool 120 times under max_tool_calls=10 made all 120). Every call a program makes now counts: successful, failed, an unknown tool, or one a pre_tool hook blocked.
  • Code-action: reaching max_tool_calls was silent, and a repair could not recover from it -- the bridge answered the call with an error and logged nothing, so a program that caught exceptions finished with a wrong answer (report.error None); and the budget was shared with the repair attempts, so a program repaired after hitting the cap failed its first call (or printed a made-up result such as None spent $0.00). Now a call beyond the cap raises ToolBudgetExceeded in the program (a RuntimeError, without running the tool, and so does every later call), the node logs a warning, and the program's output is discarded even if it caught the error. The model is told the program needed more than max_tool_calls calls and asked for one that needs fewer; each repaired program gets a fresh budget, so a run makes at most (max_repairs + 1) × max_tool_calls calls. With no repair left the run raises GraphExecutionError, and ExecutionReport.error names max_tool_calls. The prompt states the limit. A crash's error now ends the final error message (program failed after N attempt(s); last attempt: ...).
  • Code-action: MCP tool results reached the program as JSON text -- an MCP tool answers with the text of the server's result, so list_orders("2026-09") arrived as the string '["SO-9001", ...]' and programs iterated its characters, although the docs said data arrives as lists and dicts. An MCP tool's text result that is valid JSON now reaches the program decoded (lists, dicts, numbers; other text stays a str), also behind the approval, guardrail and tracing wrappers. A Promptise error envelope ({"error": {"code": ..., "message": ...}}) raises in the program as an MCPToolError, like an MCP error result. Python tools are unchanged: a str they return stays a str.
  • Code-action: ExecutionReport.tool_calls was always 0 -- CodeActionNode recorded its calls without counting them. The report now counts the programs' calls, failed ones included, and each observation has its duration.
  • Sandbox cleanup always took 5 s -- SandboxSession.cleanup() stopped the container gracefully before removing it, and the container's /bin/bash (PID 1) ignores SIGTERM, so every cleanup waited out docker stop's 5 s timeout, blocking the event loop. Code-action paid it once per program and once more for build_agent's probe. A non-persistent session's container is now killed (new SandboxBackend.kill_container(), which defaults to stop_container()) and removed, about 0.1 s; a code-action program with no tool calls takes about 1 s in all. The Docker calls in stop_container() / remove_container() no longer block the event loop.
  • Open mode: create_tool ignored its parameters -- every agent-written tool took one input string, so a tool written as def run(a, b) failed with run() got an unexpected keyword argument 'input'. The declared JSON object schema is now the tool's argument schema: the model is offered its parameters, the arguments are validated (invalid ones raise ToolArgumentError), and run gets them as keyword arguments, as plain JSON, with each property's schema default for one left out. A parameters that is not an object schema (or lists parameters without properties, or names one that is not a valid Python keyword argument) is refused with an error and no tool is created. Empty parameters keep the free-form input string.
  • Open mode: a change refused by max_rebuilds was applied on the next restart -- the meta-tools set the new instructions, tool or server before the hot reload checked the limit, and the refused change stayed in the process's dynamic state, which the next start() (deploy, crash recovery, restart_policy) built the agent from. The limit is now checked before anything changes, under the rebuild lock, and a refused change is not made at all: modify_instructions, create_tool and connect_mcp_server answer Error: max rebuilds (N) reached; the change was not applied (create_tool and connect_mcp_server used to say the tool was created or the server connected). A change whose rebuild fails is undone too (a failed create_tool left its tool behind). The limit counts over the process object's lifetime, as documented; restarts do not reset it.

Documentation

  • Runtime reference corrected where it did not match the code: mission states are active / paused / completed / failed, eval_every defaults to 1, the injected [Mission] block and what a low-confidence or failed evaluation does (docs/runtime/governance/mission.md); the injected [Budget Remaining] block and the budget escalation payload ("type": "budget_violation", violation as an object with limit_name, limit_value, current_value, tool_name) (budget.md); on_anomaly accepts "log", "pause" and "escalate" (the docs listed "stop", which HealthConfig rejects), when the action runs, error-rate detection counts failed invocations, and the HealthMonitor method list (health.md); the instructions and journal endpoints (api.md), hot reload (meta-tools.md) and stop/restart (processes.md).
  • testing.md "Testing with authentication": passed algorithm= (not a JWTAuth parameter), relied on a guard without auth=True, and made a token without exp; it now runs, and the test suite runs it.
  • auth-security.md "Custom auth provider": stored the claims under a key AuthMiddleware did not read; it now shows the jwt_payload contract and runs.
  • caching-performance.md "Tiered rate limiting": defined async check(key, ctx), which RateLimitMiddleware never calls (every call failed with INTERNAL_ERROR); it now uses consume(key) with a key_func that puts the tier in the key. A new note says when buckets are per client.
  • showcase.md: the database and payments MCP server snippets used MCPServer(auth=...), AuthMiddleware() without a provider and RateLimitMiddleware(requests_per_minute=...), none of which exist, and passed "${JWT_SECRET}" / "${AUDIT_SECRET}" literally as keys. They read the keys from the environment and run.
  • approval-gates.md "Audit visibility": the approval request id is in the error the client receives and in the gate's log lines, not in the audit entry as the page said.
  • observability.md: a key rotation warning comes on the first audited call after startup, not at startup.
  • identity/guide.md: an explicit per-server api_key (or x-api-key header) wins over the agent identity, like bearer_token.
  • multi-user-systems.md and secure-multi-tenant-platform.md: AuditMiddleware(secret=...) raised TypeError (the parameter is hmac_secret), and JWTAuth, AuditMiddleware and PostgresConversationStore were given "${JWT_SECRET}"-style strings, which these classes take literally (only .superagent files and a few documented parameters expand them). The examples read the keys from the environment with os.environ[...] and the audit examples write audit.jsonl, verifiable with promptise audit verify. The 30-tool server in multi-user-systems.md also used MCPServer(auth=...), RateLimitMiddleware(requests_per_second=...) and server.mount(router), none of which exist, and an on_authenticate hook with the wrong signature; it runs now, and the test suite runs it and both audit examples. The same literal-key fix in config.md (HTTPServerSpec) and a blog post (JWTAuth); showcase.md's gateway example includes its routers with include_router().
  • Server Specs, Multi-User Identity, Multi-Tenancy, Building Agents and Tool Adapter pages: which credential a server sees for a caller with a token (forward_caller_token unset, True or False, with and without an API key).
  • agentic-runtime.md no longer shows MissionConfig(on_complete=...); mission.md lists it as deprecated.
  • reasoning-patterns.md and engine-prebuilts.md: debate, deliberate, research and autonomous described as they now run (debate's critic and judge loops and answer stage, deliberate's routes and answer stage, research's model-judged verify loop and its bound, autonomous's tool loops, routing and final answer); engine-nodes.md and engine-internals.md: AutonomousNode (answer_node, the tool loop per step, routing kept out of the conversation).
  • Code-action (guide, engine-prebuilts.md, reasoning-patterns.md, building-agents.md): the tool-call budget, MCP results arriving as data, and the latency per program (about 1 s, not 1-2 s). Open mode (docs/runtime/meta-tools.md, configuration.md): how create_tool uses parameters, and what max_rebuilds counts and refuses. Sandbox (docs/core/sandbox.md): cleanup() kills the container, and kill_container().

Deprecated

  • MissionConfig.on_complete -- it was documented ("stop", "continue" or "suspend") but never read: a completed mission stopped the process whenever auto_complete=True, whatever on_complete said. Setting it now emits a FutureWarning; it will be removed. Use auto_complete (False keeps the process running after completion; its later invocations are skipped).

Changed

  • A tool, resource or prompt with a built-in identity guard is now authenticated: a caller without credentials gets AUTHENTICATION_ERROR where it got ACCESS_DENIED (it was denied either way).
  • MCPMultiClient.list_tools() / list_prompts() return one entry per name when servers collide (the copy calls are routed to).
  • A form-encoded request to the built-in token endpoint must carry grant_type (RFC 6749); a JSON request still defaults to client_credentials.
  • HTTPServerSpec.forward_caller_token defaults to None: forward the caller's token unless the server has an API key. A server with an API key no longer receives caller tokens unless it sets forward_caller_token=True; True and False behave as before.
  • PromptGraph.debate() has a fourth node, answer, and new parameters tools, blocks and max_proposer_steps; max_rounds now bounds the critic and the judge (default 3, was 5 for the proposer, which never looped). PromptGraph.deliberate() has a sixth node, answer, and new parameters blocks and max_act_steps. PromptGraph.research() takes max_search_steps and max_verify_rounds, and its verify node is a ValidateNode (a model call per check) instead of a GuardNode; pass verify=False to skip it. PromptGraph.autonomous() takes blocks and max_tool_rounds. The multi-stage patterns make one more model call than before: their answer stage (and research's verify).
  • Code-action's max_tool_calls caps each program, not the whole run: a repaired program gets a fresh budget (a run makes at most (max_repairs + 1) × max_tool_calls calls), and failed calls count. A program that reaches it fails instead of answering.
  • Code-action programs get MCP tool results that are JSON text decoded, so code that parsed the text itself (json.loads(list_orders(...))) now gets a list and must drop the parse.
  • AgentProcess.rollback() is no longer refused once max_rebuilds is reached (it cleared the dynamic state and returned the refusal, so the rollback happened only at the next restart); it counts as a rebuild. If its rebuild fails, the previous state is restored.
  • An open-mode tool created with parameters takes those parameters instead of input.

Full changelog: https://github.com/promptise-com/foundry/blob/main/CHANGELOG.md