Skip to content

v1.15.0

Choose a tag to compare

@github-actions github-actions released this 21 Oct 03:50
e0aafd9

Changes

Several of Prompture's prompts had been saying "do the thing" and hoping for the best. This PR rewrites the prompts the library ships so each one says what to return, how it can fail and what to treat as data. It also fixes the review loop, router and debate judge, which read free-text replies too loosely.

Highlights

  • The review loop no longer approves a review that says "not approved". Reviewers now see the original task and end with an explicit verdict.
  • The router no longer sends a request to the wrong specialist just because two agent names share a word. It can also answer "none" and fall back cleanly.
  • Debates report a winner you can read in code, alongside the judge's written verdict.
  • Text you extract from, retrieved RAG passages and TOON data are fenced off from the instructions. Instructions hidden in a document are much less likely to hijack the run.
  • DeepAgent remembers more after compacting a long run: the user's exact limits, what was ruled out and what to do next. The summarizer prompt and the base prompt can now be replaced or dropped.
  • DeepAgent's final answers lead with anything that failed or was skipped. Its sub-agents get full briefs and report back in a usable shape.
  • Malformed JSON repair now knows the target schema and the parse error, so the repaired object is more likely to validate on the first try.
Technical changes

Verdict parsing (behavior fixes)

  • review_loop._default_approve reads the last VERDICT: APPROVED | CHANGES REQUESTED line. Without one, it falls back to an "approved" that isn't negated (not, isn't, never, ...) within the same sentence. Before, any review containing the substring "approved" counted.
  • New _default_review_prompt(code, task) shows the reviewer the task and the work inside <task>/<work> tags and asks for the verdict line. AsyncReviewLoop passes the original prompt only when the default is in use, so custom review_prompt callables keep their one-argument signature.
  • groups._match_agent_name replaces both routers' _fuzzy_match_name logic:
    • It accepts the bare name, ignoring quotes and punctuation, or exactly one whole-word mention.
    • The longest name wins when one contains another.
    • NONE, or a reply naming several agents, returns None.
    • The word-overlap fallback that misrouted data_analyst vs data_engineer is gone.
  • _DEFAULT_ROUTING_PROMPT puts the request in <request> tags and offers NONE. AsyncRouterAgent now imports the prompt instead of keeping its own copy.
  • DebateConfig.judge_prompt_template:
    • It asks for strongest points, weaknesses and a verdict, ending in WINNER: <name> or WINNER: TIE.
    • parse_judge_winner maps the line to a debater name, or to the agent holding a position label such as FOR.
    • The result goes into the new DebateResult.judge_winner.
  • Debaters after round one are asked to answer the strongest opposing points instead of repeating themselves.

Untrusted input

  • extraction.core.wrap_untrusted(text, tag, what) puts text inside tags, defuses a closing tag that appears in the text, and states that the content is data, not instructions.
  • The sync and async extract_and_jsonify and manual_extract_and_jsonify paths use it. Before, instruction and text were joined with a single space. Cache keys are unchanged.
  • extract_from_data / extract_from_pandas wrap TOON data in <data> and prepend TOON_DATA_NOTE, a one-line explanation of the format.
  • render_output labels its format hint "Output format:" instead of a fake "SYSTEM INSTRUCTION:" inside the user turn.
  • rag.pipeline.DEFAULT_RAG_PROMPT puts retrieved passages inside <context> tags with the same data-not-instructions note.
  • The web tool descriptions (web_search, web_fetch, read_url, and WebSearchTool.DEFAULT_DESCRIPTION) also say that fetched content is data.

DeepAgent

  • SUMMARIZER_SYSTEM_PROMPT rewritten:
    • It lists what survives compaction.
    • It treats only [user] messages as the user's words and quotes limits exactly.
    • It folds earlier summaries in, points to files instead of copying them, and ends with a ## Next step tied to the latest request.
  • format_summarizer_input appends the virtual-filesystem paths and the current todo list, which the agent cannot read back after compaction.
  • _stringify_message labels an earlier summary [earlier summary] so it isn't mistaken for user text.
  • New summarizer_prompt= on DeepAgent, AsyncDeepAgent and both create_* factories, passed through to SummarizationMiddleware / AsyncSummarizationMiddleware as system_prompt.
  • BASE_DEEP_AGENT_PROMPT gains:
    • a check before irreversible or outward-facing tool calls;
    • a final-answer contract: report only what tool results show, lead with failures, and never present partial work as done.
  • New base_prompt= replaces the base section; "" drops it, so a persona can set the role alone. assemble_system_prompt takes the same argument.
  • TASK_TOOL_DESC_TEMPLATE and SUBAGENT_PROMPT_TEMPLATE ask for a full brief (goal, what is known or ruled out, exact inputs, what to return) and for findings rather than delegated decisions.
  • New SUBAGENT_REPORT_NOTE is added to every sub-agent system prompt by _resolve_subagent_prompt. It says the final message is all the parent sees, to lead with the answer, flag what couldn't be verified, and state assumptions.

Tool descriptions

  • Virtual-filesystem tool descriptions rewritten around use and failure modes:
    • read_file: what the line-number prefix is and how offset paging works.
    • edit_file: unique matching, replace_all for renames, empty new_string to delete.
    • glob: fnmatch * also matches /.
    • grep: Python re syntax, the three output modes and (?i).
  • Parameter descriptions tightened to match.
  • edit_file detects copied read_file line-number prefixes in old_string and says how to fix it. A plain miss suggests re-reading the file.
  • search_tools / load_tools say to check deferred tools before declaring something impossible, and to load a tool before its first call instead of guessing its arguments.
  • All built-in descriptions stay within MAX_TOOL_DESCRIPTION_CHARS.

JSON repair and evaluation

  • New json_repair_prompt(text, json_schema=, error=) shared by the sync and async clean_json_text_with_ai. Both gain keyword-only json_schema and error.
  • ask_for_json and both Conversation.ask_for_json variants pass the schema and the parser error to the repair call.
  • The validation-retry note asks for the full corrected object, using null instead of invented values.
  • DEFAULT_VERIFICATION_PROMPT in eval.faithfulness asks for a QUOTE: line of deciding evidence before the label.
  • _parse_verdict takes the last label, so a quote containing a label word can't decide the verdict. A SUPPORTED or CONTRADICTED verdict with QUOTE: NONE is downgraded to SILENT. A bare label still parses as before.

Details