fix: Parlant tool descriptions drift from SDK master models - #545
Conversation
Parlant hand-wrote its tool docstrings instead of reading the master Pydantic models in runtime/tools.py, so a reword never reached Parlant agents and the docs drifted (stale unprefixed add_participant/ remove_participant example calls for tools actually named band_add_participant/band_remove_participant). Introduces a band_tool decorator that sets each function's __doc__ from get_tool_description(func.__name__) before registering it with p.tool, so the tool name is never retyped as a string and decorator syntax is preserved. Adds a regression test asserting every Parlant tool description matches its master source. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0114NCg97KJ6mNNK5RmvsbpH
Comments and docstrings state facts about the code, not ticket references. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0114NCg97KJ6mNNK5RmvsbpH
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0114NCg97KJ6mNNK5RmvsbpH
Two composed descriptions were factually wrong for how these Parlant wrappers actually behave: - band_send_message's appended mentions note said "not a list" right after the master text's "mentions array" phrase, reading as self-contradictory. Reworded to state the actual shape positively. - band_lookup_peers inherited the master model's claim that the tool "Returns dict with data list of peers and metadata" — this wrapper actually formats the result into a plain-text summary. Appended a correction so the LLM isn't told to expect fields that never arrive. Also trims the master docstring before appending extra text, so the composed description doesn't carry a doubled blank line, and hoists the remaining inline imports in the test file to module level. Replaces the drift guardrail test with a stronger one: it mutates the actual TOOL_MODELS docstrings and asserts the change reaches the Parlant tool description, rather than re-deriving the expected text through get_tool_description() (the function under test's own dependency) — a hand-written docstring that happened to match today's master text would have passed the old assertion. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0114NCg97KJ6mNNK5RmvsbpH
band_tool() (added earlier in this PR) fixes the tool-level docstring, but Parlant's own schema builder never reads a docstring's Args: section for per-argument text -- it only reads typing.Annotated[T, ToolParameterOptions(description=...)] on a parameter's type annotation. Plain str/int annotations produced zero description, silently -- every Parlant tool argument reached the LLM with nothing. Extend band_tool() to wrap each parameter's annotation with the matching master field's description before registering, skipping context (must stay ToolContext) and any parameter with no master description or no master-model counterpart. mentions gets a short per-argument correction (comma-separated string, not the master's list[str]), mirroring the existing tool-level extra_doc pattern. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019mZ9meinC1E4WPLzNjY2LE
amit-gazal-band
left a comment
There was a problem hiding this comment.
Reviewed the diff and ran it live: booted examples/parlant/01_basic_agent.py against a real Band platform with parlant 3.3.2 and drove it as a user through three turns (greet + band_get_participants; band_lookup_peers + band_add_participant; membership re-check). It works — the agent replies, completes multi-tool turns, and the add-participant really landed. tests/integrations/parlant/: 44 passed on this head.
Most of what I found on the previous head (4e32a5b2) — every parameter reaching the LLM with description=None, the lost @john/agent-name handle format on band_add_contact, the lost identifier guidance on band_add_participant, the lost either/or hints on the contact tools — is fixed by 4a0b5794. I re-dumped every constructed ToolEntry on this head and confirmed each one now carries real per-argument text. Nice fix, and the Annotated[...]-wrapping approach keeps the single-source-of-truth property intact.
One gap survives it, inline below. Everything else from my earlier pass is resolved.
One thing outside this diff, for a follow-up: examples/parlant/01_basic_agent.py:89 and :110 instruct the agent to call band_add_participant / band_remove_participant "with the name parameter" — both take identifier. In my live run the model dropped the argument once ([ToolCaller] Inference::Completion: Argument 'identifier' is missing) before recovering, so it costs a wasted inference round rather than breaking the turn. The restored identifier description makes it much less likely; the guidelines are still worth correcting.
Both tools take identifier, not name — the stale guideline text cost the model a wasted inference round recovering from a missing-argument error in live testing. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HxEomYWJ9jUUHrdqBsKYYb
…rlant's LLM sent_status/status was hand-duplicated as an identical inline Literal in two master models (ListContactRequestsInput, ListSentContactRequestsInput); factor it into one ContactRequestSentStatus alias, matching the existing EventMessageType precedent. Parlant's own schema builder crashes if handed a bare Literal[...] parameter type directly (it only turns a real enum.Enum into a JSON Schema enum), so band_list_contact_requests' sent_status parameter reached the LLM as a plain string with no enumerated choices. Fold the master field's Literal choices into the parameter description as prose instead — verified live against the real parlant SDK. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HxEomYWJ9jUUHrdqBsKYYb
|
Addressed both points from the review, pushed in two commits:
|
test_loopback_bind_auto_dns_rebinding_protection_accepts_real_clients hangs the full 30s pytest-timeout on windows-latest CI, stuck inside asyncio's Proactor _poll waiting on a completion that never arrives. This is a real, isolated bug in the spike's own throwaway _RunningApp helper, not in production code: LocalMCPServer.start() uses the identical raw-socket-to-uvicorn handoff (_reserve_socket + uvicorn.Server(sockets= [...])), and its own real-client tests (test_local_server.py's test_serves_sse_tools_on_localhost / test_serves_streamable_http_tools_ on_localhost) already exercise that exact path successfully on Windows CI today (verified against main via PR #545/#547's green windows runs) -- so the production path isn't broken, only this spike's copy of it. The spike's own docstring already says it's disposable once local_server.py lands ("this test either moves onto it or is deleted -- it is a feasibility gate, not permanent product code"). That already happened; this test's unique coverage (a real client isn't 421'd on loopback) is now fully redundant with test_local_server.py's tests against the real class, so delete it rather than debug a Windows Proactor quirk in disposable spike code. The sibling rejects_spoofed_host test stays -- it has no live-server equivalent elsewhere. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GfY2tLM1peE7YwhYDsMs1M
test_loopback_bind_auto_dns_rebinding_protection_accepts_real_clients hangs the full 30s pytest-timeout on windows-latest CI, stuck inside asyncio's Proactor _poll waiting on a completion that never arrives. This is a real, isolated bug in the spike's own throwaway _RunningApp helper, not in production code: LocalMCPServer.start() uses the identical raw-socket-to-uvicorn handoff (_reserve_socket + uvicorn.Server(sockets= [...])), and its own real-client tests (test_local_server.py's test_serves_sse_tools_on_localhost / test_serves_streamable_http_tools_ on_localhost) already exercise that exact path successfully on Windows CI today (verified against main via PR #545/#547's green windows runs) -- so the production path isn't broken, only this spike's copy of it. The spike's own docstring already says it's disposable once local_server.py lands ("this test either moves onto it or is deleted -- it is a feasibility gate, not permanent product code"). That already happened; this test's unique coverage (a real client isn't 421'd on loopback) is now fully redundant with test_local_server.py's tests against the real class, so delete it rather than debug a Windows Proactor quirk in disposable spike code. The sibling rejects_spoofed_host test stays -- it has no live-server equivalent elsewhere. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GfY2tLM1peE7YwhYDsMs1M
Summary
runtime/tools.py, so a master reword never reached Parlant agents — it had already drifted (stale unprefixedadd_participant(...)/remove_participant(...)example calls for tools actually namedband_add_participant/band_remove_participant).band_tooldecorator that sets each function's__doc__fromget_tool_description(func.__name__)before registering withp.tool— the tool name is never retyped as a string (it's justfunc.__name__, always written to match itsTOOL_MODELSentry), and decorator syntax is preserved.band_send_message(Parlant takes a comma-separated string; the master model describes a list).test_descriptions_match_master_source) asserting every Parlant tool description starts with its masterget_tool_description()text.Fixes INT-1170.
Test plan
uv run --extra dev-parlant pytest tests/integrations/parlant/test_tools.py -v --no-cov— 39 passed (real Parlant SDK, not mocked)uv run ruff check/uv run ruff format --checkuv run pyrefly check🤖 Generated with Claude Code
https://claude.ai/code/session_0114NCg97KJ6mNNK5RmvsbpH