Skip to content

fix: Parlant tool descriptions drift from SDK master models - #545

Merged
AlexanderZ-Band merged 7 commits into
mainfrom
fix/parlant-tool-descriptions-are-hand-written-and-dri-INT-1170
Aug 18, 2026
Merged

fix: Parlant tool descriptions drift from SDK master models#545
AlexanderZ-Band merged 7 commits into
mainfrom
fix/parlant-tool-descriptions-are-hand-written-and-dri-INT-1170

Conversation

@AlexanderZ-Band

Copy link
Copy Markdown
Collaborator

Summary

  • Parlant hand-wrote its 12 tool docstrings instead of reading the master Pydantic models in runtime/tools.py, so a master reword never reached Parlant agents — it had already drifted (stale unprefixed add_participant(...)/remove_participant(...) example calls for tools actually named band_add_participant/band_remove_participant).
  • Adds a band_tool decorator that sets each function's __doc__ from get_tool_description(func.__name__) before registering with p.tool — the tool name is never retyped as a string (it's just func.__name__, always written to match its TOOL_MODELS entry), and decorator syntax is preserved.
  • Only genuine Parlant-specific addition kept: a mentions-format note on band_send_message (Parlant takes a comma-separated string; the master model describes a list).
  • Adds a regression test (test_descriptions_match_master_source) asserting every Parlant tool description starts with its master get_tool_description() text.

Fixes INT-1170.

Test plan

  • uv run --extra dev-parlant pytest tests/integrations/parlant/test_tools.py -v --no-cov — 39 passed (real Parlant SDK, not mocked)
  • uv run ruff check / uv run ruff format --check
  • uv run pyrefly check

🤖 Generated with Claude Code

https://claude.ai/code/session_0114NCg97KJ6mNNK5RmvsbpH

Parlant hand-wrote its tool docstrings instead of reading the master
Pydantic models in runtime/tools.py, so a reword never reached Parlant
agents and the docs drifted (stale unprefixed add_participant/
remove_participant example calls for tools actually named
band_add_participant/band_remove_participant).

Introduces a band_tool decorator that sets each function's __doc__ from
get_tool_description(func.__name__) before registering it with p.tool,
so the tool name is never retyped as a string and decorator syntax is
preserved. Adds a regression test asserting every Parlant tool
description matches its master source.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0114NCg97KJ6mNNK5RmvsbpH
@linear-code

linear-code Bot commented Aug 16, 2026

Copy link
Copy Markdown

INT-1170

AlexanderZ-Band and others added 3 commits August 16, 2026 22:44
Comments and docstrings state facts about the code, not ticket references.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0114NCg97KJ6mNNK5RmvsbpH
Two composed descriptions were factually wrong for how these Parlant
wrappers actually behave:

- band_send_message's appended mentions note said "not a list" right
  after the master text's "mentions array" phrase, reading as
  self-contradictory. Reworded to state the actual shape positively.
- band_lookup_peers inherited the master model's claim that the tool
  "Returns dict with data list of peers and metadata" — this wrapper
  actually formats the result into a plain-text summary. Appended a
  correction so the LLM isn't told to expect fields that never arrive.

Also trims the master docstring before appending extra text, so the
composed description doesn't carry a doubled blank line, and hoists
the remaining inline imports in the test file to module level.

Replaces the drift guardrail test with a stronger one: it mutates the
actual TOOL_MODELS docstrings and asserts the change reaches the
Parlant tool description, rather than re-deriving the expected text
through get_tool_description() (the function under test's own
dependency) — a hand-written docstring that happened to match today's
master text would have passed the old assertion.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0114NCg97KJ6mNNK5RmvsbpH
band_tool() (added earlier in this PR) fixes the tool-level docstring, but
Parlant's own schema builder never reads a docstring's Args: section for
per-argument text -- it only reads typing.Annotated[T,
ToolParameterOptions(description=...)] on a parameter's type annotation.
Plain str/int annotations produced zero description, silently -- every
Parlant tool argument reached the LLM with nothing.

Extend band_tool() to wrap each parameter's annotation with the matching
master field's description before registering, skipping context (must stay
ToolContext) and any parameter with no master description or no
master-model counterpart. mentions gets a short per-argument correction
(comma-separated string, not the master's list[str]), mirroring the
existing tool-level extra_doc pattern.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mZ9meinC1E4WPLzNjY2LE

@amit-gazal-band amit-gazal-band left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the diff and ran it live: booted examples/parlant/01_basic_agent.py against a real Band platform with parlant 3.3.2 and drove it as a user through three turns (greet + band_get_participants; band_lookup_peers + band_add_participant; membership re-check). It works — the agent replies, completes multi-tool turns, and the add-participant really landed. tests/integrations/parlant/: 44 passed on this head.

Most of what I found on the previous head (4e32a5b2) — every parameter reaching the LLM with description=None, the lost @john/agent-name handle format on band_add_contact, the lost identifier guidance on band_add_participant, the lost either/or hints on the contact tools — is fixed by 4a0b5794. I re-dumped every constructed ToolEntry on this head and confirmed each one now carries real per-argument text. Nice fix, and the Annotated[...]-wrapping approach keeps the single-source-of-truth property intact.

One gap survives it, inline below. Everything else from my earlier pass is resolved.

One thing outside this diff, for a follow-up: examples/parlant/01_basic_agent.py:89 and :110 instruct the agent to call band_add_participant / band_remove_participant "with the name parameter" — both take identifier. In my live run the model dropped the argument once ([ToolCaller] Inference::Completion: Argument 'identifier' is missing) before recovering, so it costs a wasted inference round rather than breaking the turn. The restored identifier description makes it much less likely; the guidelines are still worth correcting.

Comment thread src/band/integrations/parlant/tools.py
AlexanderZ-Band and others added 2 commits August 18, 2026 12:18
Both tools take identifier, not name — the stale guideline text cost
the model a wasted inference round recovering from a missing-argument
error in live testing.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxEomYWJ9jUUHrdqBsKYYb
…rlant's LLM

sent_status/status was hand-duplicated as an identical inline Literal
in two master models (ListContactRequestsInput, ListSentContactRequestsInput);
factor it into one ContactRequestSentStatus alias, matching the existing
EventMessageType precedent.

Parlant's own schema builder crashes if handed a bare Literal[...] parameter
type directly (it only turns a real enum.Enum into a JSON Schema enum), so
band_list_contact_requests' sent_status parameter reached the LLM as a plain
string with no enumerated choices. Fold the master field's Literal choices
into the parameter description as prose instead — verified live against the
real parlant SDK.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxEomYWJ9jUUHrdqBsKYYb
@AlexanderZ-Band

Copy link
Copy Markdown
Collaborator Author

Addressed both points from the review, pushed in two commits:

examples/parlant/01_basic_agent.py:89/:110 (the "outside this diff" follow-up): fixed — both guidelines now say identifier instead of name, matching AddParticipantInput/RemoveParticipantInput's actual field. (a38c9b3)

sent_status inline comment: replied inline with the fix + why a bare Literal doesn't work here (parlant's schema builder only special-cases real enum.Enum, and crashes on Literal[...] — verified live). Also deduplicated the master's own Literal["pending", "approved", "rejected", "cancelled", "all"], which was independently hand-typed in two places (ListContactRequestsInput and ListSentContactRequestsInput) — one ContactRequestSentStatus alias now, matching the existing EventMessageType pattern. (048ac46)

tests/integrations/parlant/: 46 passed (44 + 1 new regression test) on dev-parlant; tests/runtime/, tests/core/, and the rest of the default-venv suite (4499 passed) unaffected. ruff check/format and pyrefly check clean on the touched files.

@AlexanderZ-Band
AlexanderZ-Band added this pull request to the merge queue Aug 18, 2026
Merged via the queue into main with commit d2876b7 Aug 18, 2026
20 checks passed
@AlexanderZ-Band
AlexanderZ-Band deleted the fix/parlant-tool-descriptions-are-hand-written-and-dri-INT-1170 branch August 18, 2026 10:18
AlexanderZ-Band added a commit that referenced this pull request Aug 18, 2026
test_loopback_bind_auto_dns_rebinding_protection_accepts_real_clients
hangs the full 30s pytest-timeout on windows-latest CI, stuck inside
asyncio's Proactor _poll waiting on a completion that never arrives.

This is a real, isolated bug in the spike's own throwaway _RunningApp
helper, not in production code: LocalMCPServer.start() uses the identical
raw-socket-to-uvicorn handoff (_reserve_socket + uvicorn.Server(sockets=
[...])), and its own real-client tests (test_local_server.py's
test_serves_sse_tools_on_localhost / test_serves_streamable_http_tools_
on_localhost) already exercise that exact path successfully on Windows CI
today (verified against main via PR #545/#547's green windows runs) --
so the production path isn't broken, only this spike's copy of it.

The spike's own docstring already says it's disposable once local_server.py
lands ("this test either moves onto it or is deleted -- it is a feasibility
gate, not permanent product code"). That already happened; this test's
unique coverage (a real client isn't 421'd on loopback) is now fully
redundant with test_local_server.py's tests against the real class, so
delete it rather than debug a Windows Proactor quirk in disposable spike
code. The sibling rejects_spoofed_host test stays -- it has no live-server
equivalent elsewhere.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GfY2tLM1peE7YwhYDsMs1M
AlexanderZ-Band added a commit that referenced this pull request Aug 18, 2026
test_loopback_bind_auto_dns_rebinding_protection_accepts_real_clients
hangs the full 30s pytest-timeout on windows-latest CI, stuck inside
asyncio's Proactor _poll waiting on a completion that never arrives.

This is a real, isolated bug in the spike's own throwaway _RunningApp
helper, not in production code: LocalMCPServer.start() uses the identical
raw-socket-to-uvicorn handoff (_reserve_socket + uvicorn.Server(sockets=
[...])), and its own real-client tests (test_local_server.py's
test_serves_sse_tools_on_localhost / test_serves_streamable_http_tools_
on_localhost) already exercise that exact path successfully on Windows CI
today (verified against main via PR #545/#547's green windows runs) --
so the production path isn't broken, only this spike's copy of it.

The spike's own docstring already says it's disposable once local_server.py
lands ("this test either moves onto it or is deleted -- it is a feasibility
gate, not permanent product code"). That already happened; this test's
unique coverage (a real client isn't 421'd on loopback) is now fully
redundant with test_local_server.py's tests against the real class, so
delete it rather than debug a Windows Proactor quirk in disposable spike
code. The sibling rejects_spoofed_host test stays -- it has no live-server
equivalent elsewhere.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GfY2tLM1peE7YwhYDsMs1M
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants