fix(pipeline): inject built-in transfer_call / end_call tools into LLMLoop#115
Merged
nicolotognoni merged 1 commit intoMay 27, 2026
Merged
Conversation
…MLoop
Pipeline mode (STT → LLM → TTS) was constructing ``LLMLoop`` with only
the user-provided tools (``tools=self.agent.tools``), so the LLM never
saw ``transfer_call`` or ``end_call`` even though they're documented
as built-in across all three modes (Realtime, ConvAI, Pipeline).
The realtime path at ``stream_handler.py:997`` already injects them:
openai_tools = agent_tools + [TRANSFER_CALL_TOOL, END_CALL_TOOL]
Mirror that here with a new helper ``_augment_with_builtin_handoff_tools``
that builds handler closures wired to the telephony-level
``_transfer_fn`` / ``_hangup_fn`` callbacks already attached to the
``PipelineStreamHandler``. The handler signature ``(arguments,
call_context)`` matches ``ToolExecutor._invoke_handler``'s calling
convention.
Tools are appended (not prepended) so user-provided tool order is
preserved. Built-ins are skipped when the corresponding telephony fn
is unavailable (e.g. test harness without a Twilio adapter).
Includes 6 unit tests covering empty-user-tools / non-empty-user-tools /
missing-fn / handler-dispatch / graceful-empty-args paths.
Closes PatterAI#110.
6 tasks
nicolotognoni
added a commit
that referenced
this pull request
May 27, 2026
…115) (#118) - Pipeline mode in `libraries/typescript/src/stream-handler.ts` did not inject the built-in `transfer_call` / `end_call` tools into the `LLMLoop` (both `new LLMLoop(...)` call sites at lines 1891 and 1906 passed `agent.tools` through unchanged). The Realtime path injects them at `server.ts:374`; pipeline was missing the parity. Added `augmentWithBuiltinHandoffTools` helper mirroring the Python helper shipped in #115; it builds handler closures that validate E.164 / default `reason` and dispatch to the existing telephony bridge methods (`bridge.transferCall` / `bridge.endCall`). Built-ins are skipped when the corresponding callback is missing, keeping non-telephony test harnesses clean. Exported `TRANSFER_CALL_TOOL` / `END_CALL_TOOL` from `server.ts` so the helper can re-use the canonical schema. - Docs: `docs/typescript-sdk/events.mdx` advertised the same non-existent `phone.events.on(PatterEventType.X, handler)` API as the Python events page (closed by #114). Replaced the broken `EventBus` section with documentation of the APIs that actually exist on the TypeScript `Patter` class: speech-edge attribute setters (`onUserSpeechStarted` / `onAgentSpeechEnded` / `onLlmToken` / `onAudioOut` etc.) and tool events via `onTranscript` with `role === "tool"`. - CHANGELOG: backfilled `## Unreleased` `### Fixed` entries for upstream PRs #113, #114, #115 (Python fixes by @knowsuchagency that landed without the corresponding TypeScript / changelog updates), plus entries for the two TypeScript parity fixes shipped here. Test plan: - Python: covered by tests added in the upstream PRs (#113, #115). - TypeScript: 8 new unit tests in `libraries/typescript/tests/pipeline-builtin-tools.test.ts` cover empty-user-tools, non-empty user tools order, missing-callback skip, transfer dispatch + E.164 rejection, end_call default and user-supplied `reason`. Full suite: 1521 passed / 8 skipped, lint clean, build green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PipelineStreamHandler→LLMLoop) was passing only the user-provided tools through, so the LLM never sawtransfer_callorend_call— even though the docs and theadd-tools-and-handoffsAgent Skill (PR docs: add Agent Skills bundle for AI agent harnesses #108) describe them as available across all three modes.agent_tools + [TRANSFER_CALL_TOOL, END_CALL_TOOL]injection (stream_handler.py:997) via a new helper_augment_with_builtin_handoff_toolsthat builds handler closures wired to the telephony-level_transfer_fn/_hangup_fnalready attached to the stream handler.tests/test_pipeline_builtin_tools.pycover empty-user-tools, non-empty-user-tools, missing-fn, handler-dispatch (transfer + hangup), and graceful-empty-args paths.Closes #110.
Why
Reproduced against
gpt-4o-miniin pipeline mode (OpenAILLM+DeepgramSTT+ElevenLabsRestTTS). The LLM correctly heard the affirmative ("Yeah. Go ahead and transfer me.") and replied "Alright, transferring you now." — but emitted no tool call. Twilio call events showed zeroPOST /Calls/{sid}.json. The tool simply wasn't registered.stream_handler.py:2422-2435(before this PR):vs realtime
stream_handler.py:997:Implementation notes
The handler shape inside
LLMLoopis(arguments_dict, call_context_dict)(seeToolExecutor._invoke_handlerattools/tool_executor.py:46-63). The helper builds two async closures with that exact signature and routes them toself._transfer_fn(number)/self._hangup_fn()which already exist onPipelineStreamHandler(assigned atstream_handler.py:1899-1900from the telephony adapter attelephony/twilio.py:483-484).Built-ins are skipped when the corresponding telephony fn is missing — keeps the unit-test harness path clean (no Twilio adapter → no transfer/hangup tools, matching the prior behaviour for non-telephony tests).
TypeScript parity
The TypeScript SDK's
libraries/typescript/src/server.tsalready injects the same two built-ins for both paths (agentTools + [TRANSFER_CALL_TOOL, END_CALL_TOOL]). Spot-checked against the source — TS is already correct; this PR brings Python to parity, not the other way around.Test plan
pytest libraries/python/tests/test_pipeline_builtin_tools.py -v— 6/6 passOpenAILLM(model="gpt-4o-mini")+DeepgramSTT+ElevenLabsRestTTSon Twilio: caller says "transfer me," LLM emitstransfer_call({"number": "+1..."}), helper dispatches to_twilio_transfer, Twilio call-update REST fires, GV number rings, audio bridges.