Skip to content

feat: add client side tools to bridge and new cas events [JAR-9629]#1638

Open
norman-le wants to merge 11 commits into
mainfrom
feat/jar-9629-client-side-tools-cas-urt-v2
Open

feat: add client side tools to bridge and new cas events [JAR-9629]#1638
norman-le wants to merge 11 commits into
mainfrom
feat/jar-9629-client-side-tools-cas-urt-v2

Conversation

@norman-le
Copy link
Copy Markdown
Contributor

@norman-le norman-le commented May 18, 2026

Had to make new PR (old PR here: #1609)

Changes across repos for client-side tools feature:
uipath-python:
- Removed toolName from ExecutingToolCallEvent model. 
- Added client-side tool declaration model on exchange start. 
- Bridge gains emit_executing_tool_call method and wait_for_resume buffering for early responses.

uipath-runtime-python:
- Added emit_executing_tool_call to protocol. 
- Runtime loop emits executingToolCall after tool-call confirmations resume.

uipath-langchain-python:
- Client-side tool factory with @durable_interrupt. Names-only tool filtering via apply_tool_filter. 
- MessageMapper emits executingToolCall for all non-confirmed tools, adds isClientSideTool/outputSchema to startToolCall, suppresses endToolCall for client-side results. 
- Runtime discovers client-side tools through graph introspection. 
- Init node applies per-exchange filtering.

uipath-agents-python:
- Strips client-side tool metadata during evals.

uipath-dev-python:
- Protocol stub for emit_executing_tool_call.

AgentInterfaces:
- Schema updates (ExecutingToolCallEvent, ClientSideToolDeclaration, exchange start). 
- CAS passes client-side tool names to runtime input, forwards confirmToolCall to CLI bridge. 
- React SDK handles executingToolCall for client-side tools.

Videos in CAS localhost (Agent Builder changes already published, but this will be the debug experience as well when the sdk is upgraded):

Without tool confirmation for client side tool:

Screen.Recording.2026-05-13.at.9.32.26.PM.mov

With tool confirmation for client side tool:

Screen.Recording.2026-05-13.at.9.27.56.PM.mov

URT Eval:
image

Related to changes in other PRs:
CAS: https://github.com/UiPath/AgentInterfaces/pull/949
uipath-langchain-python (message mapping and firing executing tool call if no confirmation): UiPath/uipath-langchain-python#856
uipath-python (definitions): #1638
uipath-agents-python (skipping for evals, so it can be mocked): https://github.com/UiPath/uipath-agents-python/pull/485
uipath-runtime-python (protocol changing to emit executing tool call event after confirmation): UiPath/uipath-runtime-python#115
Another Agents PR fixing some issues with debug: https://github.com/UiPath/Agents/pull/5250

Copilot AI review requested due to automatic review settings May 18, 2026 15:09
@github-actions github-actions Bot added test:uipath-langchain Triggers tests in the uipath-langchain-python repository test:uipath-runtime test:uipath-integrations labels May 18, 2026
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds client-side tool support to the Python SDK bridge and conversation models, including new tool-call event schemas and agent model parsing for ClientSide tools.

Changes:

  • Adds ClientSide agent tool resource parsing and tests.
  • Adds executingToolCall, client-side tool declaration, and tool schema fields to chat event models.
  • Updates the SocketIO chat bridge to emit execution-phase tool events and resume from client-originated endToolCall.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
packages/uipath/src/uipath/_cli/_chat/_bridge.py Emits executingToolCall and accepts endToolCall as a resume signal.
packages/uipath/src/uipath/agent/models/agent.py Adds ClientSide tool type and resource config support.
packages/uipath-core/src/uipath/core/chat/tool.py Adds client-side/tool execution fields and executingToolCall schema.
packages/uipath-core/src/uipath/core/chat/exchange.py Adds client-side tool declarations to exchange start events.
packages/uipath-core/src/uipath/core/chat/__init__.py Exports the new executing tool-call event model.
packages/uipath/tests/cli/chat/test_bridge.py Adds bridge tests for execution-phase events and resume from tool events.
packages/uipath/tests/agent/models/test_agent.py Adds agent model tests for client-side tool resources.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread packages/uipath-core/src/uipath/core/chat/__init__.py
Comment thread packages/uipath/src/uipath/_cli/_chat/_bridge.py Outdated
Comment thread packages/uipath/src/uipath/_cli/_chat/_bridge.py Outdated
Comment thread packages/uipath/src/uipath/_cli/_chat/_bridge.py Outdated
@github-actions
Copy link
Copy Markdown

🚨 Heads up: uipath-langchain cross-tests are FAILING 🚨

Your changes may break the uipath-langchain-python integration.

⚠️ These checks are NOT enforced by branch protection rules. Please review the failures before merging.

🔍 Inspect the failed run →

@sonarqubecloud
Copy link
Copy Markdown

mode="python", by_alias=False
)
if value:
return value.model_dump(mode="python", by_alias=False)
Copy link
Copy Markdown
Contributor

@maxduu maxduu May 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious why we have the check for None, and then a clear when its None. Shouldn't the overall logic be identical to before, only difference being some renames because it can be either a confirmation-event or end-event? I might be missing something though

Copy link
Copy Markdown
Contributor Author

@norman-le norman-le May 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Responded in the thread but posting here too:

wait_for_resume is called after the graph suspends, but executingToolCall was sent earlier during streaming (during the mapper). The client receives it, executes, and could send endToolCall back before wait_for_resume is called. The guard checks if a response already arrived and skips the wait. Without it, the response would be cleared and wait_for_resume hangs. Unlikely in practice (network latency helps), but technically possible

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha - cool, I'm good for this but I think let's let @JoshParkSJ give final approval on this repo's PR since it mainly touches tool-confirmation logic

)
await self.emit_message_event(executing_event)

async def wait_for_resume(self) -> dict[str, Any]:
Copy link
Copy Markdown
Contributor

@maxduu maxduu May 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we can make the return type, our core types UiPathConversationToolCallConfirmationEvent | UiPathConversationToolCallEndEvent for stronger typing?

Copy link
Copy Markdown
Contributor

@maxduu maxduu May 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see, those types are BaseModels so might not be possible due to the model_dump. Could be worth a quick Claude check though, not a blocker

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test:uipath-integrations test:uipath-langchain Triggers tests in the uipath-langchain-python repository test:uipath-runtime

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants