Skip to content

[aisdk] Pass correct CallID and args on computer use tool call #578

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Nov 10, 2025
Merged

[aisdk] Pass correct CallID and args on computer use tool call #578

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
8f015d2
[aisdk] Pass correct CallID and args on computer use tool call
ipince Nov 10, 2025
16ace8b
remove pending safety checks field in one test
ipince Nov 10, 2025
9211509
fix
ipince Nov 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions aisdk/ai/provider/openai/internal/codec/decode.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -187,9 +187,11 @@ func decodeComputerCall(computerCall responses.ResponseComputerToolCall) (*api.T

// Create tool call block with provider metadata
return &api.ToolCallBlock{
ToolCallID: computerCall.ID, // TODO: use computerCall.CallID instead
// The CallID is what's needed to respond to the tool call, not the "ID" field.
// https://platform.openai.com/docs/api-reference/responses/object#responses-object-output-computer_tool_call
ToolCallID: computerCall.CallID,
ToolName: ComputerUseToolID,
Args: json.RawMessage(computerCall.RawJSON()), // TODO: pass the Action instead.
Args: json.RawMessage(computerCall.Action.RawJSON()), // The Action is the only argument to the computer call.
ProviderMetadata: api.NewProviderMetadata(map[string]any{ProviderName: metadata}),
}, nil
}
Expand Down
34 changes: 19 additions & 15 deletions aisdk/ai/provider/openai/internal/codec/decode_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -640,22 +640,24 @@ func TestDecodeToolCalls(t *testing.T) {
},
},
{
name: "computer call",
name: "computer call with no safety checks field",
response: `{
"output": [
{
"type": "computer_call",
"id": "comp1",
"command": "ls -la",
"working_directory": "/tmp"
"call_id": "call_123",
"action": {
Copy link
Contributor

@loreto loreto Nov 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we are really polishing the computer call use case, it could be good to add test cases for all the possible actions open ai returns.

And maybe another test for pending safety checks (omitted, empty, and filled)

Copy link
Contributor Author

@ipince ipince Nov 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll do that in a separate PR

"type": "screenshot"
}
}
]
}`,
want: []api.ContentBlock{
&api.ToolCallBlock{
ToolCallID: "comp1",
ToolCallID: "call_123",
ToolName: "openai.computer_use_preview",
Args: json.RawMessage(`{"command":"ls -la","working_directory":"/tmp","id":"comp1","type":"computer_call"}`),
Args: json.RawMessage(`{"type":"screenshot"}`),
ProviderMetadata: api.NewProviderMetadata(map[string]any{
"openai": &Metadata{
ComputerSafetyChecks: []ComputerSafetyCheck{},
Expand All @@ -670,9 +672,10 @@ func TestDecodeToolCalls(t *testing.T) {
"output": [
{
"type": "computer_call",
"id": "comp2",
"command": "rm -rf /",
"working_directory": "/",
"call_id": "call_123",
"action": {
"type": "screenshot"
},
"pending_safety_checks": [
{
"id": "check1",
Expand All @@ -690,9 +693,9 @@ func TestDecodeToolCalls(t *testing.T) {
}`,
want: []api.ContentBlock{
&api.ToolCallBlock{
ToolCallID: "comp2",
ToolCallID: "call_123",
ToolName: "openai.computer_use_preview",
Args: json.RawMessage(`{"command":"rm -rf /","working_directory":"/","id":"comp2","type":"computer_call","pending_safety_checks":[{"id":"check1","code":"file_access","message":"File access requested"},{"id":"check2","code":"dangerous_command","message":"Potentially dangerous command detected"}]}`),
Args: json.RawMessage(`{"type":"screenshot"}`),
ProviderMetadata: api.NewProviderMetadata(map[string]any{
"openai": &Metadata{
ComputerSafetyChecks: []ComputerSafetyCheck{
Expand All @@ -718,18 +721,19 @@ func TestDecodeToolCalls(t *testing.T) {
"output": [
{
"type": "computer_call",
"id": "comp3",
"command": "echo hello",
"working_directory": "/tmp",
"call_id": "call_123",
"action": {
"type": "screenshot"
},
"pending_safety_checks": []
}
]
}`,
want: []api.ContentBlock{
&api.ToolCallBlock{
ToolCallID: "comp3",
ToolCallID: "call_123",
ToolName: "openai.computer_use_preview",
Args: json.RawMessage(`{"command":"echo hello","working_directory":"/tmp","id":"comp3","type":"computer_call","pending_safety_checks":[]}`),
Args: json.RawMessage(`{"type":"screenshot"}`),
ProviderMetadata: api.NewProviderMetadata(map[string]any{
"openai": &Metadata{
ComputerSafetyChecks: []ComputerSafetyCheck{},
Expand Down
Loading