Prompts that ask for tool invocation exemplar code cause 4xx errors

[shapirodanieladam]

Describe the bug

prompts that elicit responses which include tool invocation exemplar code cause goose to issue failed tool invocations, yielding a 4xx response and an unrecoverable session

To Reproduce
Steps to reproduce the behavior:

1. prompt goose with either of the following statements

how are tool invocation messages handled in goose?

what's the expected message format for tool invocations?

2. observe that the response yields content which is immediately parsed as a "real" tool invocation request, resulting in a 4xx error

Expected behavior

Goose can hold a multi-turn conversation about agentic tools without attempting to run exemplar tool code returned by the underlying model; sessions are not rendered unrecoverable

Screenshots

Please provide following information:

• OS & Arch: macos m3 max / 15.3.1 (24D70)
• Interface: ui
• Version: [Goose] Version 1.0.15-block.202503190039-808b0 (1.0.15-block.202503190039-808b0)
• Extensions enabled: Developer (only)
• Provider & Model: Claude Sonnet 3.5

Additional context

[ahau-square]

@shapirodanieladam taking a look at this, I think you may have actually triggered a weird edge cases error in the models themselves.

For example in the logs of a similar session with Goose, I'm seeing that claude 3.5 actually generated a bad tool call with a tool that isn't in its provided tools:

"choices": [
   {
     "finish_reason": "tool_calls",
     "index": 0,
     "message": {
       "content": "# Tool Invocation Format\n\nAs you can see from the above examples, tool invocations in Goose are handled with the following format:\n\n1. **XML-style Invocation Block**\n   ```xml\n   ",
       "role": "assistant",
       "tool_calls": [
         {
           "function": {
             "arguments": "{\"parameter_name\":\"parameter_value\"}",
             **"name": "extension__toolname"**
           },
           "id": "toolu_bdrk_01TUkQrUgV28i3pCKFiXQyeC",
           "type": "function"
         }
       ]
     }
   }
 ],

So Goose is getting that tool call and then trying to run it with no success. Also seeing that in subsequent generated messages from claude, it starts generating invalid markdown (which leads to those weird 'undefined' blocks.

So I actually don't think this is a Goose issue as it's rendering the generated outputs correctly, it's just that those outputs are malformed.

[shapirodanieladam]

So Goose is getting that tool call and then trying to run it with no success. Also seeing that in subsequent generated messages from claude, it starts generating invalid markdown (which leads to those weird 'undefined' blocks.

So I actually don't think this is a Goose issue as it's rendering the generated outputs correctly, it's just that those outputs are malformed.

fascinating, thanks for the deep dive. why is goose attempting to run invalid tool calls, rather than failing gracefully?

[DOsinga]

we've changed our tool calling now, and this looks like fixed on main. thanks for the research