Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Anthropic tool support via Vertex AI #4493

Open
3 tasks done
gboston opened this issue Mar 5, 2025 · 0 comments
Open
3 tasks done

Anthropic tool support via Vertex AI #4493

gboston opened this issue Mar 5, 2025 · 0 comments
Assignees
@RomneyDa
Labels
area:chat Relates to chat interface ide:vscode Relates specifically to VS Code extension kind:bug Indicates an unexpected problem or unintended behavior "needs-triage"

Comments

@gboston
Copy link

gboston commented Mar 5, 2025

Before submitting your bug report

Relevant environment info

- OS: mac
- Continue version:1.0.2
- IDE version: vscode 1.97.2 (Universal)
- Model: claude-3-7-sonnet@20250219 via LiteLLM vertex AI
- config:
  
{
      "provider": "openai",
      "model": "claude-3-7-sonnet",
      "title": "Claude 3.7 (litellm)",
      "capabilities": {
        "tools": true
      },
      "apiBase": "https://xxxx",
      "apiKey": "sk-xxxx",
      "systemMessage": "You are an expert software developer. You give helpful and concise responses."
    }
  
  OR link to assistant in Continue hub:

Description

When calling Claude via LiteLLM I get the following error on LiteLLM:

litellm.llms.anthropic.common_utils.AnthropicError: b'{"type":"error","error":{"type":"invalid_request_error","message":"messages.0: tool_result block(s) provided when previous message does not contain any tool_use blocks"}}'

similar issue was raised here: https://gitlab.com/gitlab-org/duo-workflow/duo-workflow-service/-/merge_requests/59

Don't know if this is caused by LiteLLM or continue extension

To reproduce

  1. Chat with model with tools enabled
  2. let it run for 1-2 tool calls
  3. 400 exception is thrown, likely caused by errors in tool calls not being gracefully handled

Log output

$onExtensionRuntimeError @ mainThreadExtensionService.ts:79
log.ts:456   ERR An unknown error occurred. Please consult the log for more details.
error @ log.ts:456
localProcessExtensionHost.ts:275 Extension Host
localProcessExtensionHost.ts:276 INFO:mcp.server.lowlevel.server:Processing request of type CallToolRequest
localProcessExtensionHost.ts:275 Extension Host
localProcessExtensionHost.ts:276 ERROR:mcp-atlassian:Search failed with error: Current user not permitted to use Confluence
localProcessExtensionHost.ts:275 Extension Host
localProcessExtensionHost.ts:276 INFO:mcp.server.lowlevel.server:Processing request of type CallToolRequest
localProcessExtensionHost.ts:275 Extension Host
localProcessExtensionHost.ts:276 INFO:mcp.server.lowlevel.server:Processing request of type CallToolRequest
localProcessExtensionHost.ts:275 Extension Host
localProcessExtensionHost.ts:276 ERROR:mcp-atlassian:Search failed with error: Current user not permitted to use Confluence
localProcessExtensionHost.ts:275 Extension Host
localProcessExtensionHost.ts:276 INFO:mcp.server.lowlevel.server:Processing request of type CallToolRequest
log.ts:456   ERR [Extension Host] Error handling webview message: {
  "msg": {
    "messageId": "0144df62-73bd-4b1b-87ce-6b3a402a1fb0",
    "messageType": "tools/call",
    "data": {
      "toolCall": {
        "id": "toolu_vrtx_01TbQrgedecfJJ5oUSvb7nU2",
        "type": "function",
        "function": {
          "name": "jira_search",
          "arguments": "{\"jql\": \"project = \\\"chatitp\\\" OR summary ~ \\\"chatitp\\\" OR description ~ \\\"chatitp\\\"\"}"
        }
      },
      "selectedModelTitle": "Claude 3.7 (litellm)"
    }
  }
}

Error: Failed to call tool: jira_search
error @ log.ts:456
console.ts:137 [Extension Host] Error handling webview message: {
  "msg": {
    "messageId": "0144df62-73bd-4b1b-87ce-6b3a402a1fb0",
    "messageType": "tools/call",
    "data": {
      "toolCall": {
        "id": "toolu_vrtx_01TbQrgedecfJJ5oUSvb7nU2",
        "type": "function",
        "function": {
          "name": "jira_search",
          "arguments": "{\"jql\": \"project = \\\"chatitp\\\" OR summary ~ \\\"chatitp\\\" OR description ~ \\\"chatitp\\\"\"}"
        }
      },
      "selectedModelTitle": "Claude 3.7 (litellm)"
    }
  }
}

Error: Failed to call tool: jira_search
Ats @ console.ts:137
localProcessExtensionHost.ts:275 Extension Host
localProcessExtensionHost.ts:276 ERROR:mcp-jira:Error searching issues with JQL project = "chatitp" OR summary ~ "chatitp" OR description ~ "chatitp": The value 'chatitp' does not exist for the field 'project'.ERROR:mcp-atlassian:Tool execution error: The value 'chatitp' does not exist for the field 'project'.
console.ts:137 [Extension Host] rejected promise not handled within 1 second: Error: Failed to resolve env "/Users/gbostoen/projects/melexis/xtdenv/.venv"
Ats @ console.ts:137
console.ts:137 [Extension Host] stack trace: Error: Failed to resolve env "/Users/gbostoen/projects/melexis/xtdenv/.venv"
	at ae (/Users/gbostoen/.vscode/extensions/ms-python.python-2025.0.0-darwin-arm64/out/client/extension.js:2:1973990)
	at oe (/Users/gbostoen/.vscode/extensions/ms-python.python-2025.0.0-darwin-arm64/out/client/extension.js:2:1971950)
	at Immediate.<anonymous> (/Users/gbostoen/.vscode/extensions/ms-python.python-2025.0.0-darwin-arm64/out/client/extension.js:2:1968244)
	at process.processImmediate (node:internal/timers:483:21)
Ats @ console.ts:137
$onExtensionRuntimeError @ mainThreadExtensionService.ts:78
mainThreadExtensionService.ts:79 Error: Failed to resolve env "/Users/gbostoen/projects/melexis/xtdenv/.venv"
	at ae (/Users/gbostoen/.vscode/extensions/ms-python.python-2025.0.0-darwin-arm64/out/client/extension.js:2:1973990)
	at oe (/Users/gbostoen/.vscode/extensions/ms-python.python-2025.0.0-darwin-arm64/out/client/extension.js:2:1971950)
	at Immediate.<anonymous> (/Users/gbostoen/.vscode/extensions/ms-python.python-2025.0.0-darwin-arm64/out/client/extension.js:2:1968244)
	at process.processImmediate (node:internal/timers:483:21)
$onExtensionRuntimeError @ mainThreadExtensionService.ts:79
log.ts:456   ERR An unknown error occurred. Please consult the log for more details.
error @ log.ts:456
console.ts:137 [Extension Host] Error: 400 litellm.BadRequestError: VertexAIException BadRequestError - b'{"type":"error","error":{"type":"invalid_request_error","message":"messages.10: Did not find 1 `tool_result` block(s) at the beginning of this message. Messages following `tool_use` blocks must begin with a matching number of `tool_result` blocks."}}'No fallback model group found for original model_group=claude-3-7-sonnet. Fallbacks=[{'gpt-4o': ['gemini-2.0-flash-001']}]. Received Model Group=claude-3-7-sonnet
Available Model Group Fallbacks=None
Error doing the fallback: litellm.BadRequestError: VertexAIException BadRequestError - b'{"type":"error","error":{"type":"invalid_request_error","message":"messages.10: Did not find 1 `tool_result` block(s) at the beginning of this message. Messages following `tool_use` blocks must begin with a matching number of `tool_result` blocks."}}'No fallback model group found for original model_group=claude-3-7-sonnet. Fallbacks=[{'gpt-4o': ['gemini-2.0-flash-001']}] LiteLLM Retried: 1 times, LiteLLM Max Retries: 2
	at APIError.generate (/Users/gbostoen/.vscode/extensions/continue.continue-1.0.2-darwin-arm64/out/extension.js:92866:18)
	at OpenAI.makeStatusError (/Users/gbostoen/.vscode/extensions/continue.continue-1.0.2-darwin-arm64/out/extension.js:93777:25)
	at OpenAI.makeRequest (/Users/gbostoen/.vscode/extensions/continue.continue-1.0.2-darwin-arm64/out/extension.js:93821:29)
	at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
	at async OpenAIApi.chatCompletionStream (/Users/gbostoen/.vscode/extensions/continue.continue-1.0.2-darwin-arm64/out/extension.js:97865:26)
	at async OpenAI2.streamChat (/Users/gbostoen/.vscode/extensions/continue.continue-1.0.2-darwin-arm64/out/extension.js:122440:34)
	at async llmStreamChat (/Users/gbostoen/.vscode/extensions/continue.continue-1.0.2-darwin-arm64/out/extension.js:606724:23)
	at async Ah.handleMessage [as value] (/Users/gbostoen/.vscode/extensions/continue.continue-1.0.2-darwin-arm64/out/extension.js:619097:29)
log.ts:456   ERR [Extension Host] Error handling webview message: {
  "msg": {
    "messageId": "25166fe4-32f7-4801-a883-e917bf69d10f",
    "messageType": "llm/streamChat",
    "data": {
      "messages": [
        {
          "role": "system",
          "content": "Always include the language and file name in the info string when you write code blocks, for example ' file.py'.\n\nWhen using tools, follow the following guidelines:\n- Avoid calling tools unless they are absolutely necessary. For example, if you are asked a simple programming question you do not need web search. As another example, if the user asks you to explain something about code, do not create a new file."
        },
        {
          "role": "user",
          "content": [
            {
              "type": "text",
              "text": "are there any findings on confluence or jira?"
            }
          ]
        },
        {
          "role": "assistant",
          "content": "I'd be happy to help you search for findings in Confluence and Jira. To provide you with relevant information, I need to search these systems. Let me do that for you.\n\nLet's start by searching Confluence:"
        },
        {
          "role": "assistant",
          "content": "",
          "toolCalls": [
            {
              "id": "toolu_vrtx_01HLhVaaaMhxZQbFywMN96Lb",
              "type": "function",
              "function": {
                "name": "confluence_search",
                "arguments": "{\"query\": \"type=page AND title ~ \\\"findings\\\"\"}"
              }
            }
          ]
        },
        {
          "role": "tool",
          "content": "[]",
          "toolCallId": "toolu_vrtx_01HLhVaaaMhxZQbFywMN96Lb"
        },
        {
          "role": "assistant",
          "content": "\n\nNow let's search Jira for issues that might contain findings:"
        },
        {
          "role": "assistant",
          "content": "",
          "toolCalls": [
            {
              "id": "toolu_vrtx_011oTcjAPgyaz5E5QzPHHxRa",
              "type": "function",
              "function": {
                "name": "jira_search",
                "arguments": "{\"jql\": \"summary ~ \\\"findings\\\" OR description ~ \\\"findings\\\"\"}"
              }
            }
          ]
        },
        {
          "role": "tool",
          "content": "[]",
          "toolCallId": "toolu_vrtx_011oTcjAPgyaz5E5QzPHHxRa"
        },
        {
          "role": "assistant",
          "content": "\n\nI've searched both Confluence and Jira for content related to \"findings\" but didn't find any results. \n\nWould you like me to try with different search terms? For example, I could search for terms like \"report\", \"issues\", \"bugs\", \"audit\", or any other specific terms that might relate to the findings you're looking for. \n\nOr, if you have more specific information like a project key in Jira or a space name in Confluence, that would help narrow down the search."
        },
        {
          "role": "user",
          "content": [
            {
              "type": "text",
              "text": "It's chatitp"
            }
          ]
        },
        {
          "role": "assistant",
          "content": "Thank you for the clarification. Let me search for findings related to \"chatitp\" in both Confluence and Jira."
        },
        {
          "role": "assistant",
          "content": "",
          "toolCalls": [
            {
              "id": "toolu_vrtx_01MQsnh9E4ZPeqvozTnV3P1A",
              "type": "function",
              "function": {
                "name": "confluence_search",
                "arguments": "{\"query\": \"type=page AND (title ~ \\\"chatitp\\\" OR text ~ \\\"chatitp\\\")\"}"
              }
            }
          ]
        },
        {
          "role": "tool",
          "content": "[]",
          "toolCallId": "toolu_vrtx_01MQsnh9E4ZPeqvozTnV3P1A"
        },
        {
          "role": "assistant",
          "content": "\n\nNow let's check Jira for issues related to \"chatitp\":"
        },
        {
          "role": "assistant",
          "content": "",
          "toolCalls": [
            {
              "id": "toolu_vrtx_01TbQrgedecfJJ5oUSvb7nU2",
              "type": "function",
              "function": {
                "name": "jira_search",
                "arguments": "{\"jql\": \"project = \\\"chatitp\\\" OR summary ~ \\\"chatitp\\\" OR description ~ \\\"chatitp\\\"\"}"
              }
            }
          ]
        },
        {
          "role": "user",
          "content": [
            {
              "type": "text",
              "text": "I got the following error, can you please help explain how to fix it?\n\n"
            }
          ]
        },
        {
          "role": "assistant",
          "content": ""
        }
      ],
      "title": "Claude 3.7 (litellm)",
      "completionOptions": {
        "tools": [
          {
            "displayTitle": "MCP Server confluence_search",
           
error @ log.ts:456
console.ts:137 [Extension Host] Error handling webview message: {
  "msg": {
    "messageId": "25166fe4-32f7-4801-a883-e917bf69d10f",
    "messageType": "llm/streamChat",
    "data": {
      "messages": [
        {
          "role": "system",
          "content": "Always include the language and file name in the info string when you write code blocks, for example ' file.py'.\n\nWhen using tools, follow the following guidelines:\n- Avoid calling tools unless they are absolutely necessary. For example, if you are asked a simple programming question you do not need web search. As another example, if the user asks you to explain something about code, do not create a new file."
        },
        {
          "role": "user",
          "content": [
            {
              "type": "text",
              "text": "are there any findings on confluence or jira?"
            }
          ]
        },
        {
          "role": "assistant",
          "content": "I'd be happy to help you search for findings in Confluence and Jira. To provide you with relevant information, I need to search these systems. Let me do that for you.\n\nLet's start by searching Confluence:"
        },
        {
          "role": "assistant",
          "content": "",
          "toolCalls": [
            {
              "id": "toolu_vrtx_01HLhVaaaMhxZQbFywMN96Lb",
              "type": "function",
              "function": {
                "name": "confluence_search",
                "arguments": "{\"query\": \"type=page AND title ~ \\\"findings\\\"\"}"
              }
            }
          ]
        },
        {
          "role": "tool",
          "content": "[]",
          "toolCallId": "toolu_vrtx_01HLhVaaaMhxZQbFywMN96Lb"
        },
        {
          "role": "assistant",
          "content": "\n\nNow let's search Jira for issues that might contain findings:"
        },
        {
          "role": "assistant",
          "content": "",
          "toolCalls": [
            {
              "id": "toolu_vrtx_011oTcjAPgyaz5E5QzPHHxRa",
              "type": "function",
              "function": {
                "name": "jira_search",
                "arguments": "{\"jql\": \"summary ~ \\\"findings\\\" OR description ~ \\\"findings\\\"\"}"
              }
            }
          ]
        },
        {
          "role": "tool",
          "content": "[]",
          "toolCallId": "toolu_vrtx_011oTcjAPgyaz5E5QzPHHxRa"
        },
        {
          "role": "assistant",
          "content": "\n\nI've searched both Confluence and Jira for content related to \"findings\" but didn't find any results. \n\nWould you like me to try with different search terms? For example, I could search for terms like \"report\", \"issues\", \"bugs\", \"audit\", or any other specific terms that might relate to the findings you're looking for. \n\nOr, if you have more specific information like a project key in Jira or a space name in Confluence, that would help narrow down the search."
        },
        {
          "role": "user",
          "content": [
            {
              "type": "text",
              "text": "It's chatitp"
            }
          ]
        },
        {
          "role": "assistant",
          "content": "Thank you for the clarification. Let me search for findings related to \"chatitp\" in both Confluence and Jira."
        },
        {
          "role": "assistant",
          "content": "",
          "toolCalls": [
            {
              "id": "toolu_vrtx_01MQsnh9E4ZPeqvozTnV3P1A",
              "type": "function",
              "function": {
                "name": "confluence_search",
                "arguments": "{\"query\": \"type=page AND (title ~ \\\"chatitp\\\" OR text ~ \\\"chatitp\\\")\"}"
              }
            }
          ]
        },
        {
          "role": "tool",
          "content": "[]",
          "toolCallId": "toolu_vrtx_01MQsnh9E4ZPeqvozTnV3P1A"
        },
        {
          "role": "assistant",
          "content": "\n\nNow let's check Jira for issues related to \"chatitp\":"
        },
        {
          "role": "assistant",
          "content": "",
          "toolCalls": [
            {
              "id": "toolu_vrtx_01TbQrgedecfJJ5oUSvb7nU2",
              "type": "function",
              "function": {
                "name": "jira_search",
                "arguments": "{\"jql\": \"project = \\\"chatitp\\\" OR summary ~ \\\"chatitp\\\" OR description ~ \\\"chatitp\\\"\"}"
              }
            }
          ]
        },
        {
          "role": "user",
          "content": [
            {
              "type": "text",
              "text": "I got the following error, can you please help explain how to fix it?\n\n"
            }
          ]
        },
        {
          "role": "assistant",
          "content": ""
        }
      ],
      "title": "Claude 3.7 (litellm)",
      "completionOptions": {
        "tools": [
          {
            "displayTitle": "MCP Server confluence_search",
            "function": {
  
Ats @ console.ts:137
@dosubot dosubot bot added area:chat Relates to chat interface ide:vscode Relates specifically to VS Code extension kind:bug Indicates an unexpected problem or unintended behavior labels Mar 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@RomneyDa RomneyDa

Labels
area:chat Relates to chat interface ide:vscode Relates specifically to VS Code extension kind:bug Indicates an unexpected problem or unintended behavior "needs-triage"
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@gboston @RomneyDa