feat(tools): Parallel function calling

[mudler]

Description

This PR fixes #1275 and provides a drop-in support for https://platform.openai.com/docs/guides/function-calling/parallel-function-calling

This feature is gated by a config option that needs to be explicitly enabled in the model YAML configuration file as it is still experimental:

function:
  parallel_calls: true

Notes for Reviewers

Signed commits

• Yes, I signed my commits.

This PR didn't went through testing (yet)

[netlify]

✅ Deploy Preview for localai canceled.

Name	Link
🔨 Latest commit	ab7603c
🔍 Latest deploy log	https://app.netlify.com/sites/localai/deploys/65d4efdea4d4b10008ec9b93

[mudler]

@stippi2 this is quite experimental here, but I'd be happy if you can give some early feedback - at least to double check if at API level we are doing the correct thing

[stippi2]

Thanks for the heads-up! Will give it a shot.

[stippi2]

BTW, last time you mentioned I was running the wrong model for successful function calling. Can you recommend one I should test with? I am still learning what I should look out for in a model description. I have a 12 core Windows PC with a 3090 in it, and an i9 8 core Intel MacBook. Both computers have 32 GiG of RAM and the 3090 has 24 GiG of VRAM. My setup for building LocalAI from source in on the MacBook, so if there is a model I could try on the Mac, that would be best. Thanks for any pointers.

[mudler]

BTW, last time you mentioned I was running the wrong model for successful function calling. Can you recommend one I should test with? I am still learning what I should look out for in a model description. I have a 12 core Windows PC with a 3090 in it, and an i9 8 core Intel MacBook. Both computers have 32 GiG of RAM and the 3090 has 24 GiG of VRAM. My setup for building LocalAI from source in on the MacBook, so if there is a model I could try on the Mac, that would be best. Thanks for any pointers.

I had good results with WizardLM 30b. However, I think you can find models which are fine-tuned against dataset which have more training data with functions. For instance, good ones that pops in my mind are:

• https://huggingface.co/TheBloke/NexusRaven-V2-13B-GGUF ( https://huggingface.co/Nexusflow/NexusRaven-V2-13B )
• https://huggingface.co/meetkai/functionary-medium-v2.2-GGUF

More broadly, models that are fine-tuned against https://huggingface.co/datasets/togethercomputer/glaive-function-calling-v2-formatted have more chances to get this right.

To note, best results are with models >=70b with my tests. Speculative sampling can help here as well.

Also: I got good results only when I started to construct a good prompt template explaining the tools available, what they do, and how should be used with some example.

[mudler]

I've been testing locally this with mistral openorca, and seems to work:

curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
  "model": "mistral-openorca", 
  "messages": [
    {
      "role": "user",
      "content": "What is the weather like in Boston and in san francisco?"
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_current_weather",
        "description": "Get the current weather in a given location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The city and state, e.g. San Francisco, CA"
            },
            "unit": {
              "type": "string",
              "enum": ["celsius", "fahrenheit"]
            }
          },
          "required": ["location"]
        }
      }
    }
  ],
  "tool_choice": "auto"
}' | jq

result:

{           
  "created": 1708453855,
  "object": "chat.completion",
  "id": "1e8099ca-e73d-49a6-b2c8-2f0e29b77e12",
  "model": "mistral-openorca",        
  "choices": [                                                                                                        
    {                  
      "index": 0,          
      "finish_reason": "tool_calls",
      "message": {       
        "role": "assistant",   
        "content": null,                                                                                              
        "tool_calls": [
          {          
            "index": 0,        
            "id": "1e8099ca-e73d-49a6-b2c8-2f0e29b77e12",
            "type": "function",
            "function": {
              "name": "get_current_weather",
              "arguments": "{\"location\":\"Boston\",\"unit\":\"fahrenheit\"}"
            }
          },
          {
            "index": 0,
            "id": "1e8099ca-e73d-49a6-b2c8-2f0e29b77e12",
            "type": "function",
            "function": {
              "name": "get_current_weather",
              "arguments": "{\"location\":\"San Francisco\",\"unit\":\"fahrenheit\"}"
            }
          }
        ]
      }
    }
  ],
  "usage": {
    "prompt_tokens": 0,
    "completion_tokens": 0,
    "total_tokens": 0
  }
}