Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: allow patching req body with client config #534

Merged
merged 3 commits into from
May 22, 2024
Merged

feat: allow patching req body with client config #534

merged 3 commits into from
May 22, 2024

Conversation

sigoden
Copy link
Owner

@sigoden sigoden commented May 22, 2024

Each client has a new patches configuration

 - type: <client>
   patches: 
      <regex>:                                    # The regex to match model names, e.g. '.*' 'gpt-4o' 'gpt-4o|gpt-4-.*'
        request_body:                             # The JSON to be merged with the request body.

We can use this pattern to modify the request body.

Enable web-search for cohere models

  - type: cohere
    patches:
      ".*":
        request_body:
          connectors:
            - id: web-search

Change safetySettings of gemini models

  - type: gemini
    patches:
      'gemini-.*':                                           
        request_body:                                 # Override safetySettings for gemini models
          safetySettings:
            - category: HARM_CATEGORY_HARASSMENT
              threshold: BLOCK_NONE
            - category: HARM_CATEGORY_HATE_SPEECH
              threshold: BLOCK_NONE
            - category: HARM_CATEGORY_SEXUALLY_EXPLICIT
              threshold: BLOCK_NONE
            - category: HARM_CATEGORY_DANGEROUS_CONTENT
              threshold: BLOCK_NONE

This PR replaces the previous strategy of adding fields through extra_fields #298

@sigoden sigoden changed the title feat: allow patching req body through configuration feat: allow patching req body with client config May 22, 2024
@sigoden sigoden merged commit ba3bcfd into main May 22, 2024
3 checks passed
@sigoden sigoden deleted the feat branch May 22, 2024 13:29
@jiahut
Copy link

jiahut commented May 23, 2024

@sigoden report bug ( patched the web-search connector )

> .model cohere:command-r-plus


> 最近一个月发布了哪些开源的LLM
Failed to get answer

Caused by:
    incomplete utf-8 byte sequence from index 2749

The above error did not occur consistently. I conducted three consecutive inquiries, and the first two attempts resulted in errors, while the third attempt successfully received a response.

@sigoden
Copy link
Owner Author

sigoden commented May 23, 2024

#538 fixed the bug

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants