Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make OpenAI client and ChatOpenAI compatible with OpenRouter API #292

Closed
3 tasks done
davidmigloz opened this issue Jan 13, 2024 · 1 comment
Closed
3 tasks done
Assignees
Labels
c:chat-models Chat models. t:enhancement New feature or request
Milestone

Comments

@davidmigloz
Copy link
Owner

davidmigloz commented Jan 13, 2024

https://openrouter.ai/

Current blocking issues:

  • OpenRouter does not send ChatCompletionResponseChoice.index field. Which is required by the OpenAI client currently
  • CreateChatCompletionResponse.id is null for some models
  • Tiktoken throws an exception when the model is not an OpenAI model (e.g. with mistralai/mistral-small)

Sample response using openai/gpt-3.5-turbo:

{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "I am an AI language model called ChatGPT. I don't have a personal name, but you can call me ChatGPT. How can I assist you today?"
      },
      "finish_reason": "stop"
    }
  ],
  "model": "openai/gpt-3.5-turbo",
  "usage": {
    "prompt_tokens": 35,
    "completion_tokens": 35,
    "total_tokens": 70
  },
  "id": "gen-UERvZMd21eaHdzLlvu0vi8GGBTEG",
  "object": "chat.completion",
  "created": 1705138269
}

Sample response using mistralai/mistral-small:

{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "123456789\n\nHere's how I did it:\n\nI simply wrote out the numbers from 1 to 9 in order without any spaces or commas in between."
      },
      "finish_reason": "stop"
    }
  ],
  "model": "mistralai/mistral-small",
  "usage": {
    "prompt_tokens": 40,
    "total_tokens": 85,
    "completion_tokens": 45
  },
  "id": "gen-ZlfNeKmOlxmoGNMlF5kHi8Spzfx7",
  "object": "chat.completion",
  "created": 1705142544
}
@davidmigloz davidmigloz added t:enhancement New feature or request c:chat-models Chat models. labels Jan 13, 2024
@davidmigloz davidmigloz added this to the v0.4.0 milestone Jan 13, 2024
@davidmigloz davidmigloz self-assigned this Jan 13, 2024
@davidmigloz davidmigloz changed the title Make ChatOpenAI compatible with OpenRouter API Make OpenAI client and ChatOpenAI compatible with OpenRouter API Jan 13, 2024
@davidmigloz
Copy link
Owner Author

KennethKnudsen97 pushed a commit to KennethKnudsen97/langchain_dart that referenced this issue Apr 22, 2024
KennethKnudsen97 pushed a commit to KennethKnudsen97/langchain_dart that referenced this issue Apr 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c:chat-models Chat models. t:enhancement New feature or request
Projects
Status: Done
Development

No branches or pull requests

1 participant