Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Tracking] Support more OpenAI parameters for chat completion endpoint in REST API #683

Closed
7 tasks done
sudeepag opened this issue Aug 7, 2023 · 1 comment
Closed
7 tasks done

[Tracking] Support more OpenAI parameters for chat completion endpoint in REST API #683

sudeepag opened this issue Aug 7, 2023 · 1 comment
Assignees
@davidpissarra
Labels
status: tracking Tracking work in progress

Comments

@sudeepag
Copy link
Collaborator

sudeepag commented Aug 7, 2023 •
edited by davidpissarra
Loading

Overview

The OpenAI chat completion API currently supports a number of parameters that are not currently supported in the mlc-llm REST API, such as temperature, stop sequences, etc. This issue tracks adding support for these parameters, so that they can be used by downstream dependencies of the REST API, such as Langchain integration.

Note that some parameters may require changes in the underlying chat module or llm_chat.

Action Items

  • temperature
  • top_p
  • n
  • stop
  • max_tokens
  • presence_penalty
  • frequency_penalty

See the OpenAI API Reference for details on each of these parameters.

Links to Related Issues and PRs
@sudeepag sudeepag added the status: tracking Tracking work in progress label Aug 7, 2023
@tqchen tqchen moved this to Todo in MLC LLM Tracking Aug 8, 2023
@sudeepag sudeepag mentioned this issue Aug 8, 2023
Closed
@davidpissarra davidpissarra self-assigned this Oct 1, 2023
@davidpissarra davidpissarra moved this from Todo to In Progress in MLC LLM Tracking Oct 2, 2023
@davidpissarra davidpissarra mentioned this issue Oct 5, 2023
Merged
@github-project-automation github-project-automation bot moved this from In Progress to Done in MLC LLM Tracking Oct 24, 2023
@davidpissarra
Copy link
Member

Concluded by #1107

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@davidpissarra davidpissarra

Labels
status: tracking Tracking work in progress
Projects
MLC LLM Tracking
Status: Done
Milestone
No milestone
Development

No branches or pull requests
2 participants
@sudeepag @davidpissarra