Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor ChatSkill to use new Chat Completions prompt format #337

Merged

Conversation

teresaqhoang
Copy link
Contributor

@teresaqhoang teresaqhoang commented Sep 12, 2023

Motivation and Context

This PR refactors the ChatSkill to use the new ChatCompletions prompt format as the meta-prompt template. The new format serves the prompt template as a series of context messages, where each message can be tagged a role, to prime the model in a certain way. These roles include: System, User, and Assistant.

Description

For ChatSkill, the prompt template for the meta-bot response request utilizes the message roles in the following way:

  • System: for any instructions or additional context provided to the model, including bot persona, relevant memories, or planner output.
  • Assistant: Any bot response message in chat history previously outputted as chat completion result.
  • User: Any user input.

The series of messages between the user and the assistant is included both as few shot examples and additional context (chat history). The last user message is always included.

Other changes:

  • This PR also adds a custom token calculator, since chat streaming doesn't return token usage details.
  • System continuation preamble is no longer needed with this new format, since it's been optimized for chat responses.
  • Added visible scroll bar to prompt dialog.
  • Simplified SystemResponse since this conversational interface is already optimized for chat.
image image

Contribution Checklist

@teresaqhoang teresaqhoang added enhancement New feature or request dependencies Pull requests that update a dependency file labels Sep 12, 2023
@teresaqhoang teresaqhoang requested review from TaoChenOSU and a team September 12, 2023 23:15
@teresaqhoang teresaqhoang self-assigned this Sep 12, 2023
@github-actions github-actions bot added webapp Pull requests that update Typescript code webapi Pull requests that update .net code PR: ready for review and removed dependencies Pull requests that update a dependency file labels Sep 12, 2023
@teresaqhoang teresaqhoang changed the title Refactor chat message prompt message types Refactor ChatSkill to use new Chat Completions prompt format Sep 12, 2023
webapi/Auth/PassThroughAuthenticationHandler.cs Outdated Show resolved Hide resolved
webapi/Skills/ChatSkills/ChatSkill.cs Outdated Show resolved Hide resolved
webapi/Skills/ChatSkills/ChatSkill.cs Outdated Show resolved Hide resolved
webapi/Skills/TokenUtilities.cs Outdated Show resolved Hide resolved
webapi/Models/Response/BotResponsePrompt.cs Outdated Show resolved Hide resolved
webapi/Models/Response/BotResponsePrompt.cs Outdated Show resolved Hide resolved
webapi/Skills/TokenUtilities.cs Show resolved Hide resolved
webapi/Skills/ChatSkills/ChatSkill.cs Outdated Show resolved Hide resolved
webapi/Skills/ChatSkills/ChatSkill.cs Outdated Show resolved Hide resolved
webapi/Skills/ChatSkills/ChatSkill.cs Show resolved Hide resolved
webapi/Skills/ChatSkills/ChatSkill.cs Show resolved Hide resolved
@teresaqhoang teresaqhoang added this pull request to the merge queue Sep 18, 2023
Merged via the queue into microsoft:main with commit ce76b50 Sep 18, 2023
8 checks passed
@teresaqhoang teresaqhoang deleted the Refactor-chat-message-completion-types branch September 18, 2023 21:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request webapi Pull requests that update .net code webapp Pull requests that update Typescript code
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

None yet

4 participants