Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix multi-turn prompting error handling and extra newline #1569

Merged
merged 2 commits into from
Jul 10, 2024
Merged

Conversation

apaz-cli
Copy link
Collaborator

Using .format() may overwrite user input. Rewrote in the style of the llama prompt format, and removed the extra newline from the system prompt.

@rasbt
Copy link
Collaborator

rasbt commented Jul 10, 2024

Thanks for the contribution!

Based on checking https://llama.meta.com/docs/model-cards-and-prompt-formats/meta-llama-3/, there shouldn't be a newline indeed. Any thoughts @Andrei-Aksionov ?

@Andrei-Aksionov
Copy link
Collaborator

No, I don't have any clue why an extra newline was added in the first place.
Additionally, I checked that \n\n separately (like in Meta repo) and in concatenation with other tokens like <|end_header_id|>\n\n is encoded as the same token. Sometimes encode(prompt_1) + encode(prompt_2) != encode(prompt_1 + prompt_2) #1567.

Thanks for the PR @apaz-cli 👍

@rasbt rasbt merged commit d49ce98 into main Jul 10, 2024
9 checks passed
@rasbt rasbt deleted the ap-l3prompt branch July 10, 2024 18:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants