Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exclude prompt from generated response #1485

Merged
merged 2 commits into from
Jun 12, 2024
Merged

Exclude prompt from generated response #1485

merged 2 commits into from
Jun 12, 2024

Conversation

rasbt
Copy link
Collaborator

@rasbt rasbt commented Jun 12, 2024

The default setting in generate/base.py returns the formatted prompt plus response, which is useful for demonstration and seeing what's going on. However, when using the model for serving, it can lead to ugly outputs like

system

You are a helpful assistant.
user

Fix typos in the following sentence: Exampel input
assistant

The corrected sentence is: "Example input".

So this PR fixes that and allows the generate function in generate/base.py to be consistent with the chat behavior.

CC @aniketmaurya

@rasbt rasbt merged commit c0f7686 into main Jun 12, 2024
8 of 9 checks passed
@rasbt rasbt deleted the exclude-prompt branch June 12, 2024 22:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant