Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] Interrupting Generation Discards Extra Swipes #2282

Open
3 tasks done
Beinsezii opened this issue May 20, 2024 · 0 comments
Open
3 tasks done

[BUG] Interrupting Generation Discards Extra Swipes #2282

Beinsezii opened this issue May 20, 2024 · 0 comments
Labels
馃悰 Bug [ISSUE] Ticket describing something that isn't working 馃惂 Linux [ISSUE] The Environment is on Linux

Comments

@Beinsezii
Copy link

Environment

馃惂 Linux

System

vllm/PyTorch 2.3.0+roc6.0, Firefox 126

Version

12.0 release & staging df2710 both affected

Desktop Information

No response

Describe the problem

Using a backend that supports multiple swipes per gen such as vllm, if the user interrupts generation all but the first swipe are discarded.

Repro

  • Set backend to vllm or something else with multiple swipes
  • Set swipes per gen to 8
  • Send a message, receive 8 swipes
  • Send a message and interrupt it, receive 1 swipe

Additional info

No response

Please tick the boxes
@Beinsezii Beinsezii added the 馃悰 Bug [ISSUE] Ticket describing something that isn't working label May 20, 2024
@github-actions github-actions bot added the 馃惂 Linux [ISSUE] The Environment is on Linux label May 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
馃悰 Bug [ISSUE] Ticket describing something that isn't working 馃惂 Linux [ISSUE] The Environment is on Linux
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@Beinsezii