Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid displaying KeyboardInterrupt exception on terminal if user arbitrarily interrupts the inference #10

Merged
merged 1 commit into from
Aug 2, 2023
Merged

Conversation

felladrin
Copy link
Contributor

@felladrin felladrin commented Aug 2, 2023

Currently, if the user interrupts the inference via Ctrl/Cmd + C, it displays the following in the terminal:

Traceback (most recent call last):
  File "test.py", line 18, in <module>
    for output in minigpt4_chatbot.generate(
  File "minigpt4_library.py", line 646, in generate
    token = self.library.minigpt4_end_chat(self.ctx, self.n_threads, temp, top_k, top_p, tfs_z, typical_p, repeat_last_n, repeat_penalty, alpha_presence, alpha_frequency, mirostat, mirostat_tau, mirostat_eta, penalize_nl)
  File "minigpt4_library.py", line 419, in minigpt4_end_chat
    self.panic_if_error(self.library.minigpt4_end_chat(ctx.ptr, ctypes.pointer(token), n_threads, temp, top_k, top_p, tfs_z, typical_p, repeat_last_n, repeat_penalty, alpha_presence, alpha_frequency, mirostat, mirostat_tau, mirostat_eta, penalize_nl))
KeyboardInterrupt

Changes on this PR prevent the exception from being displayed if the user arbitrarily interrupts the inference, stopping the inference process cleanly.

@Maknee
Copy link
Owner

Maknee commented Aug 2, 2023

LGTM. Thanks for the catch -- it's pretty common for people to interrupt chat and I've done that before often.

@Maknee Maknee merged commit a275f9c into Maknee:master Aug 2, 2023
13 checks passed
@felladrin felladrin deleted the accept-keyboard-interrupt branch August 2, 2023 20:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants