Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modify to support batch size N #15

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

quinle
Copy link

@quinle quinle commented Oct 14, 2022

Hi,
Thank you so much for the great project, it is amazing. This PR is for adding support for OPT inference batch size N.
I'm looking forward to your review ^^

@yxuansu
Copy link
Owner

yxuansu commented Oct 19, 2022

Hi @quinle -- Thank you for your interest in our work and the great effort! We will take a closer look to your commits, many thanks!

@gmftbyGMFTBY Are you able to review the PR by any chance?

@yxuansu
Copy link
Owner

yxuansu commented Nov 5, 2022

Hi @quinle -- Thank you again for your interest in our work. We are glad that Huggingface has officially supported contrastive search in transformers. Here, we provide a short tutorial. In transformers, contrastive search supports batch inference by default.

Hope this resource could help you :-)

Best,

Yixuan

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants