-
Notifications
You must be signed in to change notification settings - Fork 9.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
infill mode for main and server #3296
Conversation
Should we make a separate |
It shares a lot with main but it might well make sense to not clutter the functionality of main too much. |
I think a separate minimal |
whats the difference between infill/this pr and fim/#2934 ? |
So more or less taking over the structure of #2934 and fill it with this example? Sounds good. What about the --interactive flag, is that valuable or do you just want the bare minimum? As it's already working with --interactive it shouldn't be any extra work to keep it. |
* reverted changes to main and added infill example
Ok, I'm done, no more changes in main.cpp, all moved to infill.cpp and added readme. |
I tried to merge master into this PR, but your branch is protected so it won't let me:
|
could you try again, I unprotected my master |
Done, should all be working now. |
* vvhg-code-infill (ggerganov#1) * infill in separate example (ggerganov#2) * reverted changes to main and added infill example * cleanup * naming improvement * make : add missing blank line * fix missing semicolon * brought infill up to current main code * cleanup --------- Co-authored-by: Cebtenzzre <cebtenzzre@gmail.com>
Hi, is there any plan to implement non-stream mode for infill in server side? |
Contributions are welcome |
This adds support for the infill tasks in ./main and ./server.
There are still a few things that should be improved, but it works as is.
TODO:
PS hope it helps, thanks for the great work!!