You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@avacaondata
so basically you need to add MinPLogitsWarper to /server/text_generation_server/utils/logits_process.py and the min_p parameter to NextTokenChooser class in /server/text_generation_server/utils/tokens.py
Then go over a ton of files and add the min_p parameter wherever needed. Here I would suggest you searching for one of the existing generation params like top_p and adding the missing lines of code for the new parameter min_p
After you done that, just build the image with the default Dockerfile and use it as you usually do
If you have any further questions, feel free to ask :)
Feature request
Hey! I've noticed that
min_p
parameter is now available in transformers but haven't found it in TGI. Are you planning to add it in the nearest future?The parameter itself is very well described here
I have implemented it in TGI for myself locally and I would be glad to contribute by submitting a PR :)
The text was updated successfully, but these errors were encountered: