You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Just imagine this, a dynamic logit token remover that knows exactly how many tokens to remove for every new token generation. That could be possible if this sampler is working with AI:
You train that AI like this:
For the input it's gonna be the logits (could be Mixtral logits for example so it works really well for that specific model and tokenizer)
For the output it's gonna a new logit that has a 0% probability on all the "bad" tokens
With enough training data the AI would know exactly what are the garbage tokens and what are the viable ones.
Theorically this would solve the samplers problem, @kalomaze what do you think?
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Just imagine this, a dynamic logit token remover that knows exactly how many tokens to remove for every new token generation. That could be possible if this sampler is working with AI:
You train that AI like this:
With enough training data the AI would know exactly what are the garbage tokens and what are the viable ones.
Theorically this would solve the samplers problem, @kalomaze what do you think?
Beta Was this translation helpful? Give feedback.
All reactions