The huggingface backend for alora support doesn't use hashed values / uuids for results. It uses the decoded result directly as the index into the cache. For short outputs, there will likely be many conflicts, especially if the model is directed to output only yes or no.
We will have to think about how to keep track of the specific output as well when checking the cache.
The huggingface backend for alora support doesn't use hashed values / uuids for results. It uses the decoded result directly as the index into the cache. For short outputs, there will likely be many conflicts, especially if the model is directed to output only yes or no.
We will have to think about how to keep track of the specific output as well when checking the cache.