Replies: 2 comments
|
+1, cached queries don't really test much other than faiss/search working as expected? |
0 replies
|
+1, I think it actually makes more sense reproducibility wise to make sure re-encoding the queries give the expected results, ie explicitly including query encoding to always be part of the pipeline instead of relying on pre encoded embeddings |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Currently, for many reproductions involving encoders, we have two versions:
The original intent was that for faster iteration, we could just use the cached queries... but:
My current thinking: there's little point in having cached queries - more to test, and they don't appear to solve an actual pain point...
So, my proposal is to remove cached queries in our reproductions.
Thoughts?
All reactions