Describe the bug
A few methods in the memory interface don't scale beyond a few records. This is because they query by ID and have to insert all the IDs into the query which runs into a limit.
What should be done about it
We should
a) find out what the limit is (per memory type, but fine to start with SQLite only)
b) edit get_scores_by_prompt_ids to batch the requests, i.e., if you're given 10000 IDs then don't put all of them into a single request, but rather send them in batches of 100 (or whatever the limit from a) is).
The affected methods as far as I can tell are all the ones using get_scores_by_prompt_ids
We also need unit tests for these methods to do the batching properly, and integration tests where we exceed the limit and it still works.
Steps/Code to Reproduce
Just call any of these methods with 1000 scores and it'll fail.
Expected Results
Should retrieve the scores without error.
Actual Results
exception
Describe the bug
A few methods in the memory interface don't scale beyond a few records. This is because they query by ID and have to insert all the IDs into the query which runs into a limit.
What should be done about it
We should
a) find out what the limit is (per memory type, but fine to start with SQLite only)
b) edit
get_scores_by_prompt_idsto batch the requests, i.e., if you're given 10000 IDs then don't put all of them into a single request, but rather send them in batches of 100 (or whatever the limit from a) is).The affected methods as far as I can tell are all the ones using
get_scores_by_prompt_idsWe also need unit tests for these methods to do the batching properly, and integration tests where we exceed the limit and it still works.
Steps/Code to Reproduce
Just call any of these methods with 1000 scores and it'll fail.
Expected Results
Should retrieve the scores without error.
Actual Results
exception