Query to fetch single record from 10K total keys is very slow (~12 seconds) #207
Comments
|
Just adding some notes here... here's the index creation command that results from the code above: I think this might have performance issues in your use case as the |
|
You can define this field as class Person(HashModel):
first_name: str = Field(index=True)
last_name: str = Field(index=True)
emp_no: int = Field(index=True, sortable=True)This then creates it as a sortable field in the underlying RediSearch index: However, this doesn't improve performance for: Person.find().sort_by('-emp_no').first()which does find the right record: but in doing so it fires off 1000 I'll have to take a look at this. |
|
This is partly caused by Redis OM telling RediSearch to return results in a page size of 10, and iterating over the whole 10,000 records. I'm going to change this default to 1000 and look at making it configurable per call to |
|
This will be fixed in the upcoming 0.0.25 release. |
|
@msarm fixed in 0.0.25 which is now available. |
|
@simonprickett - I see 'ft.search' round trip got reduced now after defaulting the page size to 1000 and this helps other queries to run even faster but my results are not coming this time may be a different issue now. When I perform the Person query: Here is the 'ft.search' performed. When the last fired 'ft.search' is performed it does not return any results it retries for every. Since are looking just for the first record, can we just fire the first 'ft.search' and return the results? When I set explicitly the exhaust_results=False in the first() function, I see quick results with no extra query. Is that right the fix? |
|
@simonprickett - After explicitly setting the exhaust_results=False in the first() function, I'm able to verify it supported 10 million keys @ ~11K commands/sec without any performance issues.
|
|
Hi there - thanks for the extra info, I'll treat the |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The text was updated successfully, but these errors were encountered: