Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cache the result of the indexes stats #3540

Closed
irevoire opened this issue Feb 23, 2023 · 3 comments · Fixed by #3541
Closed

Cache the result of the indexes stats #3540

irevoire opened this issue Feb 23, 2023 · 3 comments · Fixed by #3541
Labels
enhancement New feature or improvement performance Related to the performance in term of search/indexation speed or RAM/CPU/Disk consumption v1.1.0 PRs/issues solved in v1.1.0 released on 2023-04-03
Milestone
v1.1.0

Comments

@irevoire
Copy link
Member

Since #3331 has been merged, we now have the possibility to store thousands of indexes in meilisearch.
But at the same time, opening indexes became slower.

That can be a real issue when calling the global stats.
And it will also degrade the usage of the new experimental /metrics route.

After chatting with @dureuill we decided it would be better to cache the result of the stats on disk to avoid opening the indexing just to extract a few information out of the index.
@irevoire irevoire added enhancement New feature or improvement performance Related to the performance in term of search/indexation speed or RAM/CPU/Disk consumption labels Feb 23, 2023
@irevoire irevoire added this to the v1.1.0 milestone Feb 23, 2023
@irevoire irevoire mentioned this issue Feb 23, 2023
Merged
@gmourier
Copy link
Member

gmourier commented Feb 24, 2023 •
edited

Is the cache invalidated as soon as an index operation impacts its stats number? Just wondering if the information is always up-to-date when requested by /stats 🤓

@irevoire
Copy link
Member Author

The cache is updated after any update is applied to an index.
Thus it should always be up to date.

bors bot added a commit that referenced this issue Mar 9, 2023
@bors @irevoire @dureuill
Merge #3541
667bb87 
3541: Add cache on the indexes stats r=dureuill a=irevoire

Fix #3540

Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
@curquiza
Copy link
Member

curquiza commented Mar 9, 2023

Fixed by #3541
Will be released in rc1 today

@curquiza curquiza closed this as completed Mar 9, 2023
@meili-bot meili-bot added the v1.1.0 PRs/issues solved in v1.1.0 released on 2023-04-03 label Apr 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or improvement performance Related to the performance in term of search/indexation speed or RAM/CPU/Disk consumption v1.1.0 PRs/issues solved in v1.1.0 released on 2023-04-03
Projects
None yet
Milestone
v1.1.0
Development

Successfully merging a pull request may close this issue.

Add cache on the indexes stats meilisearch/meilisearch
4 participants
@gmourier @irevoire @curquiza @meili-bot