Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid a full index deletion in Transform when changing settings #4478

Closed
3 tasks done
ManyTheFish opened this issue Mar 12, 2024 · 1 comment · Fixed by #4504
Closed
3 tasks done

Avoid a full index deletion in Transform when changing settings #4478

ManyTheFish opened this issue Mar 12, 2024 · 1 comment · Fixed by #4504
Labels
performance Related to the performance in term of search/indexation speed or RAM/CPU/Disk consumption settings diff-indexing Issues related to settings diff-indexing v1.8.0 PRs/issues solved in v1.8.0 released on 2024-05-06
Milestone

Comments

@ManyTheFish
Copy link
Member

ManyTheFish commented Mar 12, 2024

Related product team resources: PRD (internal only)

Summary

This issue is a subset of the work implementing the settings diff-indexing enhancement.

The current approach of Transform::prepare_for_documents_reindexing is to dump all the documents in the index into a Grenad::File and delete the whole index to reindex all the documents, taking into account the Settings changes.

This function should instead create document deltas without clearing the index.

TODO

@ManyTheFish ManyTheFish added performance Related to the performance in term of search/indexation speed or RAM/CPU/Disk consumption settings diff-indexing Issues related to settings diff-indexing labels Mar 12, 2024
@ManyTheFish ManyTheFish changed the title Avoid a full index deletion in Transform::prepare_for_documents_reindexing methods Avoid a full index deletion in Transform when changing settings Mar 12, 2024
meili-bors bot added a commit that referenced this issue Apr 17, 2024
4504: Avoid clearing db in transform r=ManyTheFish a=ManyTheFish

# Pull Request

## Related issue
Fixes #4478



Co-authored-by: ManyTheFish <many@meilisearch.com>
@ManyTheFish
Copy link
Member Author

fixed by #4504

@curquiza curquiza added this to the v1.8.0 milestone Apr 18, 2024
@meili-bot meili-bot added the v1.8.0 PRs/issues solved in v1.8.0 released on 2024-05-06 label May 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance Related to the performance in term of search/indexation speed or RAM/CPU/Disk consumption settings diff-indexing Issues related to settings diff-indexing v1.8.0 PRs/issues solved in v1.8.0 released on 2024-05-06
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants