Circumvent reaching the task DB size limit

[curquiza]

Related product team resources: roadmap card (internal only)
Related product discussion: meilisearch/specifications#238

Motivation

When Meilisearch reaches the maximum size for the task DB (the hard-coded value of 10GB), Meilisearch cannot accept any new tasks, including tasks deletion. It's even impossible to flush the task history.
In this situation, the users are stuck and cannot do anything.
We need to provide a way to by pass this issue.

Usage

No solution has been decided yet, needs to be discussed with @macraig and @gmourier.

Refer to the final spec to know the details and the final decisions about the usage.

TODO

• Fix Fix the health route #3644
• Merge Implement the auto deletion of tasks #3693
• Merge changes into main
• Update the spec
• Make the @meilisearch/cloud-team test the RC

Impacted teams

@meilisearch/docs-team -> the new behavior should be described, at least, as an engine limitation.
@meilisearch/cloud-team

[curquiza]

This issue (#3644) should be fixed as well as part of this current issue solving.

[curquiza]

@nicolasvienot @maryamsulemani97 @brunoocasali @gillian-meilisearch

For your information, so far, we don't plan to do a specification, all the implementation details should be present in the changelogs:

Maximum number of tasks the task queue can hold now limited to 1M. Fixes issues with enqueuing new tasks when the database is full.

Once the task queue reaches this limit, Meilisearch will try to delete the oldest 100k tasks. If not all 100k tasks can be deleted due to some of them being unfinished, we will delete as many tasks as possible.

You’ll still be able to register new tasks even after reaching this limit. The engine will only stop you from adding new tasks once the hard limit of 10GiB of tasks is reached (that’s between 5M and 15M of tasks depending on your workflow).

If automated deletion fails to delete anything, Meilisearch will log a warning notifying the user the engine is not working properly and no tasks were enqueued.

If it's not the case, I mean, if you would like to know more information, or if you would rather a specification written for this, let us know. Feel free to ask any question about the implementation here.

Pinging @macraig and @irevoire as team member of the product squad

UPDATE: finally, a spec is available here

[curquiza]

Hello everyone following this issue 👋

We have just released the first RC (release candidate) of Meilisearch containing this new implementation!
You can this feature it by using

• the release assets
• or the Meilisearch Docker image

docker run -it --rm -p 7700:7700 -v $(pwd)/meili_data:/meili_data getmeili/meilisearch:v1.2.0-rc.0

You are more than welcome to communicate any feedback about this new implementation in this current issue.
If you encountered any bugs, please report them here.

Thanks in advance for your help and your involvement in Meilisearch ❤️

🎉 Official and stable release containing this change will be available on 5th June 2023

⚠️ RC (release candidates) are not recommended for production