Slow /stats update

[zurferr]

Describe the bug
I try to use the isIndexing attribute to decide, when the index is ready.
To be sure I check again the attribute numberOfDocuments. However, while isIndexing return False. The numberOfDocuments can return 0. If I wait longer I get the correct result.

https://docs.meilisearch.com/reference/api/stats.html#get-stat-of-an-index

To Reproduce

    # fill index will lots of objects (roughly 1000)

    # wait for index to finish building
    turns = 0
    while my_index.get_stats()["isIndexing"] and turns < 15:
        time.sleep(5)
        turns += 1

    # check search is filled
    if not my_index.get_stats()["isIndexing"]:
        time.sleep(1) # needs to be increased to 60 seconds, then the test works
        assert 100 < my_index.get_stats()['numberOfDocuments']

Expected behavior
As soon as the index is ready the number of documents should be correct.

Environment:

• OS: Docker Linux
• MeiliVersion 19
• Python Driver

This is not explicitly documented, so it might be intended behavior. Still it would be nice to have more guidance on this.

[curquiza]

Hello @zurferr!

Thanks for this report.

To know when the documents are pushed and if the action succeeded you should definitely check the update status instead. Indeed, the stats route will not inform you about the failure/success status. The stats route informs you about the state of your index but should not be used for this purpose.
Check the update status: https://docs.meilisearch.com/reference/api/updates.html
More about the asynchronous action of MeiliSearch: https://docs.meilisearch.com/learn/advanced/asynchronous_updates.html

If you use the Python SDK, you might want to use the wait_for_pending_update method that checks the update status for you 😉
Here is the usage: https://github.com/meilisearch/meilisearch-python/blob/c6aa009c412901548bfd04ee9089f3767faa6046/meilisearch/tests/conftest.py#L51-L52
And here is the definition of the method, for your information: https://github.com/meilisearch/meilisearch-python/blob/937b75f282111ecf13dd824119ff2d02650eb92e/meilisearch/index.py#L181-L212

Thanks anyway for reporting this lag 👍

[curquiza]

I did not succeed to reproduce it using the main branch of https://github.com/meilisearch/transplant/

[curquiza]

Fixed by the integration of the new search engine in #1421