Ease MeiliSearch version upgrade

[tpayet]

If I am running v0.20 and want to run the last version of MeiliSearch (v0.21), I need to make a dump of my dataset somewhere then use this dump to restart MeiliSearch.

It would be easier if I could just shut down the old version of MeiliSearch and restart the new one using the same data.ms. It should be transparent for the user and if MeiliSearch need to reindex the data, so be it.

[curquiza]

I like this feature! :)
At the moment we cannot use the SaaS for our demos since we need to delete the instance and re-create one in order to upgrade the MeiliSearch version -> this is not convenient to maintain our demos since it involves changing all the links on the demo's side.

The SaaS could develop this feature. But I think this can be relevant for every user. At the start, MeiliSearch could try to read the DB; if not, it can create a dump and index it internally so that the users don't have to do the dump manipulation :)

[gmourier]

Ok! Thanks, @curquiza. As you said It can be the automation of the dump process when the MeiliSearch binary detects the need.

[bidoubiwa]

Meanwhile, we made an update MeiliSearch version script to update demo's created using our cloud scripts (which means some requirements in the configuration described in the readme).

If this is your case as well it might help you out: https://github.com/meilisearch/meilisearch-migration/
It automatically creates and import a dump if the version is not compatible with the current version of your instance. If they are compatible, it only updates MeiliSearch without importing the dumps.

⚠️ Breaking changes on previous settings might make the dumps import fail. In which case you will have to re-index your data.

[bidoubiwa]

The script is finished for those interested: https://github.com/meilisearch/meilisearch-migration/

It now has fallback to the original state in case of failure during migration.

[gmourier]

Hi @tpayet 👋

Why is the dump system not enough? Are the manipulations difficult to operate?

Could we potentially automate the upgrade process with all the dump manipulation like the script @bidoubiwa shared here?

@meilisearch/core-team I hardly see how the binary could embed this 🤔 but you may have an idea!

[tpayet]

Why is the dump system not enough? Are the manipulations difficult to operate?

With new updates every 4 weeks, I would prefer to have an easier process to update my running MeiliSearch instances. I'd love for MeiliSearch to have @bidoubiwa's upgrade process integrated into MeiliSearch.

When we have replication, we will even be able to have rolling upgrades 🎉

[MarinPostma]

@gmourier, if we stabilize the documents database and settings database, then we could use that instead of dumps to upgrade MeiliSearch. I don't thinks this would be very hard to implement

[gmourier]

Cross-linking the technical solutions discussion related to that need meilisearch/meilisearch#2570