Atomic migrations with __migrate

[tupui]

In Stellar smart contracts we have some handy features to update the contract WASM. When updating contracts, the storage schemas might evolve and require some migration. xref some discussions https://github.com/orgs/stellar/discussions/1877 and this thread which prompted this discussion https://discord.com/channels/897514728459468821/971932562765250701/1523730237387706469.

If there is a need to migrate the data (because the schema changed in a way the above issue would not be enough, like adding a new computed field which is mandatory), you need some extra function to run. What one can do now, and not limited to this:

1. starting with an upgradeable contract,
2. deploy a new version which has both the new schema and a migrate function,
3. run the migration.

Now doing that you can bloat your contract with code which is only going to run once. You could publish a new revision at that point which does not have the said migration.

Doing all that flow is possible in an atomic fashion using a wrapper contract. That's all fine, but at the same time, it's an easy footgun.

In a similar fashion as we have __constructor, we could have a __migrate function. Hence you would not have to potentially pause the contract, assuming it's even a possibility. That function would be call in a similar fashion as __constructor. You would upgrade your contract and as it deploys __migrate is doing it's job.

What I personally don't like with all that is that in the end we still have some bloat in the contract. One can still do another upgrade to remove the migrate function I suppose.

Now writting this, and thinking it's the same with __constructor which also is some leftover.

Some arguments raised in the discussions are about the potential fees required to actually migrate data. E.g. if you wanted to update a list of 1000 users, it could be prohibitive to do so using normal get/set. Meaning that the need for a migration semantic might be questionable. Unless we had special tools to do bulk operations (user defined functions (UDF) kind of things for the DB folks.) I am still scratching my head there as to me behind the scenes a DB is a DB. So I am used to migration and rollback scripts which are not shipped to the executable itself and just connect to the DB to do the data migration part (well here you even have different schools of thoughts with people only changing schemas in migrations and having separate jobs to migrate the data itself. You get the idea.) In this context, you usually keep your migrations around and in tests you even replay all migrations. Something like this is of course prohibitive to do in a contract and you end up rewritting the same function over and over again, losing the history and replay. I digress slightly.

Hence the naive proposal I had to have a way to either use a side-car WASM or one-off functions you could use and drop. So that the WASM that stays on-chain is lean with only code that is actually needed.

Interested to read how people deal with migration and approach considered.