Allow migrations between releases or between different configurations

[kderme]

So far when there is a new major release, db-sync users have to drop the db and sync everything from genesis, which can be a time-wasting and painful procedure. One of the reasons this is necessary is that there is no concept of backwards compatibility in the ledger and in the ledger events. So on an major upgrade, ledger snapshots become invalid and need to be recreated from genesis. The ledger events also change, making it impossible to predict changes to inserted entries in the db. So dropping and recreating the db has been the way to go.

Two recent changes make it possible to migrate in a better way for future releases:

• the leger events are much more stable now.
• we have separated the data that come from ledger/events from the rest of the data, with the intoduction of disable-ledger

For future releases, when disable-ledger is enabled, db-sync users should never have to delete the db.
When it's not, we should still have a way to replay just the ledger rules, without resyncing the db. While replaying, db-sync should only modify the data related to ledger/events. When the ledger reaches the tip of the db, db-sync should continue syncing normally. The same approach can be followed when there is a migration from enabled disable-ledger, which misses some data, to disabled disable-ledger. The additional data should be filled without the need to resync everything.

[marshada]

Revise this issue, to clarify the work required to put proper migration infrastucture in place. @kderme create a list or summary of the specific work items that are needed to improve migrations, and perhaps then create an epic and identify what improvements are a priority.