New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

multi: Migration for utxo set semantics reversal. #1520

Merged
merged 1 commit into from Nov 13, 2018

Conversation

Projects
None yet
6 participants
@davecgh
Copy link
Member

davecgh commented Nov 9, 2018

This requires PR #1471 and #1500.

Testing Notes

As of this PR, the expected behavior is that there is a single migration that takes an hour or two to complete, depending on enabled indexes, after which it will no longer be possible to downgrade.

As the warning above notes, if you try to run an older software version after this migration has completed, you will get an error message similar to Unable to start server on [:9108]: the current blockchain database is no longer compatible with this version of the software (5 > 4).


This adds code to migrate the database to the new schema required by the recent utxo set semantics reversal changes.

It involves removing the existing utxo set, spend journal, ticket database, various indexes, resetting the best chain tip back to the genesis block, and finally performing a full reindex. The process can be interrupted at any point and future invocations will resume from the point it was interrupted.

In addition, some new infrastructure to remove the ticket database and support for index versions is added to support the migration.

Closes #1525.


This is work towards #1145.

@davecgh davecgh added this to the 1.4.0 milestone Nov 9, 2018

@davecgh davecgh referenced this pull request Nov 9, 2018

Open

Multi-peer Checklist #1145

15 of 25 tasks complete

@davecgh davecgh force-pushed the davecgh:blockchain_reverse_utxo_semantics_migration branch 2 times, most recently from b828e81 to 6fe66ae Nov 10, 2018

@dnldd

dnldd approved these changes Nov 10, 2018

@dnldd

This comment has been minimized.

Copy link
Member

dnldd commented Nov 10, 2018

Testnet run smoothly without any issues. Took about 7 minutes to reindex.

@matheusd

This comment has been minimized.

Copy link
Member

matheusd commented Nov 12, 2018

The only error condition I could trigger is if you interrupt the migration right at the start (so that the utxo set is cleared but the actual migration hasn't started yet), then go back to the older version: you don't get the migration version error, but some reorg errors (0bin.net log)

I don't think this needs to be specifically tested against, since it should be pretty hard state to get to, but if we ever see some startup reorg errors in the future, this might be it.

Besides this, I tested TERMing/QUITing/KILLing the process a bunch of times throughout the migration but didn't hit any other errors.

Results for mainnet migration:

2018-11-12 13:58:38.192 [INF] CHAN: Clearing database utxoset and spend journal for upgrade...
...
2018-11-12 14:45:10.818 [INF] CHAN: Reindexing to height 290640 of 290640 (progress 100.00%)...
2018-11-12 14:45:12.812 [INF] CHAN: Blockchain database version info: chain: 5, compression: 1, block index: 2
@davecgh

This comment has been minimized.

Copy link
Member Author

davecgh commented Nov 12, 2018

@matheusd Thanks for the thorough testing. I'm not sure there is really a way to fix the "go back to old version mid upgrade" condition, because the older versions don't look for an "upgrade in progress" flag of any sort. It probably makes sense to introduce something of that sort going forward, but since the code doesn't already exist in the older version, I'm not sure anything can be done there.

@dajohi

dajohi approved these changes Nov 12, 2018

Copy link
Member

dajohi left a comment

testnet3 miner tOK

Show resolved Hide resolved blockchain/chain.go Outdated
multi: Migration for utxo set semantics reversal.
This adds code to migrate the database to the new schema required by the
recent utxo set semantics reversal changes.

It involves removing the existing utxo set, spend journal, ticket
database, various indexes, resetting the best chain tip back to the
genesis block, and finally performing a full reindex.  The process can
be interrupted at any point and future invocations will resume from the
point it was interrupted.

In addition, some new infrastructure to remove the ticket database and
support for index versions is added to support the migration.

@davecgh davecgh force-pushed the davecgh:blockchain_reverse_utxo_semantics_migration branch from 6fe66ae to e875495 Nov 12, 2018

@jrick

This comment has been minimized.

Copy link
Member

jrick commented Nov 13, 2018

The migration took 48 minutes on my server (mainnet) and the entire process went smoothly. The only concern (which may not actually be an issue) is that once the migration completed and peers were being served and blocks connected, the median time appears to have been created using a time sample from before the migration occurred:

2018-11-13 15:08:31.909 [DBG] CHAN: New time offset: -18m29s

This may create issues depending on how long the migration actually took, but in my case, it did not cause any problems on a non-mining node.

@alexlyp
Copy link
Member

alexlyp left a comment

migration OK on testnet3 and code looks great.

@davecgh

This comment has been minimized.

Copy link
Member Author

davecgh commented Nov 13, 2018

Regarding the time offset, I looked into both the code and performed the migration a few times now. I've confirmed that no time samples are added before the reindex and the listeners for peers to connect is not started until after the reindex is complete. It was a time sample from a peer with a bad time.

@davecgh davecgh merged commit e875495 into decred:master Nov 13, 2018

1 of 2 checks passed

continuous-integration/travis-ci/push The Travis CI build is in progress
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@davecgh davecgh deleted the davecgh:blockchain_reverse_utxo_semantics_migration branch Nov 13, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment