New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

deterministic snapshots #5956

Merged
merged 41 commits into from Oct 16, 2018

Conversation

Projects
None yet
2 participants
@wanderingbort
Contributor

wanderingbort commented Oct 10, 2018

Starting this review early.

This is the plumbing and unit tests for creating:

  • deterministic platform agnostic snapshots of all of the consensus state required to bootstrap a node at a given head block
    • these snapshots are taken of the HEAD state of the chain BUT are loaded as if irreversible so, snapshots should only be used if the head block they represent become irreversible until irreversible read mode is available.
    • these snapshots can be loaded optionally replaying blocks from a block.log if present and newer than the snapshot
    • the node may continue on perpetually but cannot act as a source of syncing for blocks which are not present in the block.log (it is valid to provide a full block log and a snapshot to just skip the time it takes to build up the state of the chain)
    • the snapshot can be binary or JSON encoded (but its really quite large most of the time)
  • a deterministic platform agnostic integrity hash of the entire chain state (comparable between nodes to check derived state).

Things left to do:

  • RPC command to drop the next convenient snapshot
  • application level versioning
  • net_plugin review to make sure incomplete block logs don't break anything critical

instructions for using snapshots

configuring a snapshot directory

By default snapshots are written to the snapshots directory relative to your nodeos data directory. This can be overridden with a relative or absolute path using the snapshots_dir configuration or using the --snapshots_dir command line option.

Snapshots are written out to files named with the pattern snapshot-<head block id in hex>.bin

Triggering the creation of a snapshot

Snapshots can be triggered at runtime using the RPC available through the producer_api_plugin. For example:

$ curl http://127.0.0.1:8888/v1/producer/create_snapshot | json_pp
{
   "snapshot_name" : "<data-dir>/snapshots/snapshot-0000002091fd7cb9f1656d25d3a20a93c8aff044c5de19d7cf6f9bab094e3a56.bin",
   "head_block_id" : "0000002091fd7cb9f1656d25d3a20a93c8aff044c5de19d7cf6f9bab094e3a56"
}

Instantiating a node from a snapshot

When starting a new instance of nodeos the --snapshot <path to snapshot> command line option can be used instead of replaying a blocks.log or resyncing from a network. If a blocks.log is provided it MUST at least contain blocks up to the snapshotted block and MAY contain additional blocks that will be applied as part of startup. In addition, any available reversible/ blocks will be applied.

When instantiating a node from a snapshot, it is illegal to pass in any --genesis-* arguments as that information is loaded from a snapshot. If a blocks.log exists, the genesis information it contains will be validated against the genesis data in the snapshot and error if they are not consistent.

NOTE instantiating a node from a snapshot without a blocks.log is valid BUT it will create a partial blocks.log which will affect its ability to service RPC/P2P requests for block data.

Instructions for using integrity hashes

Integrity hashes are a way of comparing the contents of the blockchain state database. They consist of a sha256 hash of a deterministic binary representation of all consensus affecting state.

calculating an integrity hash

Integrity hashes can be retrieved at runtime using the RPC available through the producer_api_plugin. For example:

$ curl http://127.0.0.1:8888/v1/producer/get_integrity_hash | json_pp
{
   "head_block_id" : "00000503289f5f9acd8a8f880b6d91d09817e6806e8f428c6081634a99f3e923",
   "integrity_hash" : "f6c9d3d088e6ceaf87af5e52f8b210a78caf8b831c3b8e890c96c0e6edf734c9"
}

wanderingbort added some commits Aug 24, 2018

add basic snapshot test, modify block log to support partial logs, fi…
…x several bugs that result from that concept
add contract state manipulation to the test, add test for replaying f…
…rom a snapshot, fix bugs that bubbled up as a result
add validation to the snapshot reader abstraction, hide more implemen…
…tation details in source files or detail namespaces
add RPC in producer_plugin_api to trigger the creation of a snapshot …
…or return the integrity hash over the database
@wanderingbort

This comment has been minimized.

Contributor

wanderingbort commented Oct 10, 2018

@dskvr please see the documentation above and comment on any concerns 😄

my->block_stream.write((char*)&totem, sizeof(totem));
if (first_block) {
auto ret = append(first_block);

This comment has been minimized.

@heifner

heifner Oct 11, 2018

Contributor

I would think ret would be warning with some compilers.. unused.

@heifner heifner referenced this pull request Oct 12, 2018

Closed

Snapshot plugin #4837

wanderingbort added some commits Oct 12, 2018

Fixes for foreign keys in the contracts tables
* contract tables are now in their own index set
* this index set is snapshotted and included in the integrity check through a different process
* traversal of the normal index set will include the table_id_objects
* traversal of the contracts tables is then free to use those objects to traverse tables in the order of the table_id_objects
  * this will traverse logical tables instead of the whole combined table
  * each logical table gets its own section and is ordered by the next most appropriate key depending on its type
@wanderingbort

This comment has been minimized.

Contributor

wanderingbort commented Oct 15, 2018

The original version of the snapshot was not safe WRT foreign keys, The second version of the snapshot was really slow. This represents a 3rd attempt at foreign key support that is performant. Smoke tests on existing chains are coming back positive.

@wanderingbort

This comment has been minimized.

Contributor

wanderingbort commented Oct 15, 2018

depends on EOSIO/fc#29

wanderingbort added some commits Oct 15, 2018

add basic application level versioning so that the structure of the s…
…napshot datafile and the structure of the snapshotted data are versioned separately

@wanderingbort wanderingbort added this to the Version 1.4 milestone Oct 16, 2018

@wanderingbort wanderingbort changed the title from [WIP] deterministic snapshots to deterministic snapshots Oct 16, 2018

@wanderingbort wanderingbort merged commit 1520889 into develop Oct 16, 2018

20 checks passed

buildkite/eosio Build #4841 passed (35 minutes, 33 seconds)
Details
buildkite/eosio/aws-build Passed (9 minutes, 47 seconds)
Details
buildkite/eosio/aws-np-tests Passed (10 minutes, 57 seconds)
Details
buildkite/eosio/aws-tests Passed (7 minutes, 9 seconds)
Details
buildkite/eosio/centos-build Passed (12 minutes, 32 seconds)
Details
buildkite/eosio/centos-np-tests Passed (10 minutes, 28 seconds)
Details
buildkite/eosio/centos-tests Passed (7 minutes, 25 seconds)
Details
buildkite/eosio/darwin-build Passed (11 minutes, 6 seconds)
Details
buildkite/eosio/darwin-np-tests Passed (22 minutes, 43 seconds)
Details
buildkite/eosio/darwin-tests Passed (9 minutes, 40 seconds)
Details
buildkite/eosio/fedora-build Passed (10 minutes, 20 seconds)
Details
buildkite/eosio/fedora-np-tests Passed (9 minutes, 56 seconds)
Details
buildkite/eosio/fedora-tests Passed (6 minutes, 56 seconds)
Details
buildkite/eosio/pipeline Passed (4 seconds)
Details
buildkite/eosio/ubuntu-18-dot-04-build Passed (8 minutes)
Details
buildkite/eosio/ubuntu-18-dot-04-np-tests Passed (10 minutes, 19 seconds)
Details
buildkite/eosio/ubuntu-18-dot-04-tests Passed (7 minutes, 8 seconds)
Details
buildkite/eosio/ubuntu-build Passed (9 minutes, 21 seconds)
Details
buildkite/eosio/ubuntu-np-tests Passed (10 minutes, 7 seconds)
Details
buildkite/eosio/ubuntu-tests Passed (7 minutes, 34 seconds)
Details

@wanderingbort wanderingbort deleted the feature/deterministic-snapshot branch Oct 16, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment