This repository has been archived by the owner on Jun 11, 2022. It is now read-only.
Upstreaming OBFT stuff #693
Closed
Closed
Changes from 134 commits
Commits
Show all changes
137 commits
Select commit
Hold shift + click to select a range
922115a
Created new handlers `height` and `block_by_height` for the bridge, a…
vsubhuman becb365
Implemented `get_slotid` and `get_epoch_and_slot` for block date
vsubhuman 4f62091
Replace `BTreeMap` with `LinkedHashMap` to iterate lookups in inserti…
vsubhuman 7a5948e
Implemented the `block_location_by_height` function
vsubhuman 29aa5cf
Added parameter `ordinal` to `Lookup` and implemented storing epoch-n…
vsubhuman f3c474f
Added storing unsorted offsets additionally in index files, and imple…
vsubhuman b8f2e12
Implemented new enum `IndexOffsetType` to reduce code duplication
vsubhuman 3c1b6d5
Removed new function from deprecated type and renamed new function ac…
vsubhuman d044816
Removed `IndexOffsetType` logic and implemented storing unsorted offs…
vsubhuman b84f9cc
Implemented new type `BlockLocation::Offset` and split-up some logic …
vsubhuman e465d9c
Implemented `storage::epoch::epoch_read_block_offset` which reads N-t…
vsubhuman 559daf1
Added new field `Storage.loose_idx` and implemented `Storage.build_lo…
vsubhuman 25d5b33
Implemented functions to work with loose index and also added syncing…
vsubhuman 3f6edd2
Extended `Storage.block_location_by_height` to check loose index first
vsubhuman 217d99e
Fixed reference
vsubhuman a990857
Removed unnecessary `pub`s
vsubhuman 85dabea
Fixed Iterator type spec in `write_offsets_to_file` function
vsubhuman c351032
Made `ChainDifficulty` great again. (With private `u64` field)
vsubhuman 6577089
Made `FanoutTotal` great again. (With private `u32` field)
vsubhuman 120949c
Fixed function naming to Rust conventions
vsubhuman 76a3f47
Replaced `match` with `if let`
vsubhuman 380f063
Fixed loose index initial construction order, and added checks for ne…
vsubhuman 4fa16ff
Simplified integer casting in `block_location_by_height`
vsubhuman 46d0b27
Updated `get_from_loose_index` to handle the case when requested heig…
vsubhuman 89aadc3
Simplified match case in `block_location_by_height` to work with pote…
vsubhuman 650544b
Changed asserts as loose index construction\update to validate it doe…
vsubhuman 63c7472
Prettified some code working with `storage.loose_idx`
vsubhuman 7142cb7
Fixed loose_idx entry destruct
vsubhuman b5dbbf2
Simplified `block_location_by_height` using the magic `?`
vsubhuman ae06de4
Simplified `block_location_by_height` even more with use of magic `if…
vsubhuman 6c586b1
Removed use of `ordinal` in index and removed use of `Lookups`. Imple…
vsubhuman c4d3a01
Fixed styling errors
vsubhuman ea69bdb
Fixed storage-locking
vsubhuman ed17614
Merge remote-tracking branch 'remotes/upstream/prop-up-leaning-tower'…
vsubhuman 152c3e9
Changed the terminal case check in `sync::get_unpacked_blocks_in_epoc…
vsubhuman 998436b
Added new OBFT-specific comments to the `sync::get_unpacked_blocks_in…
vsubhuman 1030315
Changed the epoch-number validation code in `verify_chain::ChainState…
vsubhuman 2998e8c
Changed the epoch-number validation if-else in `verify_chain::ChainSt…
vsubhuman 8f9e98f
Reworked the epoch-number validation if-else in `verify_chain::ChainS…
vsubhuman 1ba3b3c
Added new field `chain_state::ChainState::last_boundary_block_epoch` …
vsubhuman 0e2101f
Fixed last EBB check in `verify_chain::ChainState::verify_block`
vsubhuman d21ed19
Updated loop in `sync::net_sync_to` to be ready to epochs with no EBB
vsubhuman 8a40915
Updated epoch-switch in `sync::net_sync_to` in `net.get_blocks`-callb…
vsubhuman 19e85fe
Updated code in `sync::net_sync_to` in `net.get_blocks`-callback to c…
vsubhuman 8f7aab3
Fixed failing checks
vsubhuman ce705a9
Added `.clone()` to value
vsubhuman 7b8efe8
Merge remote-tracking branch 'remotes/upstream/prop-up-leaning-tower'…
vsubhuman ed71ab0
Introduced enum flags for callbacks to be able to continue or stop th…
vsubhuman 778eeda
Changed `get_blocks` callback in `sync.rs` so it first validates the …
vsubhuman 3c191bd
Fixed compilation problems
vsubhuman 167b73a
Fixed style problems
vsubhuman 85383cf
Added comment about possible rollback types
vsubhuman 2a3d00a
Implemented rollback detection in two possible places
vsubhuman 070d480
Merge remote-tracking branch 'remotes/origin/feature/block-by-height-…
vsubhuman 9477cd2
Implemented rollback for case when more thank K loose blocks
vsubhuman a3cc46b
Implemented rollback for loose tail case
vsubhuman 5a83de5
Extracted some common file-reading code in `epoch.rs`
vsubhuman aecb6e4
Implemented rollback for case when we are in packed tail
vsubhuman c4e91d8
Changed the terminal case check in `sync::get_unpacked_blocks_in_epoc…
vsubhuman cc3c1a8
Added new OBFT-specific comments to the `sync::get_unpacked_blocks_in…
vsubhuman 158750b
Changed the epoch-number validation code in `verify_chain::ChainState…
vsubhuman b7ec9ba
Changed the epoch-number validation if-else in `verify_chain::ChainSt…
vsubhuman aa8220a
Reworked the epoch-number validation if-else in `verify_chain::ChainS…
vsubhuman afd9bd9
Added new field `chain_state::ChainState::last_boundary_block_epoch` …
vsubhuman fcc9b78
Fixed last EBB check in `verify_chain::ChainState::verify_block`
vsubhuman 7be8a08
Updated loop in `sync::net_sync_to` to be ready to epochs with no EBB
vsubhuman 78857ac
Updated epoch-switch in `sync::net_sync_to` in `net.get_blocks`-callb…
vsubhuman 174d3ae
Updated code in `sync::net_sync_to` in `net.get_blocks`-callback to c…
vsubhuman fc6659b
Fixed failing checks
vsubhuman 1be2dae
Added `.clone()` to value
vsubhuman b249c71
Fixed style checks
vsubhuman cf3b81d
Created new handlers `height` and `block_by_height` for the bridge, a…
vsubhuman 0478b13
Implemented `get_slotid` and `get_epoch_and_slot` for block date
vsubhuman 909d5f9
Replace `BTreeMap` with `LinkedHashMap` to iterate lookups in inserti…
vsubhuman 47cdf8d
Implemented the `block_location_by_height` function
vsubhuman 704d434
Added parameter `ordinal` to `Lookup` and implemented storing epoch-n…
vsubhuman 818b296
Added storing unsorted offsets additionally in index files, and imple…
vsubhuman dc11956
Implemented new enum `IndexOffsetType` to reduce code duplication
vsubhuman 157b0fa
Removed new function from deprecated type and renamed new function ac…
vsubhuman 33aa84a
Removed `IndexOffsetType` logic and implemented storing unsorted offs…
vsubhuman c4c7c44
Implemented new type `BlockLocation::Offset` and split-up some logic …
vsubhuman 1c04ce7
Implemented `storage::epoch::epoch_read_block_offset` which reads N-t…
vsubhuman 374c951
Added new field `Storage.loose_idx` and implemented `Storage.build_lo…
vsubhuman 53def45
Implemented functions to work with loose index and also added syncing…
vsubhuman a3d1429
Extended `Storage.block_location_by_height` to check loose index first
vsubhuman f9a815e
Fixed reference
vsubhuman 0413df7
Removed unnecessary `pub`s
vsubhuman 39bb751
Fixed Iterator type spec in `write_offsets_to_file` function
vsubhuman 0e44280
Made `ChainDifficulty` great again. (With private `u64` field)
vsubhuman 6753ede
Made `FanoutTotal` great again. (With private `u32` field)
vsubhuman ddad1f9
Fixed function naming to Rust conventions
vsubhuman 8e091a0
Replaced `match` with `if let`
vsubhuman a6485a5
Fixed loose index initial construction order, and added checks for ne…
vsubhuman 48dbf6b
Simplified integer casting in `block_location_by_height`
vsubhuman 8d878c3
Updated `get_from_loose_index` to handle the case when requested heig…
vsubhuman 8dcd6d5
Simplified match case in `block_location_by_height` to work with pote…
vsubhuman 30e6581
Changed asserts as loose index construction\update to validate it doe…
vsubhuman 02d5e94
Prettified some code working with `storage.loose_idx`
vsubhuman ae9892f
Fixed loose_idx entry destruct
vsubhuman 73c8e78
Simplified `block_location_by_height` using the magic `?`
vsubhuman 113bcb2
Simplified `block_location_by_height` even more with use of magic `if…
vsubhuman 25fe837
Removed use of `ordinal` in index and removed use of `Lookups`. Imple…
vsubhuman bcae641
Fixed styling errors
vsubhuman f9496af
Fixed storage-locking
vsubhuman dd942a0
Merge `block-by-height-2` into `rollbacks`
vsubhuman 8019fbb
Removed duplicated code
vsubhuman 8324156
Extracted `perform_rollback` function
vsubhuman ce70808
Merge remote-tracking branch 'remotes/origin/master' into feature/blo…
vsubhuman c1562f2
Fixed min/max problem. Extracted common code into a function.
vsubhuman cc0869e
Codestyle
vsubhuman 25baf6d
Merge remote-tracking branch 'remotes/origin/master' into emurgo/bloc…
vsubhuman b0b70aa
Merge remote-tracking branch 'remotes/origin/emurgo/block-by-height' …
vsubhuman ba175f9
Merge remote-tracking branch 'remotes/origin/master' into emurgo/bloc…
vsubhuman e7682e8
Merge pull request #1 from Emurgo/emurgo/block-by-height
vsubhuman 773719a
Merge pull request #2 from Emurgo/emurgo/sync-obft-ready
vsubhuman 55767a4
Merge pull request #3 from Emurgo/emurgo/rollback
vsubhuman e30d57a
Implemented `net_tip` in storage. Expanded `status` endpoint to retur…
vsubhuman 0f496ce
Merge pull request #4 from Emurgo/emurgo/net-tip
vsubhuman d78dcb0
Added OBFT-testnet genesis block and add new network config for it
vsubhuman f86732a
Fixed genesis loading and first non-genesis hash config
vsubhuman 48cead1
Fixed genesis/ebb
vsubhuman c5cb50c
Merge pull request #5 from Emurgo/obft-testnet
vsubhuman afb6092
Merge remote-tracking branch 'remotes/upstream/master' into master-em…
vsubhuman 5f01985
Implemented epoch-flags and ebb-check for packed epochs
vsubhuman 3529f60
Implemented epoch-flags and ebb-check for packed epochs
vsubhuman f387cbf
Fixed last EBB date assert
vsubhuman 5e19600
Merge pull request #6 from Emurgo/block-by-height-no-ebb
vsubhuman 5b63769
Merge pull request #7 from Emurgo/master
vsubhuman 1882284
Fix LooseChainHeightEntry::header_hash()
rooooooooob 3e827e7
Fix storage asserts regarding chain difficulty/EBB
rooooooooob 217e5ef
Merge pull request #8 from Emurgo/loose-entry-hash-fix
vsubhuman a770975
Fixed formatting
vsubhuman cb32c71
Added comment about genesis hash hack
vsubhuman a66028d
Merge pull request #10 from Emurgo/fix-fmt
vsubhuman 9355e07
Removed OBFT-testnet config and genesis hash hack
vsubhuman dfb91b9
Removed mention of obft-testnet genesis hash in sources
vsubhuman 4a04081
Fixed variable reference
vsubhuman File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -14,20 +14,34 @@ impl ChainState { | |
|
||
add_error(&mut res, self.do_verify(block_hash, blk)); | ||
|
||
self.last_block = block_hash.clone(); | ||
self.last_date = Some(blk.header().blockdate()); | ||
// FIXME: count boundary blocks as part of the chain length? | ||
self.chain_length += 1; | ||
|
||
match blk { | ||
Block::BoundaryBlock(blk) => { | ||
self.last_boundary_block = Some(block_hash.clone()); | ||
self.last_boundary_block_epoch = Some(blk.header.consensus.epoch); | ||
self.slot_leaders = blk.body.slot_leaders.clone(); | ||
} | ||
|
||
Block::MainBlock(_) => {} | ||
Block::MainBlock(blk) => { | ||
if self.last_boundary_block.is_some() { | ||
let block_epoch = blk.header.consensus.slot_id.epoch; | ||
let local_epoch = self.last_date.map(|d| d.get_epochid()).unwrap_or(0); | ||
if block_epoch > local_epoch { | ||
// We are in OBFT and switched epochs without EBB | ||
// Remove last boundary block state | ||
|
||
// TODO: should cleanup EBB state here? | ||
//self.last_boundary_block = None; | ||
//self.last_boundary_block_epoch = None; | ||
//self.slot_leaders = vec![]; | ||
} | ||
} | ||
} | ||
}; | ||
|
||
self.last_block = block_hash.clone(); | ||
self.last_date = Some(blk.header().blockdate()); | ||
// FIXME: count boundary blocks as part of the chain length? | ||
self.chain_length += 1; | ||
|
||
// Update the utxos from the transactions. | ||
if let Block::MainBlock(blk) = blk { | ||
for txaux in blk.body.tx.iter() { | ||
|
@@ -65,17 +79,21 @@ impl ChainState { | |
return Err(Error::BlockDateInPast); | ||
} | ||
|
||
// If this is a genesis block, it should be the next | ||
// epoch; otherwise it should be in the current epoch. | ||
if date.get_epochid() | ||
!= (last_date.get_epochid() + if date.is_boundary() { 1 } else { 0 }) | ||
{ | ||
return Err(Error::BlockDateInFuture); | ||
// New date should be this or next epoch, not further | ||
if date.get_epochid() > last_date.get_epochid() { | ||
if date.get_epochid() > last_date.get_epochid() + 1 { | ||
return Err(Error::BlockDateInFuture); | ||
} | ||
// First next-epoch block supposed to be EBB, | ||
// unless we are in OBFT era | ||
if !date.is_boundary() { | ||
// TODO: validate we are in OBFT? | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. TODO: gotta decide whether we need to add additional checks for proper OBFT switch (e.g. block version, or some other weirdo parameters from blocks). Right now we just allow new epochs to have no EBB, and that's it. |
||
} | ||
} | ||
} | ||
|
||
None => { | ||
if date != BlockDate::Boundary(0) { | ||
if date.get_epochid() != 0 { | ||
// FIXME: use epoch_start | ||
return Err(Error::BlockDateInFuture); | ||
} | ||
|
@@ -87,19 +105,33 @@ impl ChainState { | |
Block::BoundaryBlock(_) => {} | ||
|
||
Block::MainBlock(blk) => { | ||
let slot_id = blk.header.consensus.slot_id.slotid as usize; | ||
let epoch_id = blk.header.consensus.slot_id.epoch; | ||
let epoch_with_ebb = match &self.last_boundary_block_epoch { | ||
Some(last_ebb_epoch) => epoch_id == *last_ebb_epoch, | ||
_ => false, | ||
}; | ||
|
||
if slot_id >= self.slot_leaders.len() { | ||
return Err(Error::NonExistentSlot); | ||
} | ||
if epoch_with_ebb { | ||
// If epoch contains EBB - validate block using the leader-list | ||
|
||
let slot_id = blk.header.consensus.slot_id.slotid as usize; | ||
|
||
if slot_id >= self.slot_leaders.len() { | ||
return Err(Error::NonExistentSlot); | ||
} | ||
|
||
let slot_leader = &self.slot_leaders[slot_id]; | ||
|
||
let slot_leader = &self.slot_leaders[slot_id]; | ||
// Note: the block signature was already checked in | ||
// verify_block, so here we only check the leader key | ||
// against the genesis block. | ||
if slot_leader != &address::StakeholderId::new(&blk.header.consensus.leader_key) | ||
{ | ||
return Err(Error::WrongSlotLeader); | ||
} | ||
} else { | ||
|
||
// Note: the block signature was already checked in | ||
// verify_block, so here we only check the leader key | ||
// against the genesis block. | ||
if slot_leader != &address::StakeholderId::new(&blk.header.consensus.leader_key) { | ||
return Err(Error::WrongSlotLeader); | ||
// TODO: validate OBFT leader | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. TODO: No OBFT leader validation present. We don't have the documentation yet on how it's done. |
||
} | ||
} | ||
}; | ||
|
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TODO: gotta decide whether we need to drop "Last EBB" state here after switch to OBFT. Cuz right now it just hangs there forever.
ℹ️ Gotta remember that "Last EBB" state is used to determine whether current epoch is an EBB or not, when storing epoch packs with meta-flags.