Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Light Client refactoring #237

Merged
merged 102 commits into from
May 27, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
102 commits
Select commit Hold shift + click to select a range
cceaa70
Rework predicates
romac Apr 23, 2020
a982821
WIP: Add tracing
romac Apr 23, 2020
7e2a9d5
Fix verification procedure
romac Apr 24, 2020
8509a13
Rename requester component to rpc
romac Apr 24, 2020
cdc2294
Rename Trace::run to Trace::collect
romac Apr 24, 2020
be3012f
Return meaningful data in errors
romac Apr 24, 2020
6c8a011
Proper error handling with thiserror+anomaly
romac Apr 24, 2020
333afd5
Make events PartialEq+Eq
romac Apr 24, 2020
93dbe53
Implement verifier
romac Apr 24, 2020
1f325e0
Implement scheduler and bisection
romac Apr 24, 2020
93b148b
Remove write access to trusted store for scheduler
romac Apr 24, 2020
614c870
Add a couple of FIXMEs
romac Apr 24, 2020
ded21c9
Formatting
romac Apr 24, 2020
745c001
Fix clippy warnings
romac Apr 24, 2020
0969b7b
Fix misplaced attribute
romac Apr 24, 2020
5948c7b
Enable VerificationPredicates to be made into a trait object
romac Apr 27, 2020
0661ee9
Allow cloning TSReader
romac Apr 27, 2020
de049fb
Shorter method name
romac Apr 27, 2020
799caaa
Decouple components using Router trait
romac Apr 28, 2020
79b26c6
Silence a couple Clippy warnings
romac Apr 28, 2020
66ef29f
Cleanup trace module
romac Apr 28, 2020
ede2b76
Revamp errors
romac Apr 28, 2020
884eebd
Revamp error, part 2
romac Apr 28, 2020
af46005
Bundle verification options together
romac Apr 28, 2020
fcb8c82
Cleanup
romac Apr 28, 2020
18f22ac
Use output enum for all components
romac Apr 28, 2020
e1158a8
Split queries out
romac Apr 29, 2020
a50286f
Rewrite using coroutines
romac Apr 30, 2020
3fd1d2a
cleanup
romac Apr 30, 2020
e309862
Add fork detector prototype
romac Apr 30, 2020
64a322f
Add stub example
romac Apr 30, 2020
0ce586c
Add traits to abstract of each concrete component
romac Apr 30, 2020
eca7872
Add actual commit to Commit struct
romac Apr 30, 2020
4c9ea47
Refactor and simplify
romac Apr 30, 2020
51f6939
Implement Store::latest
romac May 1, 2020
e789f8b
Better verification loop
romac May 1, 2020
cab2d44
Add pre/post conditions to demuxer::verify
romac May 1, 2020
a77f1fc
Add contract for schedule
romac May 1, 2020
2b80f14
Convert between tendermint and spike types
romac May 1, 2020
811500f
Working example
romac May 1, 2020
8f7cff0
Better working version
romac May 1, 2020
2173ec8
Implement production header hasher
romac May 4, 2020
dc1e104
Re-add proper Error type for whole client
romac May 5, 2020
d3ffe43
Cleanup
romac May 5, 2020
e43aca3
Add peers to demuxer state
romac May 5, 2020
c27ec46
Cleanup
romac May 5, 2020
bd9971c
Trace blocks needed for verification of a target block
romac May 5, 2020
2ffb7da
Split validation and trust check into their own top-level predicates
romac May 6, 2020
417be25
Add provider to LightBlock struct
romac May 6, 2020
2fa229b
Split overlap verification into its own verifier input
romac May 6, 2020
bef36ab
Cleanup
romac May 6, 2020
c31e5fa
Don't mix verifier and scheduler concerns
romac May 7, 2020
8d0db51
Validate commits and compute actual voting power
romac May 7, 2020
eaa4abf
Cleanup
romac May 7, 2020
f9817e8
Remove scheduler events
romac May 7, 2020
a23b2d9
Remove verifier events
romac May 7, 2020
63c6147
Remove fork detector events
romac May 7, 2020
c1935ec
Remove IO events
romac May 7, 2020
d5e7ca2
Cleanup
romac May 7, 2020
3c19d22
Simplify code flow by using an iterator of highest trusted states
romac May 7, 2020
40900e9
Fix example
romac May 7, 2020
772a16f
Update example to prod implementations
romac May 7, 2020
2b5de61
Stop verification when reaching already trusted state
romac May 7, 2020
5b668cb
Use height of fetched header to fetch validators (avoids issues when …
romac May 7, 2020
6aa6baf
Allow tracing same block, just ignore it
romac May 7, 2020
38d1e01
Better error reporting
romac May 7, 2020
fd94323
Fix bug in is_monotonic_height
romac May 7, 2020
9ce271b
Shorten ProductionPredicates name
romac May 7, 2020
7fb8f32
Cleanup
romac May 11, 2020
cec952f
Port over single-step tests
romac May 11, 2020
ceae1df
Port bisection tests
romac May 11, 2020
483077d
Move test types into their own module
romac May 12, 2020
29edfc0
Refactor
romac May 14, 2020
2485f76
Fix bug in validators overlap check
romac May 14, 2020
c462e05
Refactor LightStore, and introduce proper contracts
romac May 15, 2020
0853896
Cleanup
romac May 15, 2020
3a71fad
Simply LightStore trait
romac May 18, 2020
742441f
Use tendermint::node::Id as PeerId, as per the spec
romac May 18, 2020
e978af0
Add LightStore::update method
romac May 18, 2020
5605ef7
Fix clippy warnings
romac May 18, 2020
489cfbd
Extract get_or_fetch_block method from demuxer loop
romac May 18, 2020
c915d0a
Rename Demuxer to LightClient
romac May 18, 2020
202bdc1
Rename LightClientOptions to Options
romac May 18, 2020
0988f61
Implement on-disk store backed by Sled
romac May 19, 2020
3602b08
Formatting
romac May 19, 2020
56c020d
Cleanup
romac May 19, 2020
076932f
Properly implement SledStore::update
romac May 19, 2020
6b70979
Cleanup
romac May 20, 2020
e8b23fc
Add LightClient CLI to continuously pull headers from a node
romac May 20, 2020
6777956
Fix tests
romac May 25, 2020
8bbb194
Merge branch 'master' into romac/light-spike-bis
romac May 25, 2020
948ecb4
Adapt test to new JSON files organization
romac May 25, 2020
a17beb7
Rename light-spike crate to light-client
romac May 26, 2020
5323bb8
Turn production predicates into default trait impl
romac May 26, 2020
e287152
Comment out provider field of LightBlock until conformance tests are …
romac May 26, 2020
3ecdec2
Refactor is_within_trust_period to better match the spec
romac May 26, 2020
4934cbc
Add core verification loop invariant
romac May 26, 2020
862ff72
WIP: Documentation
romac May 26, 2020
7b39443
Merge branch 'master' into romac/light-spike-bis
romac May 26, 2020
5f3f914
Make cargo fmt happy
romac May 26, 2020
97e27b1
Make clippy happy
romac May 26, 2020
1f12e72
Re-enable `provider` field in LightBlock struct
romac May 27, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 1 addition & 6 deletions light-client/src/components/io.rs
Original file line number Diff line number Diff line change
Expand Up @@ -53,12 +53,7 @@ impl Io for ProdIo {
let validator_set = self.fetch_validator_set(peer, height)?;
let next_validator_set = self.fetch_validator_set(peer, height + 1)?;

let light_block = LightBlock::new(
signed_header,
validator_set,
next_validator_set,
// peer
);
let light_block = LightBlock::new(signed_header, validator_set, next_validator_set, peer);

Ok(light_block)
}
Expand Down
90 changes: 82 additions & 8 deletions light-client/src/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,16 @@ use tendermint::block::Height as HeightStr;
use tendermint::evidence::Duration as DurationStr;

#[derive(Deserialize, Clone, Debug)]
pub struct TestCases {
pub struct TestCases<LB> {
pub batch_name: String,
pub test_cases: Vec<TestCase>,
pub test_cases: Vec<TestCase<LB>>,
}

#[derive(Deserialize, Clone, Debug)]
pub struct TestCase {
pub struct TestCase<LB> {
pub description: String,
pub initial: Initial,
pub input: Vec<LightBlock>,
pub input: Vec<LB>,
pub expected_output: Option<String>,
}

Expand All @@ -28,20 +28,20 @@ pub struct Initial {
}

#[derive(Deserialize, Clone, Debug)]
pub struct TestBisection {
pub struct TestBisection<LB> {
pub description: String,
pub trust_options: TrustOptions,
pub primary: Provider,
pub primary: Provider<LB>,
pub height_to_verify: HeightStr,
pub now: Time,
pub expected_output: Option<String>,
pub expected_num_of_bisections: usize,
}

#[derive(Deserialize, Clone, Debug)]
pub struct Provider {
pub struct Provider<LB> {
pub chain_id: String,
pub lite_blocks: Vec<LightBlock>,
pub lite_blocks: Vec<LB>,
}

#[derive(Deserialize, Clone, Debug)]
Expand All @@ -66,3 +66,77 @@ impl Trusted {
}
}
}

// -----------------------------------------------------------------------------
// Everything below is a temporary workaround for the lack of `provider` field
// in the light blocks serialized in the JSON fixtures.
// -----------------------------------------------------------------------------
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @greg-szabo @Shivani912 this is relevant to testing / serialization

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to be implementation specific to me. Go code has a different approach to identify a node as a primary/secondary provider and does not deal with addresses. But also, this approach looks cleaner and reduces a lot of data from test files. Will need some more work around testing of Go code though. Need more thoughts here! @greg-szabo @liamsi @melekes


#[derive(Clone, Debug, PartialEq, Deserialize)]
pub struct AnonLightBlock {
pub signed_header: SignedHeader,
#[serde(rename = "validator_set")]
pub validators: ValidatorSet,
#[serde(rename = "next_validator_set")]
pub next_validators: ValidatorSet,
#[serde(default = "default_peer_id")]
pub provider: PeerId,
}

pub fn default_peer_id() -> PeerId {
"BADFADAD0BEFEEDC0C0ADEADBEEFC0FFEEFACADE".parse().unwrap()
}

impl From<AnonLightBlock> for LightBlock {
fn from(alb: AnonLightBlock) -> Self {
Self {
signed_header: alb.signed_header,
validators: alb.validators,
next_validators: alb.next_validators,
provider: alb.provider,
}
}
}

impl From<TestCase<AnonLightBlock>> for TestCase<LightBlock> {
fn from(tc: TestCase<AnonLightBlock>) -> Self {
Self {
description: tc.description,
initial: tc.initial,
input: tc.input.into_iter().map(Into::into).collect(),
expected_output: tc.expected_output,
}
}
}

impl From<TestCases<AnonLightBlock>> for TestCases<LightBlock> {
fn from(tc: TestCases<AnonLightBlock>) -> Self {
Self {
batch_name: tc.batch_name,
test_cases: tc.test_cases.into_iter().map(Into::into).collect(),
}
}
}

impl From<Provider<AnonLightBlock>> for Provider<LightBlock> {
fn from(p: Provider<AnonLightBlock>) -> Self {
Self {
chain_id: p.chain_id,
lite_blocks: p.lite_blocks.into_iter().map(Into::into).collect(),
}
}
}

impl From<TestBisection<AnonLightBlock>> for TestBisection<LightBlock> {
fn from(tb: TestBisection<AnonLightBlock>) -> Self {
Self {
description: tb.description,
trust_options: tb.trust_options,
primary: tb.primary.into(),
height_to_verify: tb.height_to_verify,
now: tb.now,
expected_output: tb.expected_output,
expected_num_of_bisections: tb.expected_num_of_bisections,
}
}
}
9 changes: 4 additions & 5 deletions light-client/src/types.rs
Original file line number Diff line number Diff line change
Expand Up @@ -51,9 +51,8 @@ pub struct LightBlock {
/// Validator set at the next block height
#[serde(rename = "next_validator_set")]
pub next_validators: ValidatorSet,
// FIXME: Uncomment when conformance tests are adapted to include provider
// /// The peer ID of the node that provided this block
// pub provider: PeerId,
/// The peer ID of the node that provided this block
pub provider: PeerId,
}

impl LightBlock {
Expand All @@ -62,13 +61,13 @@ impl LightBlock {
signed_header: SignedHeader,
validators: ValidatorSet,
next_validators: ValidatorSet,
// provider: PeerId,
provider: PeerId,
) -> LightBlock {
Self {
signed_header,
validators,
next_validators,
// provider,
provider,
}
}

Expand Down
23 changes: 12 additions & 11 deletions light-client/tests/light_client.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,6 @@ use std::path::{Path, PathBuf};

use tendermint::rpc;

const PEER_ID: &str = "BADFADAD0BEFEEDC0C0ADEADBEEFC0FFEEFACADE";

// Link to the commit that generated below JSON test files:
// https://github.com/Shivani912/tendermint/commit/e02f8fd54a278f0192353e54b84a027c8fe31c1e
const TEST_FILES_PATH: &str = "./tests/support/";
Expand All @@ -36,7 +34,7 @@ fn verify_single(
trusted_state.signed_header,
trusted_state.next_validators.clone(),
trusted_state.next_validators,
// PEER_ID.parse().unwrap(),
default_peer_id(),
);

let options = Options {
Expand All @@ -56,7 +54,7 @@ fn verify_single(
}
}

fn run_test_case(tc: TestCase) {
fn run_test_case(tc: TestCase<LightBlock>) {
let mut latest_trusted = Trusted::new(
tc.initial.signed_header.clone(),
tc.initial.next_validator_set.clone(),
Expand Down Expand Up @@ -145,11 +143,10 @@ fn verify_bisection(
.map(|_| light_client.get_trace(untrusted_height))
}

fn run_bisection_test(tc: TestBisection) {
fn run_bisection_test(tc: TestBisection<LightBlock>) {
println!(" - {}", tc.description);

let primary: PeerId = PEER_ID.parse().unwrap();

let primary = default_peer_id();
let untrusted_height = tc.height_to_verify.try_into().unwrap();
let trust_threshold = tc.trust_options.trust_level;
let trusting_period = tc.trust_options.period;
Expand Down Expand Up @@ -275,12 +272,16 @@ fn run_bisection_tests(dir: &str) {
}
}

fn read_test_case(file_path: &str) -> TestCase {
serde_json::from_str(read_json_fixture(file_path).as_str()).unwrap()
fn read_test_case(file_path: &str) -> TestCase<LightBlock> {
let tc: TestCase<AnonLightBlock> =
serde_json::from_str(read_json_fixture(file_path).as_str()).unwrap();
tc.into()
}

fn read_bisection_test_case(file_path: &str) -> TestBisection {
serde_json::from_str(read_json_fixture(file_path).as_str()).unwrap()
fn read_bisection_test_case(file_path: &str) -> TestBisection<LightBlock> {
let tc: TestBisection<AnonLightBlock> =
serde_json::from_str(read_json_fixture(file_path).as_str()).unwrap();
tc.into()
}

#[test]
Expand Down