Spv proof #1207

Milerius · 2022-02-08T05:33:26Z

Please check if the PR fulfills these requirements

Tests for the changes have been added (for bug fixes / features)

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)

spv_validation crate with validation functions for headers and transaction
parameters for block header storage/validation (difficulty check, constant difficulty, nb blocks to verify)
update toolchain to nightly-2022-02-01
implement SQL storage for block headers
implement an empty IndexedDB storage for block headers
implement a function to retrieve a block header either from network or storage, add to storage/validate headers if required
implement a loop/task for downloading last block headers every N minutes (configurable)
some cargo clippy fixing due to new toolchain update
new dependencies: bitcoin-spv

What is the current behavior? (You can also link to an open issue here)

No spv validation in validate_payment

What is the new behavior (if this is a feature change)?

Add an spv validation in validate_payment

Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)

No, everything is configurable and ignored if not configured.

Other information:

The new function validate_spv return immediatly Ok in native mode.
There is dependencies update.
The update of the toolchain is required because of Stabilise is_aarch64_feature_detected! under simd_aarch64 feature rust-lang/stdarch#1239
There was some false positive from clippy regarding borrow & usage, I've used clippy::allow to bypass them.
Validation inspired by btc-relay and bitcoin-spv

…into spv_proof

- native client should not return error on validate_spv_proof since there is no verification

…dule

… later

…into spv_proof

…nit test

…into spv_proof

sergeyboyko0791

Thanks for the fixes. Last comments :)

mm2src/coins/eth/eth_tests.rs

mm2src/coins/utxo.rs

sergeyboyko0791 · 2022-03-15T06:50:28Z

mm2src/coins/utxo/utxo_builder/utxo_coin_builder.rs

@@ -315,6 +320,11 @@ pub trait UtxoCoinBuilderCommonOps {

    fn ticker(&self) -> &str;

+    fn block_headers_storage(&self) -> Option<BlockHeaderStorage> {
+        let params = json::from_value(self.conf()["block_header_params"].clone()).ok()?;


Could you please return an error if we couldn't parse block_header_params?

fn block_headers_storage(&self) -> UtxoCoinBuildResult <Option<BlockHeaderStorage>> { let params: Option<_> = json::from_value(self.conf()["block_header_params"].clone())?; match params { Some(params) => BlockHeaderStorage::new_from_ctx(self.ctx().clone(), params), None => Ok(None) } }

artemii235 · 2022-03-16T07:41:23Z

@Milerius Is it ready for the next review iteration?

Milerius · 2022-03-16T07:48:10Z

@Milerius Is it ready for the next review iteration?

I'm currently checking on #1207 (comment) other than that yes.

edit: #1207 (comment) I confirm that we need to keep an Option for the block header storage, every coins doesn't need a storage for block headers, it depends if the option is set or not in the conf file - ready for review.

artemii235

Next review iteration.

mm2src/coins/utxo/utxo_block_header_storage.rs

mm2src/coins/utxo.rs

mm2src/coins/utxo/slp.rs

mm2src/coins/utxo/utxo_common.rs

mm2src/coins/utxo/utxo_builder/utxo_arc_builder.rs

sergeyboyko0791

Next review iteration

mm2src/coins/utxo/utxo_block_header_storage.rs

mm2src/coins/utxo/utxo_builder/utxo_arc_builder.rs

sergeyboyko0791

Another comment :)

mm2src/coins/utxo/utxo_block_header_storage.rs

artemii235

Few minor code improvements proposals.

mm2src/mm2_bitcoin/spv_validation/src/helpers_validation.rs

mm2src/mm2_bitcoin/spv_validation/src/spv_proof.rs

mm2src/mm2_bitcoin/spv_validation/src/helpers_validation.rs

artemii235

🔥 Great work!

artemii235 · 2022-03-22T10:12:07Z

@shamardy @sergeyboyko0791 Please check if all your comments are resolved.

shamardy

🔥

sergeyboyko0791

LGTM!

Milerius added 30 commits January 16, 2022 08:24

feat(unit_tests): add begin of unit test for spv proof

b53c03d

feat(utxo): continue test

8b1ccc5

feat(unit_tests): very wip

b175b74

feat(utxo): merkle tree verification now work; need full refactoring

3c122b1

feat(spv): start spv validation module

971503a

feat(spv): add spv proof

6570e07

feat(spv): add merkle proof unit test for a single element

da81a73

feat(spv): add complex merkle proof inclusion unit test

c888b42

feat(spv): fix some cargo warnings

a97f4d1

Merge branch 'dev' of https://github.com/KomodoPlatform/atomicDEX-API …

f9f9708

…into spv_proof

feat(spv): simplify error check for validate_spv_proof

7f09afa

- native client should not return error on validate_spv_proof since there is no verification

feat(spv): complete the first step of the spv proof validation

b43aed6

feat(spv): complete the unit test for spv proof validation in utxo mo…

8265073

…dule

feat(spv_validation): add vin and vout check for spv proof validation

9176fd1

feat(spv_validation): match exact error from the spv - will customize…

e80eafe

… later

Merge branch 'dev' of https://github.com/KomodoPlatform/atomicDEX-API …

a1940c1

…into spv_proof

Merge branch 'dev' of https://github.com/KomodoPlatform/atomicDEX-API …

1e405bb

…into spv_proof

feat(utxo): start utxo block header storage + sync with dev

db324f7

feat(utxo): continue block header storage interface

5fd7d6a

feat(utxo): revert sled - will switch to sqllite

773b27b

feat(utxo): start utxo sql block header storage trait implementation

b3b8750

feat(sql): implement partially add_block_headers_to_storage and the u…

f3728f8

…nit test

feat(sql): add get_block header functions

5c408a8

feat(sql): add extra function for block insertion

d880ff8

feat(lib_bitcoin): add raw header type

b62ab5a

feat(spv): Integrate raw block header into spv validation

3710ea7

feat(spv): remove old comments

0a0eb47

feat(spv): Adding raw header into utxo_common spv validation

deba4d0

Merge branch 'dev' of https://github.com/KomodoPlatform/atomicDEX-API …

5028d08

…into spv_proof

feat(spv): make slp great again

6bd7c36

Milerius added 3 commits March 11, 2022 09:11

feat(spv): fix unit test compilation

0e38525

feat(eth): ignore polygon unit test (unstable)

ed97f35

feat(unit tests): increase to 6 seconds for the docker unit test

6621d5b

sergeyboyko0791 reviewed Mar 15, 2022

View reviewed changes

feat(utxo): simplify block_headers_storage, add error

27a4e8b

Milerius requested a review from artemii235 March 16, 2022 09:37

artemii235 requested changes Mar 17, 2022

View reviewed changes

Milerius added 4 commits March 18, 2022 08:25

feat(review): batch of review fixes

863fcc2

feat(review): another batch of review fixes

0d09094

feat(eth_test): remove the loop

5d19724

feat(eth_test): remove the loop

b071fcd

sergeyboyko0791 suggested changes Mar 21, 2022

View reviewed changes

mm2src/coins/utxo/utxo_block_header_storage.rs Outdated Show resolved Hide resolved

mm2src/coins/utxo/utxo_builder/utxo_arc_builder.rs Outdated Show resolved Hide resolved

sergeyboyko0791 suggested changes Mar 21, 2022

View reviewed changes

mm2src/coins/utxo/utxo_block_header_storage.rs Outdated Show resolved Hide resolved

Milerius added 2 commits March 21, 2022 06:46

feat(spv_proof): review fixes

3beaf24

feat(dev): sync with dev

9c50442

artemii235 requested changes Mar 21, 2022

View reviewed changes

feat(spv_proof): review fixes

b5cadc5

artemii235 previously approved these changes Mar 22, 2022

View reviewed changes

shamardy previously approved these changes Mar 22, 2022

View reviewed changes

feat(spv): fix wrong copy paste description

4ff3f4f

Milerius dismissed stale reviews from shamardy and artemii235 via 4ff3f4f March 23, 2022 10:13

sergeyboyko0791 approved these changes Mar 23, 2022

View reviewed changes

artemii235 merged commit 03b0910 into dev Mar 23, 2022

artemii235 deleted the spv_proof branch March 23, 2022 11:33

Milerius mentioned this pull request Mar 28, 2022

fix(firo_header): add support for prog pow #1234

Closed

shamardy mentioned this pull request Jul 18, 2022

[r2r] Lightning batch transactions confirmations, spv (wip), refactors #1339

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Spv proof #1207

Spv proof #1207

Milerius commented Feb 8, 2022 •

edited by sergeyboyko0791

sergeyboyko0791 left a comment

sergeyboyko0791 Mar 15, 2022

artemii235 commented Mar 16, 2022

Milerius commented Mar 16, 2022 •

edited

artemii235 left a comment

sergeyboyko0791 left a comment

sergeyboyko0791 left a comment

artemii235 left a comment

artemii235 left a comment

artemii235 commented Mar 22, 2022

shamardy left a comment

sergeyboyko0791 left a comment

Spv proof #1207

Spv proof #1207

Conversation

Milerius commented Feb 8, 2022 • edited by sergeyboyko0791

sergeyboyko0791 left a comment

Choose a reason for hiding this comment

sergeyboyko0791 Mar 15, 2022

Choose a reason for hiding this comment

artemii235 commented Mar 16, 2022

Milerius commented Mar 16, 2022 • edited

artemii235 left a comment

Choose a reason for hiding this comment

sergeyboyko0791 left a comment

Choose a reason for hiding this comment

sergeyboyko0791 left a comment

Choose a reason for hiding this comment

artemii235 left a comment

Choose a reason for hiding this comment

artemii235 left a comment

Choose a reason for hiding this comment

artemii235 commented Mar 22, 2022

shamardy left a comment

Choose a reason for hiding this comment

sergeyboyko0791 left a comment

Choose a reason for hiding this comment

Milerius commented Feb 8, 2022 •

edited by sergeyboyko0791

Milerius commented Mar 16, 2022 •

edited