Repository navigation
BIP 181, 182, 183: BIPs for Utreexo - #1923
kcalvinalvin wants to merge 4 commits into
Conversation
9b3eafb to
a94f643
Compare
jonatack
left a comment
There was a problem hiding this comment.
Thank you for proposing these drafts. They already look quite complete with respect to the editorial requirements (BIPs 2 and 3). I've done a cursory first pass. No immediate conceptual feedback. A few editorial comments follow; feel free to ignore them during conceptual review until they are applicable.
cb2993c to
d1d0342
Compare
|
You need to justify why you're using SHA-512/256 rather than SHA-256, like the rest of the Bitcoin protocol. Right now you just link to a paper from 2011. But that paper is out of date now that hardware support for SHA-256 has become common. |
|
I strongly recommend replacing SHA-256 with SHAKE256 (from the SHA-3 standard) for the following reasons: 1. Security Advantages
2. Comparative Analysis: SHA-256 vs SHAKE256
3. Functional ExampleInput: SHAKE256 (512-bit output): SHAKE256 (256-bit output): 4. Implementation Benefits
5. Technical ReferenceFor detailed cryptographic differences: |
Sure we can update the accumulator BIP with benchmarks for SHA512/256 vs SHA256. But could you link to the aforementioned justifications for the other parts of the Bitcoin protocol that use SHA512? |
SHAKE256 is not used in Bitcoin and introduces a new hash which increases the trust-assumption. We do not want to do this. |
This comment was marked as off-topic.
This comment was marked as off-topic.
|
Some friendly moderation to keep the discussion focused on technical review -- thanks. |
SHA256 and SHA512 are quantum resistent.
Ok but this has nothing to do with this BIP. |
|
@1BitcoinBoWP1FZ4xwTNkq6XksKidmgYYw, please cut out the LLM generated comments. If any of us were interested in seeing an LLM’s prediction of what might be said about a topic, we could prompt one ourselves. |
|
On Mon, Aug 18, 2025 at 04:06:51AM -0700, Calvin Kim wrote:
kcalvinalvin left a comment (bitcoin/bips#1923)
> You need to justify why you're using SHA-512/256 rather than SHA-256, like the rest of the Bitcoin protocol. Right now you just link to a paper from 2011. But that paper is out of date now that hardware support for SHA-256 has become common.
Sure we can update the accumulator BIP with benchmarks for SHA512/256 vs SHA256.
But could you link to the aforementioned justifications for the other parts of the Bitcoin protocol that use SHA512?
No part of the Bitcoin consensus protocol uses SHA512.
|
Ok but you've stated in your previous comment "You need to justify why you're using SHA-512/256 rather than SHA-256, like the rest of the Bitcoin protocol". Would be very helpful to see what type of justifications the other protocols have made. Second, I don't think it matters if SHA512 wasn't used in the Bitcoin consensus protocol. SHA512 is used in BIP32 and the argument that SHA512 is safe for generating private keys but not safe for Bitcoin consensus isn't sound. I think our original justification (better performance with SHA512/256) mentioned in the BIP is sound. Happy to provide the benchmarks, they're being worked on at the moment. |
This comment was marked as abuse.
This comment was marked as abuse.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
| | Name | Type | Description | | ||
| | ----------------- | ------------------------ | ----------------------------------------- | | ||
| | Utreexo_Tag_V1 | 64 byte array | The version tag to be prepended to the leafhash. | | ||
| | Utreexo_Tag_V1 | 64 byte array | The version tag to be prepended to the leafhash. | |
There was a problem hiding this comment.
For clarification, is the Utreexo_Tag_V1 really used twice in preimage to the hash?
There was a problem hiding this comment.
My guess would be that this duplication is unintended.
| | Name | Type | Description | | |
| | ----------------- | ------------------------ | ----------------------------------------- | | |
| | Utreexo_Tag_V1 | 64 byte array | The version tag to be prepended to the leafhash. | | |
| | Utreexo_Tag_V1 | 64 byte array | The version tag to be prepended to the leafhash. | | |
| | Name | Type | Description | | |
| | ----------------- | ------------------------ | ----------------------------------------- | | |
| | Utreexo_Tag_V1 | 64 byte array | The version tag to be prepended to the leafhash. | |
There was a problem hiding this comment.
Oh no the duplication is intended.
Since we use SHA512/256 as the hash function, each chunk is 128 bytes. Since the version tag is only 64 bytes, we need two of them.
The question is 1) why are we added one new dependency to consensus implementations, and 2) is this actually a performance increase, given that dedicated SHA256 hardware is becoming common? Length-extension attacks are not relevant for this use-case as we are only committing to public data. |
Yes. There have been multiple protocol changes to BIP183 since then and I've been busy with the implementations for those changes. I'll get to the reviews and update the BIP. |
| `MSG_UTREEXO_PROOF` is all the data required for a CSN or archive node using the Utreexo accumulators to validate a Bitcoin block. | ||
|
|
||
| Its `cmdString` for P2PV1 is `uproof`. | ||
| Its [BIP324 P2PV2](https://github.com/bitcoin/bips/blob/master/bip-0324.mediawiki#user-content-v2_Bitcoin_P2P_message_structure) message type is `29`. |
There was a problem hiding this comment.
I think these values should be reserved in bip 324 before being added to other bips, to help avoid conflicts. (At worst, perhaps bip-324 could be updated in this PR)
There was a problem hiding this comment.
That's a good point! I was also told ages ago that we should make a PR to Core reserving the service bits we are using. @kcalvinalvin I think we should also do that?
| To save that bandwidth, we only send a Compact Leaf Data, that contains all missing information for the receiving peer to reconstruct the full leaf data. | ||
| A compact leaf data is defined as: | ||
|
|
||
| | Field | type | Description | |
There was a problem hiding this comment.
The leaf data is quite similar to CTxUndo in Bitcoin Core (de-serialized as a Coin) . I suggest this data be sent as a separate message, as non-Utreexo clients could also make use of this data, and verify its integrity to varying degrees:
- The SwiftSync protocol requires this data to perform full validation and enable parallel block downloads. The integrity of the data is verified at a pre-determined block height. A client cannot be mislead into accepting an invalid state.
BIP-157 clients may audit the construction of a compact block filter when two peers disagree on the hash of a filter for a particular block. This assumes the client is not eclipsed.I realized after the fact this is not accurate. One could change the block undo data for trivially spendable scripts and still produce a valid block and undo data combination.- BIP-157 clients do not have a way to reasonably estimate fee rates without a third party oracle. This data would allow for block-level fee rate analysis by light clients, given the client audits the undo data indeed corresponds to the block filter they have received.
- There is no protocol-native way for clients to scan for silent payments. This would allow a "light" client to download both the block undo data and compact block filter to check for potential payments by computing partial secrets locally.
If the current serialization format from Bitcoin Core is used, nodes serving this data can simply read the bytes from disk and send them directly over the wire without a de-serialization step. The trade-off here is of course the TTLs. To compensate, along with the length of the message, a header section may be included that specifies a height filter for coins that are assumed to already be in the client's cache. So a client requests "I would like all coins spent in this block, but I have the coins that were created in the last N blocks already," however this would required interpreting the undo-data from disk.
When I checked in October, undo data was approximately 90GB on disk. Is this even significant compared to the proofs?
|
Hey, I just wanted to point out that we started collecting the one-byte identifiers used for messages on p2pv2 in an auxiliary file for BIP 324 and the identifiers you propose to use for Utreexo were added to the tracking there: #2092. Should you should change your identifiers going forth, please remember to update them there as well. |
murchandamus
left a comment
There was a problem hiding this comment.
Hey, just thought I’d check on y’all. How is it coming along? When you make your updates, please also tweak the preamble to comply with BIP 3 which was deployed meanwhile.
| BIP: 183 | ||
| Layer: Peer Services | ||
| Title: Utreexo - Peer Services | ||
| Author: Tadge Dryja <rx@awsomnet.org> | ||
| Calvin Kim <bip@calvinkim.info> | ||
| Davidson Souza <bip@dlsouza.dev> | ||
| Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0183 | ||
| Status: Draft | ||
| Type: Standards Track | ||
| Created: 2024-08-08 | ||
| License: BSD-3-Clause | ||
| Requires: 181, 182 |
There was a problem hiding this comment.
| BIP: 183 | |
| Layer: Peer Services | |
| Title: Utreexo - Peer Services | |
| Author: Tadge Dryja <rx@awsomnet.org> | |
| Calvin Kim <bip@calvinkim.info> | |
| Davidson Souza <bip@dlsouza.dev> | |
| Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0183 | |
| Status: Draft | |
| Type: Standards Track | |
| Created: 2024-08-08 | |
| License: BSD-3-Clause | |
| Requires: 181, 182 | |
| BIP: 183 | |
| Layer: Peer Services | |
| Title: Utreexo - Peer Services | |
| Authors: Tadge Dryja <rx@awsomnet.org> | |
| Calvin Kim <bip@calvinkim.info> | |
| Davidson Souza <bip@dlsouza.dev> | |
| Status: Draft | |
| Type: Specification | |
| Assigned: 2025-08-29 | |
| License: BSD-3-Clause | |
| Requires: 181, 182 |
| | Peer Services | ||
| | Utreexo Accumulator Specification | ||
| | Tadge Dryja, Calvin Kim, Davidson Souza | ||
| | Standard |
There was a problem hiding this comment.
| | Standard | |
| | Specification |
| | Peer Services | ||
| | Utreexo - Transaction and block validation | ||
| | Tadge Dryja, Calvin Kim, Davidson Souza | ||
| | Standard |
There was a problem hiding this comment.
| | Standard | |
| | Specification |
| | Peer Services | ||
| | Utreexo - Peer Services | ||
| | Tadge Dryja, Calvin Kim, Davidson Souza | ||
| | Standard |
There was a problem hiding this comment.
| | Standard | |
| | Specification |
| BIP: 181 | ||
| Layer: Peer Services | ||
| Title: Utreexo Accumulator Specification | ||
| Author: Tadge Dryja <rx@awsomnet.org> | ||
| Calvin Kim <bip@calvinkim.info> | ||
| Davidson Souza <bip@dlsouza.dev> | ||
| Comments-Summary: No comments yet. | ||
| Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0181 | ||
| Status: Draft | ||
| Type: Standards Track | ||
| Created: 2025-06-18 | ||
| License: BSD-3-Clause |
There was a problem hiding this comment.
| BIP: 181 | |
| Layer: Peer Services | |
| Title: Utreexo Accumulator Specification | |
| Author: Tadge Dryja <rx@awsomnet.org> | |
| Calvin Kim <bip@calvinkim.info> | |
| Davidson Souza <bip@dlsouza.dev> | |
| Comments-Summary: No comments yet. | |
| Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0181 | |
| Status: Draft | |
| Type: Standards Track | |
| Created: 2025-06-18 | |
| License: BSD-3-Clause | |
| BIP: 181 | |
| Layer: Peer Services | |
| Title: Utreexo Accumulator Specification | |
| Authors: Tadge Dryja <rx@awsomnet.org> | |
| Calvin Kim <bip@calvinkim.info> | |
| Davidson Souza <bip@dlsouza.dev> | |
| Status: Draft | |
| Type: Specification | |
| Assigned: 2025-08-29 | |
| License: BSD-3-Clause |
| BIP: 182 | ||
| Layer: Peer Services | ||
| Title: Utreexo - Transaction and block validation | ||
| Author: Tadge Dryja <rx@awsomnet.org> | ||
| Calvin Kim <bip@calvinkim.info> | ||
| Davidson Souza <bip@dlsouza.dev> | ||
| Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0182 | ||
| Status: Draft | ||
| Type: Standards Track | ||
| Created: 2023-10-01 | ||
| License: BSD-3-Clause | ||
| Requires: 181 |
There was a problem hiding this comment.
| BIP: 182 | |
| Layer: Peer Services | |
| Title: Utreexo - Transaction and block validation | |
| Author: Tadge Dryja <rx@awsomnet.org> | |
| Calvin Kim <bip@calvinkim.info> | |
| Davidson Souza <bip@dlsouza.dev> | |
| Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0182 | |
| Status: Draft | |
| Type: Standards Track | |
| Created: 2023-10-01 | |
| License: BSD-3-Clause | |
| Requires: 181 | |
| BIP: 182 | |
| Layer: Peer Services | |
| Title: Utreexo - Transaction and block validation | |
| Authors: Tadge Dryja <rx@awsomnet.org> | |
| Calvin Kim <bip@calvinkim.info> | |
| Davidson Souza <bip@dlsouza.dev> | |
| Status: Draft | |
| Type: Specification | |
| Assigned: 2025-08-29 | |
| License: BSD-3-Clause | |
| Requires: 181 |
A few more months have gone by since #1923 (comment) -- are you still working on this? |
Mentio Utreexo BIPs from kcalvinalvin/bips since they are not yet merged into bitcoin/bips. Add a note marking them as work in progress and mention the PR on bitcoin/bips#1923.
Mentio Utreexo BIPs from kcalvinalvin/bips since they are not yet merged into bitcoin/bips. Add a note marking them as work in progress and mention the PR on bitcoin/bips#1923.
66f29cd docs: acknowledge utreexo BIP drafts and note WIP status (jaoleal) Pull request description: ### What is the purpose of this pull request? - [ ] Bug fix - [X] Documentation update - [ ] New feature - [ ] Test - [ ] Other: <!-- Please describe it --> ### Which crates are being modified? - [ ] floresta-chain - [ ] floresta-cli - [ ] floresta-common - [ ] floresta-compact-filters - [ ] floresta-electrum - [ ] floresta-watch-only - [ ] floresta-wire - [ ] floresta - [ ] florestad - [x] Other: Readme ### Description and Notes To avoid discussions and misunderstandings about utreexo itself the bips were written and are, at the time Im writing this PR, being merged. This mentions bitcoin/bips#1923 to be enough to close #296 draft until bips are merged. ### Contributor Checklist <!-- Feel free to remove this section once you've confirmed all items --> - [X] I've followed the [contribution guidelines](https://github.com/vinteumorg/Floresta/blob/master/CONTRIBUTING.md) - [X] I've verified one of the following: - Ran `just pcc` (recommended but slower) - Ran `just lint-features '-- -D warnings' && cargo test --release` - Confirmed CI passed on my fork - [X] I've linked any related issue(s) in the sections above Finally, you are encouraged to sign all your commits (it proves authorship and guards against tampering—see [How (and why) to sign Git commits](https://withblue.ink/2020/05/17/how-and-why-to-sign-git-commits.html) and [GitHub's guide to signing commits](https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits)). ACKs for top commit: moisesPompilio: ACK 66f29cd JoseSK999: ACK 66f29cd; links are working luisschwab: ACK 66f29cd Tree-SHA512: ba7d34eac1235bfbd8bfc7720b40dcb15838c9ad404bfd91c079f6e1b8d174c07bd3ef50812d32b41665c5de1fc4ee293e81c471ac3d07d7e102a0f0d6664ead
|
Update, just ran into Tadge and the spec is being reworked and updated for the validation and p2p spec, maybe less change for the accumulator spec. |
| ## References | ||
|
|
||
| [^1]: https://groups.google.com/g/bitcoindev/c/qyId8Yto45M | ||
| [^2]: https://delvingbitcoin.org/t/great-consensus-cleanup-revival/710 |
There was a problem hiding this comment.
nit: i think it's preferable to link to BIP 54 directly nowadays
| While a forest with 7 leaves will look like this: | ||
|
|
||
| ``` | ||
|
|
||
| |---------------\ | ||
| 12 | ||
| |-------\ |-------\ | ||
| 08 09 10 | ||
| |---\ |---\ |---\ |---\ | ||
| 00 01 02 03 04 05 06 | ||
| ``` |
There was a problem hiding this comment.
Nit: maybe worth it to briefly mention the roots in this example are 06, 10 and 12
|
|
||
| # Definitions | ||
|
|
||
| - `hash` refers to a vector of 32 byte arrays. |
There was a problem hiding this comment.
This seems wrong, it reads as a Vec<[u8; 32]> (a vector of hashes).
I think we should only say "32-byte array".
| - `hash` refers to a vector of 32 byte arrays. | |
| - `hash` refers to a 32-byte array. |
| - `[]hash` refers to a vector of `hash`. | ||
| - `acc` refers to the Utreexo accumulator state. An `acc` is comprised of: | ||
| - `roots` refers to the roots of the Merkle Trees. Represented as `[]hash`. | ||
| - `numleaves` refers to the number of total leaves added to the accumulator. Represented as uint64. |
There was a problem hiding this comment.
| - `numleaves` refers to the number of total leaves added to the accumulator. Represented as uint64. | |
| - `numleaves` refers to the total number of leaves ever added to the accumulator. Represented as uint64. |
| - `acc` refers to the Utreexo accumulator state. An `acc` is comprised of: | ||
| - `roots` refers to the roots of the Merkle Trees. Represented as `[]hash`. | ||
| - `numleaves` refers to the number of total leaves added to the accumulator. Represented as uint64. | ||
| - `root` refers to the top `hash` in a tree in the `acc`. |
There was a problem hiding this comment.
| - `root` refers to the top `hash` in a tree in the `acc`. | |
| - `root` refers to the top `hash` of one Merkle tree in the `acc`. |
| - `root` refers to the top `hash` in a tree in the `acc`. | ||
| - `proof` is an inclusion proof for elements in the accumulator. It's comprised of two fields: | ||
| - `targets` are the positions of the elements being proven. Represented as a vector of uint64. | ||
| - `proof` are the hashes needed to hash the roots. Represented as a `[]hash`. `proof` MUST be in ascending order by the node positions. |
There was a problem hiding this comment.
| - `proof` are the hashes needed to hash the roots. Represented as a `[]hash`. `proof` MUST be in ascending order by the node positions. | |
| - `proof` are the sibling hashes needed to compute the corresponding accumulator roots. Represented as a `[]hash`. The hashes in `proof` MUST be ordered by ascending node position. |
| stored locally for non-Utreexo nodes. Non-Utreexo nodes store this data (under "chainstate/" for Bitcoin Core) | ||
| but since utreexo nodes don't this data, it must be provided. | ||
|
|
||
| The accompanying accumulator proof proves that the given leaf data are committed |
There was a problem hiding this comment.
| The accompanying accumulator proof proves that the given leaf data are committed | |
| The accompanying accumulator proof proves that the given leaf data items are committed |
| | Field Name | Data Type | Byte Size | Description | | ||
| | ------------------- | ------------------- | --------- | ---------------------------------------- | | ||
| | Accumulator Proof | variable byte array | variable | The Utreexo proof as defined in BIP-0181 | | ||
| | UTXO hash preimages | variable byte array | variable | The UTXO data needed to validate all the transaction in the block | |
There was a problem hiding this comment.
| | UTXO hash preimages | variable byte array | variable | The UTXO data needed to validate all the transaction in the block | | |
| | UTXO hash preimages | variable byte array | variable | The UTXO data needed to validate all the transactions in the block | |
optout21
left a comment
There was a problem hiding this comment.
It's very positive to see work being done on the spec of this important topic! Though it's somewhat discerning that there's not much activity on this PR lately.
I have reviewed the documents, with focus on BIP-182.
I have a few smaller observations, placed in-line.
I do have a series of open questions, that are partly from my incomplete understanding of the topic. The spec doesn't need to answer these questions directly, but the answers should be derivable from the spec.
-
Basis for validation: When a light client validates a block+proofs it receives, does it validate the proofs against it's own copy of the accumulator state (roots), or just uses the accumulator state given by the prover?
-
If it accepts and uses the accumulator state given by the prover, how can the prover be sure that the accumulator state is correct for the whole UTXO set, and it is the correct one derived from all the blocks, and not just one created for the UTXOs being spent in the current block? Is it possible to derive the accumulator roots from the roots as of the previous block and the proofs of the current block (without the whole accumulator tree)?
-
If it uses its own copy of the accumulator state (roots), it has to match the one of the prover. Is it linked to the previous block hash, so it's guaranteed to be the same? How can a light client keep an up-to date accumulator root set, without having all the leaf data?
-
When creating a proof for a non-confirmed (mempool) transaction, what accumulator state is used? Is it the state as of the current block? What if the current block changes, is the proof re-generated? What happens if the relayed transaction+proof gets to another node, but in the meantime the current block changes?
Apologies if some of these questions are too broad or general here, the answers may be in the specs between the lines. I appreciate any explanations.
This spec is not a white paper, but I think at least some hints should be added. A reader should not have such doubts after reading the specs.
| We label positions starting at `0` on the bottom left, incrementing as we traverse the bottom row from left to right, | ||
| and then continue on to higher rows. There may be gaps in the label numbers when moving up a row; the label | ||
| numbers are "padded out" to the next perfect tree that could encompass the entire forest. |
There was a problem hiding this comment.
d89952d BIP181: Add the Utreexo accumulator BIP:
Two representations are used in this section, and I feel it confusing, it's easy to mix up. One is the forest -- several perfect disjoint binary trees, and the single incomplete tree. It's confusing that there can be gaps when previous paragraphs talked about perfect trees.
The labeling convention is more than labeling, it is a different representation of the forest, within a single tree. I think this needs more explanation, how the two representations are equivalent, how to construct the tree (e.g. that the larger tree is always put first, etc).
| 00 01 02 03 04 05 06 07 | ||
| ``` | ||
|
|
||
| While a forest with 7 leaves will look like this: |
There was a problem hiding this comment.
d89952d BIP181: Add the Utreexo accumulator BIP:
"a forest with 8 leaves will have a single tree"; this forest with 7 leaves also looks like a single tree in this representation (see my previous note on my confusion on the representations).
|
|
||
| ## Motivation | ||
|
|
||
| Although Utreexo in its current form is not proposed as a soft fork, it is essential that |
There was a problem hiding this comment.
4aa26f3 BIP182: Add the Utreexo validation BIP:
Mentioning a soft fork at the beginning of the document raises more questions than it answers, I think -- was it considered to be a soft fork? How? Why not? etc.
I propose moving the reference to soft fork possibility to the end of the document, hinting at how this could be the basis of a soft fork in the future.
|
|
||
| | Name | Type | Description | | ||
| | ------------------ | ------------------------ | ----------------------------------------- | | ||
| | Utreexo_Tag_V1 | 64-byte array | The version tag to be prepended to the leafhash. | |
There was a problem hiding this comment.
4aa26f3 BIP182: Add the Utreexo validation BIP:
The "to the leafhash" part here is confusing, it is prepended to the hash preimage. Suggestion: simply "Version tag prefix".
| | BlockHash | 32-byte array | The hash of the block in which this tx was confirmed. | | ||
| | TXID | 32-byte array | The transaction's TXID | | ||
| | Vout | 4-byte unsigned integer | The output index of this UTXO | | ||
| | Header code | 4-byte unsigned integer | The block height and iscoinbase. This is a value obtained by left shifting the block height that confirmed this transaction by one bit, and then OR-ing it with 1, only if this transaction is a coinbase. | |
There was a problem hiding this comment.
4aa26f3 BIP182: Add the Utreexo validation BIP:
What is the point of including the iscoinbase flag into the block height, when prehash data size is not an issue here? (2x64 bytes are used only for the version prefix) Would it be cleaner to include in two independent fields?
| | Output script size | varint | The output script length in bytes | | ||
| | Output script | variable byte array | The output script of the UTXO | |
There was a problem hiding this comment.
4aa26f3 BIP182: Add the Utreexo validation BIP:
Output script: It is mentioned that this is included to commit to the output script. If this is the case, I suggest using a hash of the output script, because this would get rid of the complications introduced by a variable-size field (this is the only non-fixed length field).
On the other hand, from other parts of the spec, I understand that the output script is needed by light clients validating the transaction. If this is the case, I suggest changing the "is added to commit to the output script" statement, as it is misleading.
| The UTXO proof has 2 elements: the accumulator proof and the leaf data. The | ||
| leaf data provides the necessary UTXO data for block validation that would be | ||
| stored locally for non-Utreexo nodes. Non-Utreexo nodes store this data (under "chainstate/" for Bitcoin Core) |
There was a problem hiding this comment.
4aa26f3 BIP182: Add the Utreexo validation BIP:
I'm confused by this sentence. "stored locally BY non-Utreexo nodes"?
| in the accumulator. Without this accumulator proof, the Utreexo nodes would not have | ||
| a way to ensure that the given UTXO data exists in the UTXO set. The accumulator proof |
There was a problem hiding this comment.
4aa26f3 BIP182: Add the Utreexo validation BIP:
Extra clarification "the UTXOs referenced by the given data" or "the given referenced UTXO data"
| As the amount of Bitcoin users grow, the UTXO set grows with it. | ||
|
|
||
| The UTXO set is currently around 10GB in 2025 and with pruning that's all it takes to maintain a full node. | ||
| However, as the UTXO set grows, the disk storage requirement will grow along with it and increase the barrier to running a full node. |
There was a problem hiding this comment.
4aa26f3 BIP182: Add the Utreexo validation BIP:
"a full node." --> "a fully validating node" (it may be pruned)
|
After chatting with Davidson, my understanding is that there will be drastic changes to the document, so I suggest that further review is postponed until the authors update their submission to the swiftsync-based approach. |
…2590) `btclib.utreexo` holds `Stump` (the roots and the leaf count), `Proof` and `UpdateData`: add leaves, delete proven leaves, verify a proof, update a cached proof across a block, and read and write rustreexo's bytes with `parse` and `serialize`. It follows the draft BIP 181 and BIP 182's order of a block's changes (bitcoin/bips#1923), and mit-dci/rustreexo at tag v0.7.0, commit b73cbac1f4, with utreexo/utreexo v0.18.0 as the second reference. rustreexo's `test_values/` files are vendored under `tests/_data/rustreexo/` and pinned in `tests/_data/README.md`, and every case of them runs. The crate is licensed `MIT OR Apache-2.0`. The hypothesis properties check `Stump` against a model that builds the trees by recursion. The issue's property "adding then deleting leaves gives the roots of a forest built without them" does not hold: deleting leaf `a` of `[a, b, c, d]` leaves `b` beside the node of `c` and `d`, where a forest of `[b, c, d]` pairs `b` with `c`. The test states the property that holds, with that counterexample. Left for the next slice of [ISS 2590](#2590): BIP 182's leaf hashes, Floresta's `roots.json` reproduced from `blocks.json`, the informational job that compares btclib's roots with `florestad` and `utreexod`, and the utreexod formats: the untagged stump of the Go library and the wire encoding of a proof. Not implemented: `Proof` subsetting (`get_proof_subset`). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Ferdinando Ametrano <ferdinando@ametrano.net>
…2590) `btclib.utreexo` holds `Stump` (the roots and the leaf count), `Proof` and `UpdateData`: add leaves, delete proven leaves, verify a proof, update a cached proof across a block, and read and write rustreexo's bytes with `parse` and `serialize`. It follows the draft BIP 181 and BIP 182's order of a block's changes (bitcoin/bips#1923), and mit-dci/rustreexo at tag v0.7.0, commit b73cbac1f4, with utreexo/utreexo v0.18.0 as the second reference. rustreexo's `test_values/` files are vendored under `tests/_data/rustreexo/` and pinned in `tests/_data/README.md`, and every case of them runs. The crate is licensed `MIT OR Apache-2.0`. The hypothesis properties check `Stump` against a model that builds the trees by recursion. The issue's property "adding then deleting leaves gives the roots of a forest built without them" does not hold: deleting leaf `a` of `[a, b, c, d]` leaves `b` beside the node of `c` and `d`, where a forest of `[b, c, d]` pairs `b` with `c`. The test states the property that holds, with that counterexample. Left for the next slice of [ISS 2590](#2590): BIP 182's leaf hashes, Floresta's `roots.json` reproduced from `blocks.json`, the informational job that compares btclib's roots with `florestad` and `utreexod`, and the utreexod formats: the untagged stump of the Go library and the wire encoding of a proof. Not implemented: `Proof` subsetting (`get_proof_subset`). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Ferdinando Ametrano <ferdinando@ametrano.net>
…2590) `btclib.utreexo` holds `Stump` (the roots and the leaf count), `Proof` and `UpdateData`: add leaves, delete proven leaves, verify a proof, update a cached proof across a block, and read and write rustreexo's bytes with `parse` and `serialize`. It follows the draft BIP 181 and BIP 182's order of a block's changes (bitcoin/bips#1923), and mit-dci/rustreexo at tag v0.7.0, commit b73cbac1f4, with utreexo/utreexo v0.18.0 as the second reference. rustreexo's `test_values/` files are vendored under `tests/_data/rustreexo/` and pinned in `tests/_data/README.md`, and every case of them runs. The crate is licensed `MIT OR Apache-2.0`. The hypothesis properties check `Stump` against a model that builds the trees by recursion. The issue's property "adding then deleting leaves gives the roots of a forest built without them" does not hold: deleting leaf `a` of `[a, b, c, d]` leaves `b` beside the node of `c` and `d`, where a forest of `[b, c, d]` pairs `b` with `c`. The test states the property that holds, with that counterexample. Left for the next slice of [ISS 2590](#2590): BIP 182's leaf hashes, Floresta's `roots.json` reproduced from `blocks.json`, the informational job that compares btclib's roots with `florestad` and `utreexod`, and the utreexod formats: the untagged stump of the Go library and the wire encoding of a proof. Not implemented: `Proof` subsetting (`get_proof_subset`). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Ferdinando Ametrano <ferdinando@ametrano.net>
…2590) `btclib.utreexo` holds `Stump` (the roots and the leaf count), `Proof` and `UpdateData`: add leaves, delete proven leaves, verify a proof, update a cached proof across a block, and read and write rustreexo's bytes with `parse` and `serialize`. It follows the draft BIP 181 and BIP 182's order of a block's changes (bitcoin/bips#1923), and mit-dci/rustreexo at tag v0.7.0, commit b73cbac1f4, with utreexo/utreexo v0.18.0 as the second reference. rustreexo's `test_values/` files are vendored under `tests/_data/rustreexo/` and pinned in `tests/_data/README.md`, and every case of them runs. The crate is licensed `MIT OR Apache-2.0`. The hypothesis properties check `Stump` against a model that builds the trees by recursion. The issue's property "adding then deleting leaves gives the roots of a forest built without them" does not hold: deleting leaf `a` of `[a, b, c, d]` leaves `b` beside the node of `c` and `d`, where a forest of `[b, c, d]` pairs `b` with `c`. The test states the property that holds, with that counterexample. Left for the next slice of [ISS 2590](#2590): BIP 182's leaf hashes, Floresta's `roots.json` reproduced from `blocks.json`, the informational job that compares btclib's roots with `florestad` and `utreexod`, and the utreexod formats: the untagged stump of the Go library and the wire encoding of a proof. Not implemented: `Proof` subsetting (`get_proof_subset`). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Ferdinando Ametrano <ferdinando@ametrano.net>
These are the 3 BIPs that describe Utreexo, a consensus-compatible (non-soft fork) way to send and verify transactions without storing the full UTXO set.
The 3 BIPs are for:
Mailing list post: https://groups.google.com/g/bitcoindev/c/W1lxBraKG_E