Skip to content

BIP 181, 182, 183: BIPs for Utreexo - #1923

Draft
kcalvinalvin wants to merge 4 commits into
bitcoin:masterfrom
kcalvinalvin:2025-08-10-utreexo-bips
Draft

kcalvinalvin wants to merge 4 commits into
bitcoin:masterfrom
kcalvinalvin:2025-08-10-utreexo-bips

Conversation

@kcalvinalvin

Copy link
Copy Markdown
Contributor

These are the 3 BIPs that describe Utreexo, a consensus-compatible (non-soft fork) way to send and verify transactions without storing the full UTXO set.

The 3 BIPs are for:

  1. The specification of the Utreexo accumulator.
  2. The specification of Bitcoin block and tx validation using the Utreexo accumulator.
  3. The peer to peer networking changes required to enable Utreexo nodes.

Mailing list post: https://groups.google.com/g/bitcoindev/c/W1lxBraKG_E

@kcalvinalvin
kcalvinalvin force-pushed the 2025-08-10-utreexo-bips branch 2 times, most recently from 9b3eafb to a94f643 Compare August 10, 2025 07:09

@jmoik jmoik left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some typos

Comment thread utreexo-p2p-bip.md Resolved
Comment thread utreexo-p2p-bip.md Resolved
Comment thread utreexo-p2p-bip.md Resolved
Comment thread utreexo-p2p-bip.md Resolved
Comment thread utreexo-p2p-bip.md Resolved
Comment thread utreexo-p2p-bip.md Resolved
Comment thread utreexo-p2p-bip.md Resolved
Comment thread utreexo-p2p-bip.md Resolved

@jonatack jonatack left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for proposing these drafts. They already look quite complete with respect to the editorial requirements (BIPs 2 and 3). I've done a cursory first pass. No immediate conceptual feedback. A few editorial comments follow; feel free to ignore them during conceptual review until they are applicable.

Comment thread utreexo-p2p-bip.md Resolved
Comment thread utreexo-validation-bip.md Resolved
Comment thread utreexo-accumulator-bip.md Resolved
Comment thread utreexo-accumulator-bip.md Resolved
Comment thread utreexo-validation-bip.md Resolved
Comment thread utreexo-accumulator-bip.md Resolved
Comment thread utreexo-accumulator-bip.md Resolved
Comment thread utreexo-accumulator-bip.md Resolved
@kcalvinalvin
kcalvinalvin force-pushed the 2025-08-10-utreexo-bips branch 2 times, most recently from cb2993c to d1d0342 Compare August 12, 2025 06:23
@petertodd

Copy link
Copy Markdown
Contributor

You need to justify why you're using SHA-512/256 rather than SHA-256, like the rest of the Bitcoin protocol. Right now you just link to a paper from 2011. But that paper is out of date now that hardware support for SHA-256 has become common.

@ghost

ghost commented Aug 12, 2025 •

Copy link
Copy Markdown

I strongly recommend replacing SHA-256 with SHAKE256 (from the SHA-3 standard) for the following reasons:

1. Security Advantages

  • 🔒 Provides built-in protection against length-extension attacks
  • 📏 Offers flexible output lengths (supports 128-bit and 256-bit security levels)
  • ⚙️ Based on Keccak sponge construction (NIST FIPS 202 standard)
  • 🌐 Aligns with post-quantum cryptography standards

2. Comparative Analysis: SHA-256 vs SHAKE256

Characteristic SHA-256 SHAKE256
Algorithm Family SHA-2 SHA-3 (Keccak)
Output Flexibility Fixed 256-bit Arbitrary length
Security Properties Vulnerable to length-extension Resistant to length-extension
Internal Structure Merkle-Damgård Sponge function
Standardization NIST FIPS 180-4 NIST FIPS 202

3. Functional Example

Input: Bitcoin

SHAKE256 (512-bit output):
6beb0661ba1fa7289bf359fbb81550bd9641cf5abc62a14d466c421c8a86e528e027632ec0e7ceb994650566f3c8258af2240333b6d0e9186766fd2c1ebb763a

SHAKE256 (256-bit output):
6beb0661ba1fa7289bf359fbb81550bd9641cf5abc62a14d466c421c8a86e528

4. Implementation Benefits

  • ✅ Maintains 256-bit output compatibility where needed
  • ✅ Future-proofs against emerging cryptographic vulnerabilities
  • ✅ Reduces potential attack vectors through improved design
  • ✅ Supports Bitcoin's security evolution while maintaining performance

5. Technical Reference

For detailed cryptographic differences:
Cryptographic Comparison: SHA-2 vs SHA-3

@kcalvinalvin

Copy link
Copy Markdown
Contributor Author

You need to justify why you're using SHA-512/256 rather than SHA-256, like the rest of the Bitcoin protocol. Right now you just link to a paper from 2011. But that paper is out of date now that hardware support for SHA-256 has become common.

Sure we can update the accumulator BIP with benchmarks for SHA512/256 vs SHA256.

But could you link to the aforementioned justifications for the other parts of the Bitcoin protocol that use SHA512?

@kcalvinalvin

kcalvinalvin commented Aug 18, 2025 •

Copy link
Copy Markdown
Contributor Author

I strongly recommend replacing SHA-256 with SHAKE256 (from the SHA-3 standard) for the following reasons:

SHAKE256 is not used in Bitcoin and introduces a new hash which increases the trust-assumption. We do not want to do this.

@bitcoin bitcoin deleted a comment Aug 18, 2025
@bitcoin bitcoin deleted a comment from kcalvinalvin Aug 18, 2025
@ghost

This comment was marked as off-topic.

@bitcoin bitcoin deleted a comment from kcalvinalvin Aug 18, 2025
@bitcoin bitcoin deleted a comment Aug 18, 2025
@jonatack

Copy link
Copy Markdown
Member

Some friendly moderation to keep the discussion focused on technical review -- thanks.

@kcalvinalvin

kcalvinalvin commented Aug 18, 2025 •

Copy link
Copy Markdown
Contributor Author

The reliance of Bitcoin on SHA-2—a legacy hash function designed by the National Security Agency (NSA)—introduces non-trivial security risks, particularly when considering the often-dismissed threat posed by quantum adversaries.

SHA256 and SHA512 are quantum resistent.

Migrating to SHAKE256 (a variant of SHA-3) would represent a meaningful improvement, though such a change merely delays the inevitable: Bitcoin must eventually transition to a quantum-resistant cryptographic framework. When this occurs—and it will, regardless of opposition—SHA-2, along with ECDSA private keys, public keys, and signatures, will become obsolete.
See: Lenght extension attack (Bitcoin is vulnerable because it's using SHA-256)

Ok but this has nothing to do with this BIP.

@murchandamus

Copy link
Copy Markdown
Member

@1BitcoinBoWP1FZ4xwTNkq6XksKidmgYYw, please cut out the LLM generated comments. If any of us were interested in seeing an LLM’s prediction of what might be said about a topic, we could prompt one ourselves.

@petertodd

petertodd commented Aug 18, 2025 via email

Copy link
Copy Markdown
Contributor

@kcalvinalvin

Copy link
Copy Markdown
Contributor Author

On Mon, Aug 18, 2025 at 04:06:51AM -0700, Calvin Kim wrote: kcalvinalvin left a comment (bitcoin/bips#1923) > You need to justify why you're using SHA-512/256 rather than SHA-256, like the rest of the Bitcoin protocol. Right now you just link to a paper from 2011. But that paper is out of date now that hardware support for SHA-256 has become common. Sure we can update the accumulator BIP with benchmarks for SHA512/256 vs SHA256. But could you link to the aforementioned justifications for the other parts of the Bitcoin protocol that use SHA512?
No part of the Bitcoin consensus protocol uses SHA512.

Ok but you've stated in your previous comment "You need to justify why you're using SHA-512/256 rather than SHA-256, like the rest of the Bitcoin protocol". Would be very helpful to see what type of justifications the other protocols have made.

Second, I don't think it matters if SHA512 wasn't used in the Bitcoin consensus protocol. SHA512 is used in BIP32 and the argument that SHA512 is safe for generating private keys but not safe for Bitcoin consensus isn't sound.

I think our original justification (better performance with SHA512/256) mentioned in the BIP is sound. Happy to provide the benchmarks, they're being worked on at the moment.

@ghost

This comment was marked as abuse.

@kcalvinalvin

This comment was marked as off-topic.

@kcalvinalvin

This comment was marked as off-topic.

Comment thread utreexo-validation-bip.md Outdated
Comment on lines +63 to +66
| Name | Type | Description |
| ----------------- | ------------------------ | ----------------------------------------- |
| Utreexo_Tag_V1 | 64 byte array | The version tag to be prepended to the leafhash. |
| Utreexo_Tag_V1 | 64 byte array | The version tag to be prepended to the leafhash. |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For clarification, is the Utreexo_Tag_V1 really used twice in preimage to the hash?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My guess would be that this duplication is unintended.

Suggested change
| Name | Type | Description |
| ----------------- | ------------------------ | ----------------------------------------- |
| Utreexo_Tag_V1 | 64 byte array | The version tag to be prepended to the leafhash. |
| Utreexo_Tag_V1 | 64 byte array | The version tag to be prepended to the leafhash. |
| Name | Type | Description |
| ----------------- | ------------------------ | ----------------------------------------- |
| Utreexo_Tag_V1 | 64 byte array | The version tag to be prepended to the leafhash. |

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh no the duplication is intended.

Since we use SHA512/256 as the hash function, each chunk is 128 bytes. Since the version tag is only 64 bytes, we need two of them.

@petertodd

Copy link
Copy Markdown
Contributor

On Mon, Aug 18, 2025 at 04:06:51AM -0700, Calvin Kim wrote: kcalvinalvin left a comment (bitcoin/bips#1923) > You need to justify why you're using SHA-512/256 rather than SHA-256, like the rest of the Bitcoin protocol. Right now you just link to a paper from 2011. But that paper is out of date now that hardware support for SHA-256 has become common. Sure we can update the accumulator BIP with benchmarks for SHA512/256 vs SHA256. But could you link to the aforementioned justifications for the other parts of the Bitcoin protocol that use SHA512?
No part of the Bitcoin consensus protocol uses SHA512.

Ok but you've stated in your previous comment "You need to justify why you're using SHA-512/256 rather than SHA-256, like the rest of the Bitcoin protocol". Would be very helpful to see what type of justifications the other protocols have made.

Second, I don't think it matters if SHA512 wasn't used in the Bitcoin consensus protocol. SHA512 is used in BIP32 and the argument that SHA512 is safe for generating private keys but not safe for Bitcoin consensus isn't sound.

I think our original justification (better performance with SHA512/256) mentioned in the BIP is sound. Happy to provide the benchmarks, they're being worked on at the moment.

The question is 1) why are we added one new dependency to consensus implementations, and 2) is this actually a performance increase, given that dedicated SHA256 hardware is becoming common?

Length-extension attacks are not relevant for this use-case as we are only committing to public data.

@kcalvinalvin

Copy link
Copy Markdown
Contributor Author

@kcalvinalvin there doesn't seem to be any response since late August -- do you plan to address the review and update here?

Yes. There have been multiple protocol changes to BIP183 since then and I've been busy with the implementations for those changes. I'll get to the reviews and update the BIP.

@kcalvinalvin
kcalvinalvin marked this pull request as draft November 30, 2025 17:34
Comment thread bip-0183.md
`MSG_UTREEXO_PROOF` is all the data required for a CSN or archive node using the Utreexo accumulators to validate a Bitcoin block.

Its `cmdString` for P2PV1 is `uproof`.
Its [BIP324 P2PV2](https://github.com/bitcoin/bips/blob/master/bip-0324.mediawiki#user-content-v2_Bitcoin_P2P_message_structure) message type is `29`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these values should be reserved in bip 324 before being added to other bips, to help avoid conflicts. (At worst, perhaps bip-324 could be updated in this PR)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point! I was also told ages ago that we should make a PR to Core reserving the service bits we are using. @kcalvinalvin I think we should also do that?

Comment thread bip-0183.md
To save that bandwidth, we only send a Compact Leaf Data, that contains all missing information for the receiving peer to reconstruct the full leaf data.
A compact leaf data is defined as:

| Field | type | Description |

@rustaceanrob rustaceanrob Dec 29, 2025 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The leaf data is quite similar to CTxUndo in Bitcoin Core (de-serialized as a Coin) . I suggest this data be sent as a separate message, as non-Utreexo clients could also make use of this data, and verify its integrity to varying degrees:

  • The SwiftSync protocol requires this data to perform full validation and enable parallel block downloads. The integrity of the data is verified at a pre-determined block height. A client cannot be mislead into accepting an invalid state.
  • BIP-157 clients may audit the construction of a compact block filter when two peers disagree on the hash of a filter for a particular block. This assumes the client is not eclipsed. I realized after the fact this is not accurate. One could change the block undo data for trivially spendable scripts and still produce a valid block and undo data combination.
  • BIP-157 clients do not have a way to reasonably estimate fee rates without a third party oracle. This data would allow for block-level fee rate analysis by light clients, given the client audits the undo data indeed corresponds to the block filter they have received.
  • There is no protocol-native way for clients to scan for silent payments. This would allow a "light" client to download both the block undo data and compact block filter to check for potential payments by computing partial secrets locally.

If the current serialization format from Bitcoin Core is used, nodes serving this data can simply read the bytes from disk and send them directly over the wire without a de-serialization step. The trade-off here is of course the TTLs. To compensate, along with the length of the message, a header section may be included that specifies a height filter for coins that are assumed to already be in the client's cache. So a client requests "I would like all coins spent in this block, but I have the coins that were created in the last N blocks already," however this would required interpreting the undo-data from disk.

When I checked in October, undo data was approximately 90GB on disk. Is this even significant compared to the proofs?

@murchandamus

Copy link
Copy Markdown
Member

Hey, I just wanted to point out that we started collecting the one-byte identifiers used for messages on p2pv2 in an auxiliary file for BIP 324 and the identifiers you propose to use for Utreexo were added to the tracking there: #2092.

Should you should change your identifiers going forth, please remember to update them there as well.

@murchandamus murchandamus left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, just thought I’d check on y’all. How is it coming along? When you make your updates, please also tweak the preamble to comply with BIP 3 which was deployed meanwhile.

Comment thread bip-0183.md
Comment on lines +2 to +13
BIP: 183
Layer: Peer Services
Title: Utreexo - Peer Services
Author: Tadge Dryja <rx@awsomnet.org>
Calvin Kim <bip@calvinkim.info>
Davidson Souza <bip@dlsouza.dev>
Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0183
Status: Draft
Type: Standards Track
Created: 2024-08-08
License: BSD-3-Clause
Requires: 181, 182

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
BIP: 183
Layer: Peer Services
Title: Utreexo - Peer Services
Author: Tadge Dryja <rx@awsomnet.org>
Calvin Kim <bip@calvinkim.info>
Davidson Souza <bip@dlsouza.dev>
Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0183
Status: Draft
Type: Standards Track
Created: 2024-08-08
License: BSD-3-Clause
Requires: 181, 182
BIP: 183
Layer: Peer Services
Title: Utreexo - Peer Services
Authors: Tadge Dryja <rx@awsomnet.org>
Calvin Kim <bip@calvinkim.info>
Davidson Souza <bip@dlsouza.dev>
Status: Draft
Type: Specification
Assigned: 2025-08-29
License: BSD-3-Clause
Requires: 181, 182

Comment thread README.mediawiki
| Peer Services
| Utreexo Accumulator Specification
| Tadge Dryja, Calvin Kim, Davidson Souza
| Standard

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| Standard
| Specification

Comment thread README.mediawiki
| Peer Services
| Utreexo - Transaction and block validation
| Tadge Dryja, Calvin Kim, Davidson Souza
| Standard

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| Standard
| Specification

Comment thread README.mediawiki
| Peer Services
| Utreexo - Peer Services
| Tadge Dryja, Calvin Kim, Davidson Souza
| Standard

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| Standard
| Specification

Comment thread bip-0181.md
Comment on lines +2 to +13
BIP: 181
Layer: Peer Services
Title: Utreexo Accumulator Specification
Author: Tadge Dryja <rx@awsomnet.org>
Calvin Kim <bip@calvinkim.info>
Davidson Souza <bip@dlsouza.dev>
Comments-Summary: No comments yet.
Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0181
Status: Draft
Type: Standards Track
Created: 2025-06-18
License: BSD-3-Clause

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
BIP: 181
Layer: Peer Services
Title: Utreexo Accumulator Specification
Author: Tadge Dryja <rx@awsomnet.org>
Calvin Kim <bip@calvinkim.info>
Davidson Souza <bip@dlsouza.dev>
Comments-Summary: No comments yet.
Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0181
Status: Draft
Type: Standards Track
Created: 2025-06-18
License: BSD-3-Clause
BIP: 181
Layer: Peer Services
Title: Utreexo Accumulator Specification
Authors: Tadge Dryja <rx@awsomnet.org>
Calvin Kim <bip@calvinkim.info>
Davidson Souza <bip@dlsouza.dev>
Status: Draft
Type: Specification
Assigned: 2025-08-29
License: BSD-3-Clause

Comment thread bip-0182.md
Comment on lines +2 to +13
BIP: 182
Layer: Peer Services
Title: Utreexo - Transaction and block validation
Author: Tadge Dryja <rx@awsomnet.org>
Calvin Kim <bip@calvinkim.info>
Davidson Souza <bip@dlsouza.dev>
Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0182
Status: Draft
Type: Standards Track
Created: 2023-10-01
License: BSD-3-Clause
Requires: 181

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
BIP: 182
Layer: Peer Services
Title: Utreexo - Transaction and block validation
Author: Tadge Dryja <rx@awsomnet.org>
Calvin Kim <bip@calvinkim.info>
Davidson Souza <bip@dlsouza.dev>
Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0182
Status: Draft
Type: Standards Track
Created: 2023-10-01
License: BSD-3-Clause
Requires: 181
BIP: 182
Layer: Peer Services
Title: Utreexo - Transaction and block validation
Authors: Tadge Dryja <rx@awsomnet.org>
Calvin Kim <bip@calvinkim.info>
Davidson Souza <bip@dlsouza.dev>
Status: Draft
Type: Specification
Assigned: 2025-08-29
License: BSD-3-Clause
Requires: 181

@jonatack

Copy link
Copy Markdown
Member

@kcalvinalvin there doesn't seem to be any response since late August -- do you plan to address the review and update here?

Yes. There have been multiple protocol changes to BIP183 since then and I've been busy with the implementations for those changes. I'll get to the reviews and update the BIP.

A few more months have gone by since #1923 (comment) -- are you still working on this?

jaoleal added a commit to jaoleal/FlorestaBA that referenced this pull request Apr 16, 2026
Mentio Utreexo BIPs from kcalvinalvin/bips since they are not yet merged into bitcoin/bips. Add a note marking them as work in progress and mention the PR  on bitcoin/bips#1923.
jaoleal added a commit to jaoleal/FlorestaBA that referenced this pull request Apr 16, 2026
Mentio Utreexo BIPs from kcalvinalvin/bips since they are not yet merged into bitcoin/bips. Add a note marking them as work in progress and mention the PR  on bitcoin/bips#1923.
Davidson-Souza added a commit to getfloresta/Floresta that referenced this pull request Apr 20, 2026
66f29cd docs: acknowledge utreexo BIP drafts and note WIP status (jaoleal)

Pull request description:

  ### What is the purpose of this pull request?

  - [ ] Bug fix
  - [X] Documentation update
  - [ ] New feature
  - [ ] Test
  - [ ] Other: <!-- Please describe it -->

  ### Which crates are being modified?

  - [ ] floresta-chain
  - [ ] floresta-cli
  - [ ] floresta-common
  - [ ] floresta-compact-filters
  - [ ] floresta-electrum
  - [ ] floresta-watch-only
  - [ ] floresta-wire
  - [ ] floresta
  - [ ] florestad
  - [x] Other: Readme
  ### Description and Notes

  To avoid discussions and misunderstandings about utreexo itself the bips were written and are, at the time Im writing this PR, being merged.

  This mentions bitcoin/bips#1923 to be enough to close #296

  draft until bips are merged.

  ### Contributor Checklist

  <!-- Feel free to remove this section once you've confirmed all items -->

  - [X] I've followed the [contribution guidelines](https://github.com/vinteumorg/Floresta/blob/master/CONTRIBUTING.md)
  - [X] I've verified one of the following:
    - Ran `just pcc` (recommended but slower)
    - Ran `just lint-features '-- -D warnings' && cargo test --release`
    - Confirmed CI passed on my fork
  - [X] I've linked any related issue(s) in the sections above

  Finally, you are encouraged to sign all your commits (it proves authorship and guards against tampering—see [How (and why) to sign Git commits](https://withblue.ink/2020/05/17/how-and-why-to-sign-git-commits.html) and [GitHub's guide to signing commits](https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits)).

ACKs for top commit:
  moisesPompilio:
    ACK 66f29cd
  JoseSK999:
    ACK 66f29cd; links are working
  luisschwab:
    ACK 66f29cd

Tree-SHA512: ba7d34eac1235bfbd8bfc7720b40dcb15838c9ad404bfd91c079f6e1b8d174c07bd3ef50812d32b41665c5de1fc4ee293e81c471ac3d07d7e102a0f0d6664ead
@jonatack

Copy link
Copy Markdown
Member

Update, just ran into Tadge and the spec is being reworked and updated for the validation and p2p spec, maybe less change for the accumulator spec.

Comment thread bip-0182.md
## References

[^1]: https://groups.google.com/g/bitcoindev/c/qyId8Yto45M
[^2]: https://delvingbitcoin.org/t/great-consensus-cleanup-revival/710

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: i think it's preferable to link to BIP 54 directly nowadays

Comment thread bip-0181.md
Comment on lines +99 to +109
While a forest with 7 leaves will look like this:

```

|---------------\
12
|-------\ |-------\
08 09 10
|---\ |---\ |---\ |---\
00 01 02 03 04 05 06
```

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: maybe worth it to briefly mention the roots in this example are 06, 10 and 12

Comment thread bip-0181.md

# Definitions

- `hash` refers to a vector of 32 byte arrays.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems wrong, it reads as a Vec<[u8; 32]> (a vector of hashes).

I think we should only say "32-byte array".

Suggested change
- `hash` refers to a vector of 32 byte arrays.
- `hash` refers to a 32-byte array.

Comment thread bip-0181.md
- `[]hash` refers to a vector of `hash`.
- `acc` refers to the Utreexo accumulator state. An `acc` is comprised of:
- `roots` refers to the roots of the Merkle Trees. Represented as `[]hash`.
- `numleaves` refers to the number of total leaves added to the accumulator. Represented as uint64.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- `numleaves` refers to the number of total leaves added to the accumulator. Represented as uint64.
- `numleaves` refers to the total number of leaves ever added to the accumulator. Represented as uint64.

Comment thread bip-0181.md
- `acc` refers to the Utreexo accumulator state. An `acc` is comprised of:
- `roots` refers to the roots of the Merkle Trees. Represented as `[]hash`.
- `numleaves` refers to the number of total leaves added to the accumulator. Represented as uint64.
- `root` refers to the top `hash` in a tree in the `acc`.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- `root` refers to the top `hash` in a tree in the `acc`.
- `root` refers to the top `hash` of one Merkle tree in the `acc`.

Comment thread bip-0181.md
- `root` refers to the top `hash` in a tree in the `acc`.
- `proof` is an inclusion proof for elements in the accumulator. It's comprised of two fields:
- `targets` are the positions of the elements being proven. Represented as a vector of uint64.
- `proof` are the hashes needed to hash the roots. Represented as a `[]hash`. `proof` MUST be in ascending order by the node positions.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- `proof` are the hashes needed to hash the roots. Represented as a `[]hash`. `proof` MUST be in ascending order by the node positions.
- `proof` are the sibling hashes needed to compute the corresponding accumulator roots. Represented as a `[]hash`. The hashes in `proof` MUST be ordered by ascending node position.

Comment thread bip-0182.md
stored locally for non-Utreexo nodes. Non-Utreexo nodes store this data (under "chainstate/" for Bitcoin Core)
but since utreexo nodes don't this data, it must be provided.

The accompanying accumulator proof proves that the given leaf data are committed

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The accompanying accumulator proof proves that the given leaf data are committed
The accompanying accumulator proof proves that the given leaf data items are committed

Comment thread bip-0182.md
| Field Name | Data Type | Byte Size | Description |
| ------------------- | ------------------- | --------- | ---------------------------------------- |
| Accumulator Proof | variable byte array | variable | The Utreexo proof as defined in BIP-0181 |
| UTXO hash preimages | variable byte array | variable | The UTXO data needed to validate all the transaction in the block |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| UTXO hash preimages | variable byte array | variable | The UTXO data needed to validate all the transaction in the block |
| UTXO hash preimages | variable byte array | variable | The UTXO data needed to validate all the transactions in the block |

@optout21 optout21 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's very positive to see work being done on the spec of this important topic! Though it's somewhat discerning that there's not much activity on this PR lately.

I have reviewed the documents, with focus on BIP-182.

I have a few smaller observations, placed in-line.

I do have a series of open questions, that are partly from my incomplete understanding of the topic. The spec doesn't need to answer these questions directly, but the answers should be derivable from the spec.

  • Basis for validation: When a light client validates a block+proofs it receives, does it validate the proofs against it's own copy of the accumulator state (roots), or just uses the accumulator state given by the prover?

  • If it accepts and uses the accumulator state given by the prover, how can the prover be sure that the accumulator state is correct for the whole UTXO set, and it is the correct one derived from all the blocks, and not just one created for the UTXOs being spent in the current block? Is it possible to derive the accumulator roots from the roots as of the previous block and the proofs of the current block (without the whole accumulator tree)?

  • If it uses its own copy of the accumulator state (roots), it has to match the one of the prover. Is it linked to the previous block hash, so it's guaranteed to be the same? How can a light client keep an up-to date accumulator root set, without having all the leaf data?

  • When creating a proof for a non-confirmed (mempool) transaction, what accumulator state is used? Is it the state as of the current block? What if the current block changes, is the proof re-generated? What happens if the relayed transaction+proof gets to another node, but in the meantime the current block changes?

Apologies if some of these questions are too broad or general here, the answers may be in the specs between the lines. I appreciate any explanations.
This spec is not a white paper, but I think at least some hints should be added. A reader should not have such doubts after reading the specs.

Comment thread bip-0181.md
Comment on lines +83 to +85
We label positions starting at `0` on the bottom left, incrementing as we traverse the bottom row from left to right,
and then continue on to higher rows. There may be gaps in the label numbers when moving up a row; the label
numbers are "padded out" to the next perfect tree that could encompass the entire forest.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

d89952d BIP181: Add the Utreexo accumulator BIP:

Two representations are used in this section, and I feel it confusing, it's easy to mix up. One is the forest -- several perfect disjoint binary trees, and the single incomplete tree. It's confusing that there can be gaps when previous paragraphs talked about perfect trees.

The labeling convention is more than labeling, it is a different representation of the forest, within a single tree. I think this needs more explanation, how the two representations are equivalent, how to construct the tree (e.g. that the larger tree is always put first, etc).

Comment thread bip-0181.md
00 01 02 03 04 05 06 07
```

While a forest with 7 leaves will look like this:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

d89952d BIP181: Add the Utreexo accumulator BIP:

"a forest with 8 leaves will have a single tree"; this forest with 7 leaves also looks like a single tree in this representation (see my previous note on my confusion on the representations).

Comment thread bip-0182.md

## Motivation

Although Utreexo in its current form is not proposed as a soft fork, it is essential that

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4aa26f3 BIP182: Add the Utreexo validation BIP:

Mentioning a soft fork at the beginning of the document raises more questions than it answers, I think -- was it considered to be a soft fork? How? Why not? etc.
I propose moving the reference to soft fork possibility to the end of the document, hinting at how this could be the basis of a soft fork in the future.

Comment thread bip-0182.md

| Name | Type | Description |
| ------------------ | ------------------------ | ----------------------------------------- |
| Utreexo_Tag_V1 | 64-byte array | The version tag to be prepended to the leafhash. |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4aa26f3 BIP182: Add the Utreexo validation BIP:

The "to the leafhash" part here is confusing, it is prepended to the hash preimage. Suggestion: simply "Version tag prefix".

Comment thread bip-0182.md
| BlockHash | 32-byte array | The hash of the block in which this tx was confirmed. |
| TXID | 32-byte array | The transaction's TXID |
| Vout | 4-byte unsigned integer | The output index of this UTXO |
| Header code | 4-byte unsigned integer | The block height and iscoinbase. This is a value obtained by left shifting the block height that confirmed this transaction by one bit, and then OR-ing it with 1, only if this transaction is a coinbase. |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4aa26f3 BIP182: Add the Utreexo validation BIP:

What is the point of including the iscoinbase flag into the block height, when prehash data size is not an issue here? (2x64 bytes are used only for the version prefix) Would it be cleaner to include in two independent fields?

Comment thread bip-0182.md
Comment on lines +72 to +73
| Output script size | varint | The output script length in bytes |
| Output script | variable byte array | The output script of the UTXO |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4aa26f3 BIP182: Add the Utreexo validation BIP:

Output script: It is mentioned that this is included to commit to the output script. If this is the case, I suggest using a hash of the output script, because this would get rid of the complications introduced by a variable-size field (this is the only non-fixed length field).
On the other hand, from other parts of the spec, I understand that the output script is needed by light clients validating the transaction. If this is the case, I suggest changing the "is added to commit to the output script" statement, as it is misleading.

Comment thread bip-0182.md
Comment on lines +208 to +210
The UTXO proof has 2 elements: the accumulator proof and the leaf data. The
leaf data provides the necessary UTXO data for block validation that would be
stored locally for non-Utreexo nodes. Non-Utreexo nodes store this data (under "chainstate/" for Bitcoin Core)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4aa26f3 BIP182: Add the Utreexo validation BIP:

I'm confused by this sentence. "stored locally BY non-Utreexo nodes"?

Comment thread bip-0182.md
Comment on lines +214 to +215
in the accumulator. Without this accumulator proof, the Utreexo nodes would not have
a way to ensure that the given UTXO data exists in the UTXO set. The accumulator proof

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4aa26f3 BIP182: Add the Utreexo validation BIP:

Extra clarification "the UTXOs referenced by the given data" or "the given referenced UTXO data"

Comment thread bip-0182.md
As the amount of Bitcoin users grow, the UTXO set grows with it.

The UTXO set is currently around 10GB in 2025 and with pruning that's all it takes to maintain a full node.
However, as the UTXO set grows, the disk storage requirement will grow along with it and increase the barrier to running a full node.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4aa26f3 BIP182: Add the Utreexo validation BIP:

"a full node." --> "a fully validating node" (it may be pruned)

@murchandamus

Copy link
Copy Markdown
Member

After chatting with Davidson, my understanding is that there will be drastic changes to the document, so I suggest that further review is postponed until the authors update their submission to the swiftsync-based approach.

@4xvgal 4xvgal mentioned this pull request Oct 16, 2026
3 of 8 tasks
fametrano added a commit to btclib-org/btclib that referenced this pull request Oct 21, 2026
…2590)

`btclib.utreexo` holds `Stump` (the roots and the leaf count), `Proof`
and `UpdateData`: add leaves, delete proven leaves, verify a proof,
update a cached proof across a block, and read and write rustreexo's
bytes with `parse` and `serialize`. It follows the draft BIP 181 and
BIP 182's order of a block's changes (bitcoin/bips#1923), and
mit-dci/rustreexo at tag v0.7.0, commit b73cbac1f4, with
utreexo/utreexo v0.18.0 as the second reference.

rustreexo's `test_values/` files are vendored under
`tests/_data/rustreexo/` and pinned in `tests/_data/README.md`, and
every case of them runs. The crate is licensed `MIT OR Apache-2.0`.

The hypothesis properties check `Stump` against a model that builds the
trees by recursion. The issue's property "adding then deleting leaves
gives the roots of a forest built without them" does not hold: deleting
leaf `a` of `[a, b, c, d]` leaves `b` beside the node of `c` and `d`,
where a forest of `[b, c, d]` pairs `b` with `c`. The test states the
property that holds, with that counterexample.

Left for the next slice of
[ISS 2590](#2590): BIP 182's
leaf hashes, Floresta's `roots.json` reproduced from `blocks.json`, the
informational job that compares btclib's roots with `florestad` and
`utreexod`, and the utreexod formats: the untagged stump of the Go
library and the wire encoding of a proof. Not implemented: `Proof`
subsetting (`get_proof_subset`).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Ferdinando Ametrano <ferdinando@ametrano.net>
fametrano added a commit to btclib-org/btclib that referenced this pull request Oct 21, 2026
…2590)

`btclib.utreexo` holds `Stump` (the roots and the leaf count), `Proof`
and `UpdateData`: add leaves, delete proven leaves, verify a proof,
update a cached proof across a block, and read and write rustreexo's
bytes with `parse` and `serialize`. It follows the draft BIP 181 and
BIP 182's order of a block's changes (bitcoin/bips#1923), and
mit-dci/rustreexo at tag v0.7.0, commit b73cbac1f4, with
utreexo/utreexo v0.18.0 as the second reference.

rustreexo's `test_values/` files are vendored under
`tests/_data/rustreexo/` and pinned in `tests/_data/README.md`, and
every case of them runs. The crate is licensed `MIT OR Apache-2.0`.

The hypothesis properties check `Stump` against a model that builds the
trees by recursion. The issue's property "adding then deleting leaves
gives the roots of a forest built without them" does not hold: deleting
leaf `a` of `[a, b, c, d]` leaves `b` beside the node of `c` and `d`,
where a forest of `[b, c, d]` pairs `b` with `c`. The test states the
property that holds, with that counterexample.

Left for the next slice of
[ISS 2590](#2590): BIP 182's
leaf hashes, Floresta's `roots.json` reproduced from `blocks.json`, the
informational job that compares btclib's roots with `florestad` and
`utreexod`, and the utreexod formats: the untagged stump of the Go
library and the wire encoding of a proof. Not implemented: `Proof`
subsetting (`get_proof_subset`).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Ferdinando Ametrano <ferdinando@ametrano.net>
fametrano added a commit to btclib-org/btclib that referenced this pull request Oct 30, 2026
…2590)

`btclib.utreexo` holds `Stump` (the roots and the leaf count), `Proof`
and `UpdateData`: add leaves, delete proven leaves, verify a proof,
update a cached proof across a block, and read and write rustreexo's
bytes with `parse` and `serialize`. It follows the draft BIP 181 and
BIP 182's order of a block's changes (bitcoin/bips#1923), and
mit-dci/rustreexo at tag v0.7.0, commit b73cbac1f4, with
utreexo/utreexo v0.18.0 as the second reference.

rustreexo's `test_values/` files are vendored under
`tests/_data/rustreexo/` and pinned in `tests/_data/README.md`, and
every case of them runs. The crate is licensed `MIT OR Apache-2.0`.

The hypothesis properties check `Stump` against a model that builds the
trees by recursion. The issue's property "adding then deleting leaves
gives the roots of a forest built without them" does not hold: deleting
leaf `a` of `[a, b, c, d]` leaves `b` beside the node of `c` and `d`,
where a forest of `[b, c, d]` pairs `b` with `c`. The test states the
property that holds, with that counterexample.

Left for the next slice of
[ISS 2590](#2590): BIP 182's
leaf hashes, Floresta's `roots.json` reproduced from `blocks.json`, the
informational job that compares btclib's roots with `florestad` and
`utreexod`, and the utreexod formats: the untagged stump of the Go
library and the wire encoding of a proof. Not implemented: `Proof`
subsetting (`get_proof_subset`).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Ferdinando Ametrano <ferdinando@ametrano.net>
fametrano added a commit to btclib-org/btclib that referenced this pull request Nov 1, 2026
…2590)

`btclib.utreexo` holds `Stump` (the roots and the leaf count), `Proof`
and `UpdateData`: add leaves, delete proven leaves, verify a proof,
update a cached proof across a block, and read and write rustreexo's
bytes with `parse` and `serialize`. It follows the draft BIP 181 and
BIP 182's order of a block's changes (bitcoin/bips#1923), and
mit-dci/rustreexo at tag v0.7.0, commit b73cbac1f4, with
utreexo/utreexo v0.18.0 as the second reference.

rustreexo's `test_values/` files are vendored under
`tests/_data/rustreexo/` and pinned in `tests/_data/README.md`, and
every case of them runs. The crate is licensed `MIT OR Apache-2.0`.

The hypothesis properties check `Stump` against a model that builds the
trees by recursion. The issue's property "adding then deleting leaves
gives the roots of a forest built without them" does not hold: deleting
leaf `a` of `[a, b, c, d]` leaves `b` beside the node of `c` and `d`,
where a forest of `[b, c, d]` pairs `b` with `c`. The test states the
property that holds, with that counterexample.

Left for the next slice of
[ISS 2590](#2590): BIP 182's
leaf hashes, Floresta's `roots.json` reproduced from `blocks.json`, the
informational job that compares btclib's roots with `florestad` and
`utreexod`, and the utreexod formats: the untagged stump of the Go
library and the wire encoding of a proof. Not implemented: `Proof`
subsetting (`get_proof_subset`).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Ferdinando Ametrano <ferdinando@ametrano.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

New BIP PR Author action required Needs updates, has unaddressed review comments, or is otherwise waiting for PR author

Projects

None yet

Development

Successfully merging this pull request may close these issues.