Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: spks_of_all_keychains() shouldn't return an infinite iterator for non-wildcard descriptors #1093

Merged

Conversation

danielabrozzoni
Copy link
Member

@danielabrozzoni danielabrozzoni commented Aug 24, 2023

Description

When you pass a non-wildcard descriptor in new_with_range, we make
sure that the range length is at most 1; if that's not the case, we
shorten it.
We would previously use new_with_range without this check and with a
non-wildcard descriptor in spks_of_all_keychains, this meant creating
a spkiterator that would go on producing the same spks over and over
again, causing some issues with syncing on electrum/esplora.

To reproduce the bug, run in example-crates/example_electrum:

cargo run "sh(wsh(or_d(c:pk_k(cPGudvRLDSgeV4hH9NUofLvYxYBSRjju3cpiXmBg9K8G9k1ikCMp),c:pk_k(cSBSBHRrzqSXFmrBhLkZMzQB9q4P9MnAq92v8d9a5UveBc9sLX32))))#zp9pcfs9" scan

Changelog notice

  • Fixed a bug where KeychainTxOutIndex::spks_of_all_keychains/KeychainTxOutIndex::spks_of_keychain would return an iterator yielding infinite spks even for non-wildcard descriptors.

Checklists

All Submissions:

  • I've signed all my commits
  • I followed the contribution guidelines
  • I ran cargo fmt and cargo clippy before committing

@danielabrozzoni danielabrozzoni changed the title fix: Use SpkIterator::new in spks_of{_all}_keychains fix: spks_of_all_keychains() shouldn't return an infinite iterator for non-wildcard descriptors Aug 24, 2023
@notmandatory notmandatory added this to the 1.0.0-alpha.2 milestone Aug 24, 2023
@danielabrozzoni danielabrozzoni mentioned this pull request Aug 24, 2023
4 tasks
Copy link
Member

@evanlinjin evanlinjin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for spotting and fixing this @danielabrozzoni!

I think SpkIterator::new_with_range should also detect non-wildcard descriptors, and the implementation of SpkIterator::new can be changed to this:

    /// Creates a new script pubkey iterator starting at 0 from a descriptor.
    pub fn new(descriptor: D) -> Self {
        SpkIterator::new_with_range(descriptor, 0..=BIP32_MAX_INDEX)
    }

This means, SpkIterator:new_with_range will sometimes return empty iterators.

@LLFourn
Copy link
Contributor

LLFourn commented Aug 25, 2023

+1 for what @evanlinjin suggested above.

@danielabrozzoni
Copy link
Member Author

danielabrozzoni commented Aug 25, 2023

I think SpkIterator::new_with_range should also detect non-wildcard descriptors, and the implementation of SpkIterator::new can be changed to this:

I tried to implement this by setting end = next_index in new_with_range for non-wildcard descriptors, but I'm seeing some tests failing, investigating now...

..into account

When you pass a non-wildcard descriptor in `new_with_range`, we make
sure that the range length is at most 1; if that's not the case, we
shorten it.
We would previously use `new_with_range` without this check and with a
non-wildcard descriptor in `spks_of_all_keychains`, this meant creating
a spkiterator that would go on producing the same spks over and over
again, causing some issues with syncing on electrum/esplora.

To reproduce the bug, run in `example-crates/example_electrum`:
```
cargo run "sh(wsh(or_d(c:pk_k(cPGudvRLDSgeV4hH9NUofLvYxYBSRjju3cpiXmBg9K8G9k1ikCMp),c:pk_k(cSBSBHRrzqSXFmrBhLkZMzQB9q4P9MnAq92v8d9a5UveBc9sLX32))))#zp9pcfs9" scan
```
@danielabrozzoni
Copy link
Member Author

I tried to implement this by setting end = next_index in new_with_range for non-wildcard descriptors, but I'm seeing some tests failing, investigating now...

Turns out that this was wrong. I'm now shortening whichever range passed to new_with_range to have length at most 1 if the descriptor doesn't have a wildcard, and it works perfectly. Lmk what you think :)

Copy link
Member

@evanlinjin evanlinjin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing this. This was my oversight.

As discussed (via DM), it'll be better to have non-wildcard descriptors only returns something on derivation index 0.

The follow tests should pass:

// non index-0 should NOT return an spk
assert_eq!(SpkIterator::new_with_range(&no_wildcard_descriptor, 1..1).next(), None);
assert_eq!(SpkIterator::new_with_range(&no_wildcard_descriptor, 1..=1).next(), None);
assert_eq!(SpkIterator::new_with_range(&no_wildcard_descriptor, 1..2).next(), None);
assert_eq!(SpkIterator::new_with_range(&no_wildcard_descriptor, 1..=2).next(), None);

crates/chain/src/spk_iter.rs Outdated Show resolved Hide resolved
Copy link
Member

@evanlinjin evanlinjin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK e48b911

Tested with example_electrum with non-wildcard descriptor.
Left a non-critical nit.

crates/chain/src/spk_iter.rs Show resolved Hide resolved
@evanlinjin evanlinjin merged commit 2867e88 into bitcoindevkit:master Sep 1, 2023
12 checks passed
@danielabrozzoni danielabrozzoni deleted the fix/electrum_scan_bug branch October 11, 2023 12:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

None yet

4 participants