Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Huffman Bug Fix #699

Merged
merged 2 commits into from Nov 23, 2021
Merged

Conversation

JeremyRubin
Copy link
Contributor

I noticed one cleanup & one bugfix while looking into the huffman algorithm:

  1. the cleanup: we can use a u128 to guarantee no overflows, and saturating_add to guarantee reasonable behavior in any case
  2. the bug: the binary heap is a max heap so the behavior ends up merging the nodes of the most likely entries repeatedly. a huffman encoder requires merging the least likely elements, so it should be reversed.

@JeremyRubin
Copy link
Contributor Author

a currently failing test #700

that passes if rebased on this branch.

Copy link
Collaborator

@Kixunil Kixunil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think u128 is not supported on all platforms. Could it be a problem for embedded?

// N.B.: p1 + p2 can never actually saturate as you would need to have 2**64 max u64s
// from the input to overflow. However, saturating is a reasonable behavior here as
// huffman tree construction would treat all such elements as "very likely".
let p = Reverse(p1.0.saturating_add(p2.0));
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it can never happen, why not just have +?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'd rather define the behavior correctly here should it ever overflow rather than wrapping around. Perhaaps defensively, but it'd be possible someone were to update the types/sizes later without appreciating this point.

@JeremyRubin
Copy link
Contributor Author

I think u128 is supported on all platforms, either with a native type or a wrapper. We could also do the same trick with u32 and u64 and it wouldn't be an issue either (u32 is plenty IMO).

Copy link
Member

@sanket1729 sanket1729 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for testing this. ACK modulo removing the error variant.

@@ -220,7 +221,10 @@ impl TaprootSpendInfo {
let (p1, s1) = node_weights.pop().expect("len must be at least two");
let (p2, s2) = node_weights.pop().expect("len must be at least two");
// Insert the sum of first two in the tree as a new node
let p = p1.checked_add(p2).ok_or(TaprootBuilderError::ScriptWeightOverflow)?;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also remove the Error variant from the error enum?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@sanket1729 sanket1729 added the bug label Nov 15, 2021
@sanket1729 sanket1729 added this to the 0.28.0 milestone Nov 15, 2021
Copy link
Member

@sanket1729 sanket1729 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK f2a6827

Copy link
Collaborator

@dr-orlovsky dr-orlovsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK f2a6827

@dr-orlovsky dr-orlovsky merged commit 5286d0a into rust-bitcoin:master Nov 23, 2021
dr-orlovsky added a commit that referenced this pull request Dec 11, 2021
1518517 Decrease Huffman weight type to 32 bits (Jeremy Rubin)

Pull request description:

  This builds on #699 but is the more bikesheddable part since it changes the API.

  > u32 of weight should be enough for any branch.
  -- Bill Gates

ACKs for top commit:
  dr-orlovsky:
    utACK 1518517
  Kixunil:
    ACK 1518517

Tree-SHA512: 9c507ae6129dda8dc069b0a142181a78cf89cb3ebf9d2169c46662822cb4ea9ed075bf484528f5399fe0ed383a425174a702e2d685f31c246f5a86c46ed17c3a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants