Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Field: More efficient conversion into Bignum_bigint.t #14599

Conversation

tizoc
Copy link
Member

@tizoc tizoc commented Nov 22, 2023

As discussed here, this is a more efficient conversion into Bignum_bigint.t values that avoids processing the input and output bit by bit.

It feels kind of dirty having to go through bin_prot for this, but Bignum_bigint doesn't expose any other way to interpret raw bytes to produce a bigint (which should be more efficient than the alternatives). A pro of this approach is that it didn't require any new function to be exposed (other than the compare added in the other field), but assumes that all implementations of to_bytes will produce the same output.

Explain how you tested your changes:
*

Checklist:

  • Dependency versions are unchanged
    • Notify Velocity team if dependencies must change in CI
  • Modified the current draft of release notes with details on what is completed or incomplete within this project
  • Document code purpose, how to use it
    • Mention expected invariants, implicit constraints
  • Tests were added for the new behavior
    • Document test purpose, significance of failures
    • Test names should reflect their purpose
  • All tests pass (CI will check this if you didn't)
  • Serialized types are in stable-versioned modules
  • Does this close issues? List them
  • Closes #0000

@tizoc tizoc changed the title Optimization/efficient bignum bigint conv More efficient conversion of Bigint values into Bignum_bigint.t Nov 22, 2023
@tizoc tizoc force-pushed the optimization/efficient-bignum-bigint-conv branch from 0b05b93 to 709463d Compare November 22, 2023 21:32
@tizoc tizoc changed the title More efficient conversion of Bigint values into Bignum_bigint.t Field: More efficient conversion into Bignum_bigint.t Nov 22, 2023
@tizoc
Copy link
Member Author

tizoc commented Nov 22, 2023

!ci-build-me

@tizoc
Copy link
Member Author

tizoc commented Nov 23, 2023

If I try these patches rebased on top of rampup the tests don't fail when I run dune runtest src/lib --profile=dev.

@tizoc tizoc marked this pull request as ready for review November 23, 2023 11:16
@tizoc tizoc requested a review from a team as a code owner November 23, 2023 11:16
@tizoc
Copy link
Member Author

tizoc commented Nov 23, 2023

Just noticed that there is another implementation of that function in snarky with the same issue:

https://github.com/o1-labs/snarky/blob/7edf13628872081fd7cad154de257dad8b9ba621/src/base/backend_extended.ml#L101-L110

@georgeee georgeee requested a review from a team as a code owner November 27, 2023 20:17
@tizoc tizoc changed the base branch from feature/ledger-mask-maps to rampup November 27, 2023 20:35
@tizoc tizoc force-pushed the optimization/efficient-bignum-bigint-conv branch from 709463d to 6fcd939 Compare November 27, 2023 20:37
Bignum_bigint.hash_fold_t s (to_bignum_bigint (to_bigint x))
(* For non-zero values, conversion is done by creating the bin_prot representation
of the [Bignum_bigit.t] value, and then parsing it with bin_prot. *)
match compare n zero with
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this necessary? The bigint representation will always be positive, so it should be fine (and actually more efficient) to just special-case zero using an equality check.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mrmr1993 not at all, the only purpose of all this logic is to:

  1. Recover the sign because bin_prot requires it
  2. Avoid bypassing bignum when converting, but if that is not an issue I can use Zarith directly

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mrmr1993 btw, are there any values that are common and are worth short-circuiting like zero? do you think it is worth adding one too ?

Bytes.unsafe_set buf 1 size_byte ;
(* Copy the bytes representation of the value, skip the tag and size bytes *)
Bytes.unsafe_blit ~src:bytes ~src_pos:0 ~dst_pos:2 ~len ~dst:buf ;
Bin_prot.Reader.of_bytes Bignum_bigint.Stable.V1.bin_reader_t buf
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you not use Z.of_bits directly? There should be a no-op converter to/from Bigint.t from there.

@tizoc
Copy link
Member Author

tizoc commented Nov 27, 2023

@mrmr1993 got rid of the intermediary binprot buffer construction and re-implemented the conversion using Zarith directly. Current version special-cases 0 and 1 so that the bytes representation is not created.

@tizoc
Copy link
Member Author

tizoc commented Nov 27, 2023

!ci-build-me

@deepthiskumar
Copy link
Member

!ci-nightly-me

@deepthiskumar
Copy link
Member

!ci-nightly-me

@deepthiskumar deepthiskumar merged commit 41de362 into MinaProtocol:rampup Dec 4, 2023
2 checks passed
@tizoc tizoc deleted the optimization/efficient-bignum-bigint-conv branch December 4, 2023 22:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants