Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/optimizations and cleanup. #21

Merged
merged 43 commits into from Jun 11, 2023
Merged

Conversation

bilowik
Copy link
Owner

@bilowik bilowik commented Jun 11, 2023

Changes

A lot of cleanup, a lot of optimizations on all fronts. Most breaking changes occur in just wrapped_sharing, basic_sharing is relatively untouched in terms of API.

General

  • Removed a lot of old code that was no longer in use, including a lot of the geometry module from back before the migration to using finite field arithmetic.
  • Lots of new documentation, lots of cleanup of old documentation.

Wrapped sharing

  • Deprecated most of the original sharing wrapped_sharing functions
  • Added new, much cleaner share/reconstruct functions.
  • Added infinitely more useful, optimized, cleaner Sharer and Reconstructor structs for large files or streams, compared to the old multitude of share/reconstruct functions that sought to handle those cases.
    • Heavily inspired by sha3 hash implementation :)
  • Remove old Error variants that are no longer instantiated.
    • This was missed by linters, they were only ever accessed in the Display impl but never directly instantiated anymore.

Basic sharing

  • Completely remove reliance of the very inefficient matrix transposition required when sharing slices of bytes. Included a lot of cloning, a lot of Vec allocations. The benchmark improvements from this alone were ~30%, and for very large secrets >=65536 bytes, a (potentially misleading) 90% improvement.
    • The time for sharing and reconstructing now also scales very very linearly with the length of secret!
  • Cleaned up a lot of unneeded allocations
  • Replaced all argument instances of &[u8] and Vec<u8> with AsRef<[u8]>
  • Replaced all argument instances of Vec<Vec<u8>> with U: AsRef<[u8]>, T: AsRef<[U]>
  • Remove old Error variants that are no longer instantiated.
    • This was missed by linters, they were only ever accessed in the Display impl but never directly instantiated anymore.

Todo

  • Look for more places where allocations can be avoided.
  • Create benches for share/reconstruct and Sharer/Reconstructor Saving this for another time, issue tracking here:
    • Probably should use Cursor instead of actual File objects for Sharer/Reconstructor for consistency.

bilowik and others added 30 commits June 10, 2023 00:43
This removes reliance on reduce_share by not producing the
initial Vec<(u8, u8)> for each share and directly producing the Vec<u8>
share end product.
Remove a lot of old code bloat that was no longer in use.
@bilowik bilowik changed the title Feature/rework wrapped sharing Feature/optimizations and cleanup. Jun 11, 2023
@bilowik bilowik merged commit c62fcd0 into master Jun 11, 2023
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant