Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use rolling hash splitter (data sensitive, e.g. Buzhash or Rabin) by default instead of SizeSplitter (data insensitive) #355

Open
sideeffffect opened this issue Jun 17, 2023 · 1 comment
Labels
need/triage Needs initial labeling and prioritization

Comments

@sideeffffect
Copy link

Use the Buzhash or Rabin splitter in IPFS instead of the SizeSplitter by default.
A rolling hash based data sensitive splitter has a huge obvious advantage: it creates shift-resistant chunks and thus improves data deduplication and sharing.

For example casync is doing it too:

Original ticket: ipfs/go-ipfs-chunker#13

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
need/triage Needs initial labeling and prioritization
Projects
None yet
Development

No branches or pull requests

1 participant