-
Notifications
You must be signed in to change notification settings - Fork 33
Closed
Labels
Description
Hello I'm here on behalf of the WebRecorder project.
We're planning on adding a new chunker to our codebase for making it easier to specify which segments of a file stream should be chunked.
We're calling this an offset chunker which works by taking an array of byte offsets and creates chunks at those boundries.
This chunker can be useful for stuff like archives or bundles where you know that there are chunks which may get reused between roots, or even for cases where you know the general structure of the content ahead of time but that it doesn't fit into the rabin or fixed-size chunks.
Once we get one working, would you be interested in a PR which adds this sort of chunker to the repo as well as docs and unit tests?