Skip to content

Adding a offset based chunker #241

@RangerMauve

Description

@RangerMauve

Hello I'm here on behalf of the WebRecorder project.

We're planning on adding a new chunker to our codebase for making it easier to specify which segments of a file stream should be chunked.

We're calling this an offset chunker which works by taking an array of byte offsets and creates chunks at those boundries.

This chunker can be useful for stuff like archives or bundles where you know that there are chunks which may get reused between roots, or even for cases where you know the general structure of the content ahead of time but that it doesn't fit into the rabin or fixed-size chunks.

Once we get one working, would you be interested in a PR which adds this sort of chunker to the repo as well as docs and unit tests?

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions