Skip to content
This repository has been archived by the owner on Jun 20, 2023. It is now read-only.
/ go-ipfs-chunker Public archive

go-ipfs-chunkers provides Splitter implementations for data before being ingested to IPFS

License

Notifications You must be signed in to change notification settings

ipfs/go-ipfs-chunker

Repository files navigation

go-ipfs-chunker

go-ipfs-chunker implements data Splitters for go-ipfs.

standard-readme compliant GoDoc Build Status

❗ This repo is no longer maintained.

👉 We highly recommend switching to the maintained version at https://github.com/ipfs/boxo/tree/main/chunker. 🏎️ Good news! There is tooling and documentation to expedite a switch in your repo.

⚠️ If you continue using this repo, please note that security fixes will not be provided (unless someone steps in to maintain it).

📚 Learn more, including how to take the maintainership mantle or ask questions, here.

Summary

go-ipfs-chunker provides the Splitter interface. IPFS splitters read data from a reader an create "chunks". These chunks are used to build the ipfs DAGs (Merkle Tree) and are the base unit to obtain the sums that ipfs uses to address content.

The package provides a SizeSplitter which creates chunks of equal size and it is used by default in most cases, and a rabin fingerprint chunker. This chunker will attempt to split data in a way that the resulting blocks are the same when the data has repetitive patterns, thus optimizing the resulting DAGs.

Table of Contents

Install

go-ipfs-chunker works like a regular Go module:

> go get github.com/ipfs/go-ipfs-chunker

Usage

import "github.com/ipfs/go-ipfs-chunker"

Check the GoDoc documentation

License

MIT © Protocol Labs, Inc.

About

go-ipfs-chunkers provides Splitter implementations for data before being ingested to IPFS

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages