Skip to content
A tool to transfer data between IPFS nodes or storage
Go
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
pump
.gitignore
LICENSE
README.md
go.mod
go.sum
main.go
pump_test.go

README.md

ipfs-pump

ipfs-pump is a command line tool to copy data between IPFS nodes, cluster or storage.

It support multiple interfaces:

  • the IPFS API
  • direct access to a FlatFS datastore
  • direct access to a Badger datastore
  • direct access to a S3 datastore
  • a file with a list of CID

Concepts

An Enumerator is a source that will enumerate the list of existing blocks.

A Collector is a source that where to retrieve block's data.

A Drain is a destination where to push blocks.

Note: even though you will likely want the Enumerator and the Collector to be the same source, they don't necessarily have to.

Install

go get -u github.com/INFURA/ipfs-pump

Usage and examples

You need to provide as arguments the three types of Enumerator, Collector and Drain, as well as the corresponding configuration flags.

Copy between two live nodes using the API:

ipfs-pump \
    apipin --enum-api-pin-url=127.0.0.1:5001 --enum-api-pin-stream \
    api --coll-api-url=127.0.0.1:5001 \
    api --drain-api-url=127.0.0.1:5002 \
    --worker=10

Copy from a FlatFS storage to a Badger storage when the nodes are stopped:

ipfs-pump \
    flatfs --enum-flatfs-path=~/.ipfs/blocks \
    flatfs --coll-flatfs-path=~/.ipfs/blocks \
    badger --drain-badger-path=~/.ipfs/badgerds \
    --worker=50

Parallel processing

Using the --worker flag you can enable parallel processing and greatly increase the throughput.

License

MIT

You can’t perform that action at this time.