ipfs-pump is a command line tool to copy data between IPFS nodes, cluster or storage.
It support multiple interfaces:
- the IPFS API
- direct access to a FlatFS datastore
- direct access to a Badger datastore
- direct access to a S3 datastore
- a file with a list of CID
Enumerator is a source that will enumerate the list of existing blocks.
Collector is a source that where to retrieve block's data.
Drain is a destination where to push blocks.
Note: even though you will likely want the
Enumerator and the
Collector to be the same source, they don't necessarily have to.
go get -u github.com/INFURA/ipfs-pump
Usage and examples
You need to provide as arguments the three types of
Drain, as well as the corresponding configuration flags.
Copy between two live nodes using the API:
ipfs-pump \ apipin --enum-api-pin-url=127.0.0.1:5001 --enum-api-pin-stream \ api --coll-api-url=127.0.0.1:5001 \ api --drain-api-url=127.0.0.1:5002 \ --worker=10
Copy from a FlatFS storage to a Badger storage when the nodes are stopped:
ipfs-pump \ flatfs --enum-flatfs-path=~/.ipfs/blocks \ flatfs --coll-flatfs-path=~/.ipfs/blocks \ badger --drain-badger-path=~/.ipfs/badgerds \ --worker=50
--worker flag you can enable parallel processing and greatly increase the throughput.