Skip to content
Elastos Storage for Hive Cluster
Go Shell Other
Branch: dev-master
Clone or download
Pull request Compare This branch is 16 commits ahead, 656 commits behind ipfs:master.
Yi Wang
Yi Wang Update ChangeLog for release-v0.0.1
Signed-off-by: Yi Wang <wangyi@storswift.com>
Latest commit 6646f2c Apr 14, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github Fix ipfs#412: Add an issue template May 30, 2018
.gx update file/get API; (#8) Jan 21, 2019
.snapcraft New credentials for pushing snaps Jan 10, 2018
adder merge with ipfs cluster master (#5) Jan 21, 2019
allocator Pre release hive alpha v0.1.2 (#1) Jan 17, 2019
api Fix user login and cp (#15) Mar 20, 2019
ci Pre-create and pre-connect hosts in tests Apr 5, 2018
cmd merge with ipfs cluster master (#5) Jan 21, 2019
config merge with ipfs cluster master (#5) Jan 21, 2019
consensus/raft Pre release hive alpha v0.1.2 (#1) Jan 17, 2019
deptools release hive_alpha-v0.1.1 Jan 13, 2019
docker Docker: Run with daemon --upgrade by default. Apr 27, 2018
extras/LinuxDeb enable user keys discovery and synchronising in cluster. (#12) Jan 31, 2019
informer Pre release hive alpha v0.1.2 (#1) Jan 17, 2019
ipfsconn/ipfshttp Fix user login and cp (#15) Mar 20, 2019
keystore enable user keys discovery and synchronising in cluster. (#12) Jan 31, 2019
monitor Pre release hive alpha v0.1.2 (#1) Jan 17, 2019
pintracker merge with ipfs cluster master (#5) Jan 21, 2019
pstoremgr Pre release hive alpha v0.1.2 (#1) Jan 17, 2019
rpcutil fix bugs in uid synchronization; sync uid/renew action to all of clus… Feb 1, 2019
sharness merge with ipfs cluster master (#5) Jan 21, 2019
snap Pre release hive alpha v0.1.2 (#1) Jan 17, 2019
state Pre release hive alpha v0.1.2 (#1) Jan 17, 2019
test merge with ipfs cluster master (#5) Jan 21, 2019
version merge with ipfs cluster master (#5) Jan 21, 2019
.codeclimate.yml codeclimate Aug 7, 2018
.dockerignore Issue ipfs#259: Address CR comments Dec 4, 2017
.gitignore Add a docker-compose.yml example with 2 peers and auto-bootstrap. Oct 23, 2018
.travis.yml Fix publishing snaps Oct 25, 2018
CHANGELOG.ipfs-cluster.md Update ChangeLog for release-v0.0.1 Apr 15, 2019
CHANGELOG.md Update ChangeLog for release-v0.0.1 Apr 15, 2019
CONTRIBUTING.md Docs: Move to website. Apr 27, 2018
Dockerfile Pre release hive alpha v0.1.2 (#1) Jan 17, 2019
Dockerfile-bundle Pre release hive alpha v0.1.2 (#1) Jan 17, 2019
Dockerfile-test Pre release hive alpha v0.1.2 (#1) Jan 17, 2019
LICENSE Update LICENSE (#3) Jan 18, 2019
Makefile Set CLUSTER_SECRET from shell Oct 24, 2018
README.ipfs-cluster.md Pre release hive alpha v0.1.2 (#1) Jan 17, 2019
README.md update deb package facility (#10) Jan 24, 2019
add_test.go
allocate.go Pre release hive alpha v0.1.2 (#1) Jan 17, 2019
cluster.go fix uid verifing bug. (#16) Apr 15, 2019
cluster_config.go Pre release hive alpha v0.1.2 (#1) Jan 17, 2019
cluster_config_test.go add env config; refactor LoadJSON tests Oct 26, 2018
cluster_test.go Pre release hive alpha v0.1.2 (#1) Jan 17, 2019
clusterhost.go release hive_alpha-v0.1.1 Jan 13, 2019
config_test.go Pre release hive alpha v0.1.2 (#1) Jan 17, 2019
connect_graph.go Pre release hive alpha v0.1.2 (#1) Jan 17, 2019
docker-compose.yml Set CLUSTER_SECRET from shell Oct 24, 2018
ipfscluster.go Fix user login and cp (#15) Mar 20, 2019
ipfscluster_test.go Pre release hive alpha v0.1.2 (#1) Jan 17, 2019
logging.go Unrewrite deps Dec 17, 2018
package.json enable user keys discovery and synchronising in cluster. (#12) Jan 31, 2019
peer_manager_test.go Pre release hive alpha v0.1.2 (#1) Jan 17, 2019
pnet_test.go Feat: Enable DHT-based peer discovery and routing for cluster peers Jul 24, 2018
release.sh Fix ipfs#382: Extract headers from IPFS API requests & apply them to … Dec 18, 2018
rpc_api.go Fix user login and cp (#15) Mar 20, 2019
util.go Pre release hive alpha v0.1.2 (#1) Jan 17, 2019

README.md

Elastos Hive Cluster

img img standard-readme compliant Build Status

Introduction

Elastos Hive Cluster is a decentralized File Storage Service that based on IPFS cluster. Hive Cluster use IPFS and IPFS-cluster as the base infrastructure to save Elastos data. Hive Cluster is also a stand-alone application as same as IPFS Cluster.

The typical IPFS peer is a resource hunger program. If you install IPFS daemon to your mobile device, it will take up resources and slow down your device. We are creating Hive project, which uses IPFS cluster as the Elastos App storage backend, and can be used in a low resources consumption scenario.

The project distils from the IPFS Cluster, but it will have many differences with the IPFS Cluster.

Elastos Hive Cluster maintains a big IPFS pinset for sharing. It can serve numerous virtual IPFS peers with only one IPFS peer instance running.

Hive Cluster is not only a pinset manager but also a backend for multiple IPFS clients.

Table of Contents

Install

The following requirements apply to the installation from source:

  • Git
  • Go 1.11+
  • IPFS or internet connectivity (to download depedencies).

Install Go

The build process for cluster requires Go 1.10 or higher. Download propriate binary archive from golang.org, and install it onto specific directory:

$ curl go1.11.4.linux-amd64.tar.gz -o golang.tar.gz
$ tar -xzvf golang.tar.gz -C YOUR-INSTALL-PATH

Then add path YOUR-INSTALL-PATH/go/bin to user environment variable PATH.

$ export PATH="YOUR-INSTALL-PATH/go/bin:$PATH"

Besides that, build environment for golang projects must be required to setup:

$ export GOPATH="YOUR-GO-PATH"
$ export PATH="$GOPATH/bin:$PATH"

In convinience, just add these lines to your profile $HOME/.profile, then validate it with the command:

$ . $HOME/.profile
$ go version

Then use export command to check if it validated or not.

Notice : If you run into trouble, see the Go install instructions.

Download Source

There are two ways to setup your source code. One is directly to download source code under $GOPATH environment as below:

$ cd $GOPATH/src/github.com/elastos
$ git clone https://github.com/elastos/Elastos.NET.Hive.Cluster.git

The other way is to download source to specific location, and then create linkage to that directory underp propriate $GOPATH environment:

$ cd YOUR-PATH
$ git clone https://github.com/elastos/Elastos.NET.Hive.Cluster.git 
$ cd $GOPATH/src/github.com/elastos
$ link -s YOUR-PATH/Elastos.NET.Hive.Cluster Elastos.NET.Hive.Cluster

Build Cluster

To build Cluster with the following commands:

$ cd $GOPATH/src/github.com/elastos/Elastos.NET.Hive.Cluster
$ make
$ make install

After installation, ipfs-cluster-service and ipfs-cluster-ctl would be generated under the directory $GOPATH/bin

If you would rather have them built locally, use make build instead. You can run make clean to remove any generated artifacts and rewrite the import paths to their original form.

Note that when the ipfs daemon is running locally on its default ports, the build process will use it to fetch gx, gx-go and all the needed dependencies directly from IPFS.

Building Debian package

For Linux Debian based systems. This project can generate a deb package for to distribute conveniently.

Please make sure that the IPFS and IPFS-Cluster projects were built and the binraries (ipfs, ipfs-cluster-service, ipfs-cluster-ctl) had been generated under the directory $GOPATH/bin.

To generate a deb package with the following commands:

$ cd $GOPATH/src/github.com/elastos/Elastos.NET.Hive.Cluster/extras/LinuxDeb
$ make

After the deb package generated, it will have a hive-dist-???.deb file in the current directory. You can copy it to and install it on the Debian systems. Or, you can run make clean to remove generated.

# Install package on Linux Debian based systems
$ dpkg -i hive-dist-???.deb

Usage

ipfs-cluster-service is a command line program to start cluster daemon, while ipfs-cluster-ctl is the client application to manage the cluster nodes and perform actions.

Run the following commands with hep options to see more usages:

$ ipfs-cluster-service help
...
$ ipfs-cluster-ctl help
...

Details about ipfs cluster please refer to the docs from https://cluster.ipfs.io.

Hive cluster uses HTTP interface to serve numerous clients. About HTTP API, please refer to

Get Started

Hive cluster have to be running before starting Hive cluster. About how to run Hive IPFS daemon, please refer to Elastos Hive IPFS.

Initialize config files

To start using Cluster, you must first initialize Cluster's config files on your system, which would be done with the command below:

$ ipfs-cluster-service init
$ ls $HOME/.ipfs-cluster
service.json

See ipfs-cluster-service help for more information on the optional arguments it takes.

Run as Daemon

After initialization and configure, try to run ipfs-cluster-service daemon:

$ ipfs-cluster-service daemon &

then you can use ipfs-cluster-ctl program to check if cluster is running

$ ipfs-cluster-ctl id
QmUGXgnUcqgZf9Js7GrvFP1uz7G6soq3urWk9Gz237DsQm | guest | Sees 1 other peers
  > Addresses:
    - /ip4/127.0.0.1/tcp/9096/ipfs/QmUGXgnUcqgZf9Js7GrvFP1uz7G6soq3urWk9Gz237DsQm
    - /ip4/222.222.222.222/tcp/9096/ipfs/QmUGXgnUcqgZf9Js7GrvFP1uz7G6soq3urWk9Gz237DsQm
    - /p2p-circuit/ipfs/QmUGXgnUcqgZf9Js7GrvFP1uz7G6soq3urWk9Gz237DsQm
  > IPFS: QmSPDCbxBSq7PYAnemABL7VpGRkpqBX9NT1AovEmzaXkvM
    - /ip4/127.0.0.1/tcp/4001/ipfs/QmSPDCbxBSq7PYAnemABL7VpGRkpqBX9NT1AovEmzaXkvM
    - /ip4/222.222.222.222/tcp/4001/ipfs/QmSPDCbxBSq7PYAnemABL7VpGRkpqBX9NT1AovEmzaXkvM
    - /ip6/::1/tcp/4001/ipfs/QmSPDCbxBSq7PYAnemABL7VpGRkpqBX9NT1AovEmzaXkvM

You also can run ipfs-cluster-service as a slave cluster node with following command:

$ ipfs-cluster-service daemon --bootstrap /ip4/222.222.222.222/tcp/9096/ipfs/QmNTD6Zbhdao
DmjQqGJrp8dKEPvtBQGzRxwWHNcmvNYsbK &

Contribution

We welcome contributions to the Elastos Hive Project.

Acknowledgments

A sincere thank you to all teams and projects that we rely on directly or indirectly.

License

This project is licensed under the terms of the MIT license.

You can’t perform that action at this time.