Skip to content
High availability for blockchain services
JavaScript Shell Rust Dockerfile HTML
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
bootstrap
chain
deployer add node-key-file param support for polkadot Feb 21, 2020
doc
orchestrator add node-key-file param support for polkadot Feb 21, 2020
ui [*] Change naming master -> leader + lint runtime Jan 27, 2020
.dockerignore
.gitignore [Security] Add dockerized WireGuard Feb 11, 2020
LICENCE
README.md [orchestator] Service volume support Feb 18, 2020
web3_foundation_grants_badge.svg

README.md

Archipel

stability-wip Website archipel.id License Twitter Follow

Welcome to Archipel monorepository!

What is Archipel ?

Archipel is a high availability solution for blockchain services.

Services Supported

  • Polkadot Validator Node

Why Archipel ?

Nowadays, many blockchain services are centralized on cloud infrastructure. For instance, around 70% of Ethereum nodes are in VPC and 63% of Ethereum Dapps use Infura Provider as this survey shows. We can imagine that in a few years some of blockchain services can be banned from public cloud providers. Moreover, cloud providers can have interruption of services and network issues.

The solution is to have a decentralized infrastructure at home. The problem is that it is challenging to maintain a good quality of service at home. You can have internet connection or power cuts. As a result, it is very unsafe to install a Proof-of-Stake validator at home. Your validator must be always up (24/7) and ready to execute its duty. If not, you will be slashed by the network and lose your money. To solve this problem, we are creating a solution to provide high availability for blockchain services. The first service that we target is Polkadot PoS Validator.

How it works ?

The idea behind Archipel is federating some nodes between friends and family. We are using DAppNode as the infrastructure layer for our solution. We want here to thanks DAppNode Teams for their amazing work and also supports for months now. With DAppNode, you can launch a blockchain node (Bitcoin, Ethereum ...) or any P2P software. To achieve the high availability of services, we are adding a service layer (Archipel) on top of it.

Archipel Components

Component Description
Chain Chain component is responsible for Archipel state synchronization between participants
Orchestator Orchestrator is the decision-making component in Archipel federation
UI UI is the Archipel chain state visualization component
Deployer Archipel End-To-End tests, build scripts and deploy tools
DAppNodePackage DAppNode package wrapping Archipel

Archipel Chain

To federate several nodes and have a shared state to elect leader, we created a specific blockchain using Substrate framework.

Substrate is a Parity framework that allows creating application-specific blockchains.

We created a Substrate runtime that collects all nodes metrics and sets federation leader. This helps Archipel orchestrator to select the best leader appropriately in the federation. We call this specific blockchain the Archipel Substrate Chain or Archipel Chain.

All nodes inside a federation, run Archipel Chain. In the current implementation, an Archipel must be composed of 3 nodes. That means that to operate, you have to set up 3 nodes. Try to set up nodes in different locations using different ISPs.

The idea is that in Archipel federation, all participants are trusted. They can be friends or family or other trusted social links. That allows us to have a fast chain consensus.

Orchestrator

Orchestrator is the component that is responsible for decision making in an Archipel Federation.

External services modes

  • Active mode - the service will be launched in active mode only on the leader node
  • Passive mode - the service will be launched in passive mode on all other non-leader nodes

Archipel Orchestrator Workflow

  • Launch external service in passive mode
  • Send node metrics to Archipel Chain
  • Retrieve other nodes metrics from Archipel Chain
  • Retrieve current leader from Archipel Chain and determine its availability
  • If the current leader is alive, do nothing and ensure that the service was launched in passive mode
  • If the current leader is not alive, try to take its place by making a transaction to Archipel Chain
  • If the transaction succeeded, the current node becomes the current leader of the federation
  • Do supplementary checks (anyone other is alive, Archipel Chain Node has any peers...)
  • If supplementary checks pass launch external service in active mode

Archipel Orchestrator for HA Polkadot Validator Setup

The first service targeted by Archipel is Polkadot Validator.

The Polkadot node can be launched in two modes:

  • Active mode - Polkadot node in with validator option
  • Passive mode - Polkadot node in the sync-only mode

We are also planning to support other services.

UI

The Archipel UI visualizes the federation state.

It shows:

  • the current leader elected
  • all federation nodes metrics

DAppNodePackage

DAppNode package wrapping Archipel stack.

It allows installing Archipel from the DAppNode interface in one click.

Documentation

Please refer to the README instructions in the sub-repositories for more information on building, using, and testing each software component.

Additional documentation

Running with Docker

Archipel (Validator Archipel Chain Node + Orchestrator)

# Create an .env file
cat <<EOF >.env
ARCHIPEL_NODE_ALIAS=archipel1
ARCHIPEL_KEY_SEED=mushroom ladder ...
ARCHIPEL_CHAIN_ADDITIONAL_PARAMS=
POLKADOT_NAME=test-name
POLKADOT_PREFIX=node-
SERVICE=polkadot
POLKADOT_IMAGE=parity/polkadot:latest
POLKADOT_KEY_GRAN=april shift ...
POLKADOT_KEY_BABE=region run ...
POLKADOT_KEY_IMON=screen sustain ...
POLKADOT_KEY_PARA=produce hover ...
POLKADOT_KEY_AUDI=oak tail ...
ARCHIPEL_AUTHORITIES_SR25519_LIST=5FmqMTG...
ARCHIPEL_AUTHORITIES_ED25519_LIST=5FbQNUq...
EOF

# Creating docker volumes
docker volume create archipel
docker volume create archipel_service

# Launch docker container
docker run -d --name "archipel1" \
  -p 30333:30333 \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v archipel:/root/chain/data \
  -v archipel_service:/service \
  --env-file .env \
  luguslabs/archipel:latest

Archipel Node (Non-Validator Archipel Chain Node)

# Create an .env file
cat <<EOF >.env
ARCHIPEL_AUTHORITIES_SR25519_LIST=5FmqMTG...
ARCHIPEL_AUTHORITIES_ED25519_LIST=5FbQNUq...
ARCHIPEL_NODE_ALIAS=archipel-node
ARCHIPEL_CHAIN_ADDITIONAL_PARAMS=
EOF

# Launch docker container
docker run -d --name "archipel-node" \
  -p 9944:9944 \
  -v /root/chain/data \
  --env-file .env \
  luguslabs/archipel-node:latest

Archipel with WireGuard VPN

# Create an .env file
cat <<EOF >.env
ARCHIPEL_NODE_ALIAS=archipel1
ARCHIPEL_KEY_SEED=mushroom ladder ...
ARCHIPEL_CHAIN_ADDITIONAL_PARAMS=
POLKADOT_NAME=test-name
POLKADOT_PREFIX=node-
SERVICE=polkadot
POLKADOT_IMAGE=parity/polkadot:latest
POLKADOT_KEY_GRAN=april shift ...
POLKADOT_KEY_BABE=region run ...
POLKADOT_KEY_IMON=screen sustain ...
POLKADOT_KEY_PARA=produce hover ...
POLKADOT_KEY_AUDI=oak tail ...
POLKADOT_LAUNCH_IN_VPN=true
ARCHIPEL_AUTHORITIES_SR25519_LIST=5FmqMTG...
ARCHIPEL_AUTHORITIES_ED25519_LIST=5FbQNUq...
# WireGuard config
WIREGUARD_PRIVATE_KEY=SJWzBT8....
WIREGUARD_ADDRESS=10.0.1.1/32
WIREGUARD_LISTEN_PORT=51820
WIREGUARD_PEERS_PUB_ADDR=9dcIYKj...,xg3wSS+...,gMjvfQGXWYJ...
WIREGUARD_PEERS_ALLOWED_IP=10.0.1.1/32,10.0.1.2/32,10.0.1.3/32
WIREGUARD_PEERS_EXTERNAL_ADDR=<public_ip>:51820,<public_ip>:51820,<public_ip>:51820
EOF

# Creating docker volumes
docker volume create archipel
docker volume create archipel_service

# Launch docker container
docker run -d --name "archipel-with-wireguard" \
  --cap-add net_admin --cap-add sys_module \
  -p 51820:51820 \
  -v archipel:/root/chain/data \
  -v archipel_service:/service \
  --env-file .env \
  luguslabs/archipel-with-wireguard:latest

Environment Variables

Variable Description Values
ARCHIPEL_NODE_ALIAS Node name for the Archipel Substrate node within the federation.
Example
Archipel-yourFederationName-NodeNameHere
String
ARCHIPEL_KEY_SEED 12 words mnemonic.
Can be generated with Subkey.
Use for Archipel Substrate node.
Will be used for consensus authority and transactions propagation in the Archipel Chain.
mnemonic
ARCHIPEL_CHAIN_ADDITIONAL_PARAMS Parameters supported by substrate node.
At least, you must valorize bootnodes. To valorize bootnodes you must start nodes first and then rebbot all your 3 nodes in the federation with bootnode list valorized thank to local node id find in logs.
Example :
--bootnodes /ip4/$NODE1_IP/tcp/30333/p2p/$NODE1_LOCAL_ID --bootnodes /ip4/$NODE2_IP/tcp/30333/p2p/$NODE2_LOCAL_ID --bootnodes /ip4/$NODE3_IP/tcp/30333/p2p/$NODE3_LOCAL_ID
String
ARCHIPEL_AUTHORITIES_SR25519_LIST Valorize the Genesis Spec file for Archipel chain with the list of Authorities Public Keys in SR25519 format separated by "," char.
<SR25519 Public Key Node 1>,<SR25519 Public Key Node 2>,<SR25519 Public Key Node 3>
Public Key, Public Key, Public Key
ARCHIPEL_AUTHORITIES_ED25519_LIST Valorize the Genesis Spec file for Archipel chain with the list of Authorities Public Keys in ED25519 format separated by "," char.
<ED25519 Public Key Node 1>,<ED25519 Public Key Node 2>,<ED25519 Public Key Node 3>
Public Key, Public Key, Public Key
SERVICE External service you want to launch. Only support polkadot at the moment. polkadot
POLKADOT_NAME Node name for the polkadot node. Will be visible in Polkadot telemetry. This Node Name will a a -passive or -active suffix according to the current mode.
Example
Archipel-yourFederationName-NodeNameHere
String
POLKADOT_IMAGE Polkadot docker image version to use. parity/polkadot:latest
POLKADOT_PREFIX This prefix is used to mount the docker volume for blockchain state on the server. String
POLKADOT_KEY_GRAN 12 words mnemonic.
Polkadot Sessions keys needed to operate as validator.
Gran keyType, Granpa ed25519.
Use for consensus.
More details for sessions keys
mnemonic
POLKADOT_KEY_BABE 12 words mnemonic.
Polkadot Sessions keys needed to operate as validator.
Babe keyType, Babe sr25519.
Use for consensus/block production.
More details for sessions keys
mnemonic
POLKADOT_KEY_IMON 12 words mnemonic.
Polkadot Sessions keys needed to operate as validator.
IMON keyType, IamOnline key.
Use for heartbeat/block production.
More details for sessions keys
mnemonic
POLKADOT_KEY_PARA 12 words mnemonic.
Polkadot Sessions keys needed to operate as validator.
PARA keyType. sr25519.
Use for parachain production.
More details for sessions keys
mnemonic
POLKADOT_KEY_AUDI 12 words mnemonic.
Polkadot Sessions keys needed to operate as validator.
AUDI keyType. sr25519.
Use for Audit.
More details for sessions keys
mnemonic
POLKADOT_LAUNCH_IN_VPN Launches the service container in Archipel network environment.
If Archipel is launched with WireGuard, the service container will be able to access the VPN network.
boolean

Archipel UI

docker run -d -p 8080:80 luguslabs/archipel-ui

Acknowledgements

The bootstrap development of Archipel is financed by WEB3 Foundation's grant program Wave4. Thanks a lot for support.

You can’t perform that action at this time.