Skip to content
Genesis is an end-to-end development sandbox and testing platform for Web3 and blockchain-based distributed systems.
Branch: dev
Clone or download
Latest commit 91810ef Jul 20, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github/ISSUE_TEMPLATE
bin 1337 May 8, 2019
config add defauly for ip prefix and make the naming more clear May 28, 2019
db fix destroy and remove the ips field from servers Jul 12, 2019
deploy added ability to build from a repo Jul 16, 2019
docker fix the destroy on failure option for normal build Jul 11, 2019
manager fix destroy and remove the ips field from servers Jul 12, 2019
net bug fixes Jun 12, 2019
protocols fix geth expansion mode Jul 20, 2019
resources rchain cosmetic fix Jul 16, 2019
rest parity fix Jun 24, 2019
sidecars parity fix Jun 24, 2019
ssh remove regex Jun 25, 2019
state implemented geth append Jul 19, 2019
status fix destroy and remove the ips field from servers Jul 12, 2019
testnet clean up and add new helper function Jul 16, 2019
util implemented geth append Jul 19, 2019
.gitignore add vscode stuff to git ignore Jun 24, 2019
.travis.yml regress the travis build so it no longer takes 12 minutes Jun 20, 2019
CODE_OF_CONDUCT.md Create CODE_OF_CONDUCT.md May 9, 2019
CONTRIBUTING.md Create CONTRIBUTING.md May 9, 2019
COPYING Whiteblock -> whiteblock May 11, 2019
Dockerfile use go mod in dockerfile May 30, 2019
INSTALL update INSTALL and remove appo as the default user in favor of using … May 25, 2019
LIBRARIES cleanup and fixes Apr 10, 2019
Makefile fix travis Jun 20, 2019
README.md remove native mac instructions as its support has been dropped Jun 1, 2019
cloudbuild.tag.yaml cloudbuild for tags Jun 6, 2019
cloudbuild.yaml begin switch to the new style May 28, 2019
go.mod cleanup and nibbler redownload Jun 6, 2019
go.sum go.sum fixed conflict Jun 4, 2019
main.go Add banner on startup May 30, 2019
rest.md updated readme and created rest.md May 9, 2019

README.md

Genesis

Build Maintainability Go report card codecov

Version API Reference Gitter

Overview

The Whiteblock platform allows users to provision multiple fully-functioning nodes over which they have complete control within a private test network

REST API

Documentation for the REST API can be found here.

Installation

Setup docker

set environment variables to allow SSH commands:

  1. ssh localhost docker
    • if error bash: docker: command not found, move to next step
  2. echo "PATH=$PATH" >> ~/.ssh/environment
  3. as root: echo "PermitUserEnvironment yes" >> /etc/ssh/sshd_config

Build Genesis

  • go get github.com/whiteblock/genesis
  • cd $GOPATH/src/github.com/whiteblock/genesis
  • go build

Configuration

Configuration options are located in config.json in the same directory as the binary

Option Description
sshUser The default username for ssh
sshKey The location of the ssh private key
listen The socket to listen on
verbose Enable or disable verbose mode
serverBits The bits given to each server's number
clusterBits The bits given to each clusters's number
nodeBits The bits given to each nodes's number
threadLimit The maximum number of threads that can be used for building
ipPrefix Used for the IP Scheme
dockerOutputFile The location instead the docker containers where the clients stdout and stderr will be captured
influx The influxdb endpoint
influxUser The influx auth username
influxPassword The influx auth password
serviceNetwork CIDR of the network for the services
serviceNetworkName The name for the service network
nodePrefix The prefix for each node name
nodeNetworkPrefix The prefix for each cluster network
servicePrefix The prefix for each service
nodesPublicKey Location of the public key for the nodes
nodesPrivateKey Location of the private key for the nodes
handleNodeSshKeys Should genesis handle the nodes ssh keys?
maxNodes Set a maximum number of nodes that a client can build
maxNode-memory Set the max memory per node that a client can use
maxNodeCpu Set the max cpus per node that a client can use

Config Environment Overrides

These will override what is set in the config.json file, and allow configuration via only ENV variables

  • SSH_USER
  • SSH_KEY
  • LISTEN
  • VERBOSE (only need to set it)
  • SERVER_BITS
  • CLUSTER_BITS
  • NODE_BITS
  • THREAD_LIMIT
  • IP_PREFIX
  • DOCKER_OUTPUT_FILE
  • INFLUX
  • INFLUX_USER
  • INFLUX_PASSWORD
  • SERVICE_NETWORK
  • SERVICE_NETWORK_NAME
  • NODE_PREFIX
  • NODE_NETWORK_PREFIX
  • SERVICE_PREFIX
  • NODES_PUBLIC_KEY
  • NODES_PRIVATE_KEY
  • HANDLE_NODE_SSH_KEYS (only need to set it)
  • MAX_NODES
  • MAX_NODE_MEMORY
  • MAX_NODE_CPU

Additional Information

  • Config order of priority ENV -> config file -> defaults

IP Scheme

We are using ipv4 so each address will have 32 bits.

The following assumptions will be made

  • Each server will have a relatively unique serverId
  • This uniqueness need only apply to servers which will contain nodes which communicate with each other
  • There are going to be 3 IP addresses reserved from each subnet
  • Nodes in the same docker network are able to route between each other by default

For simplicity, the following variables will be used

  • A = ipPrefix
  • B = serverBits
  • C = clusterBits
  • D = nodeBits

Note the following rules

  • A,B,C, and D must be greater than 0
  • ceil(log2(A)) + B + C + D <= 32
  • D must be atleast 2
  • (2^B) = The maximum number of servers
  • (2^C) = The number of cluster in a given server
  • (2^D - 3) = How many nodes are groups together in each cluster
  • (2^D - 3) * (2^C) = The max number of nodes on a server
  • (2^D - 3) * (2^C) * (2^B) = The maximum number of nodes that could be on the platform

What is a cluster?

Each cluster corresponds to a subnet, docker network, and vlan.

Containers in the same cluster will have minimal latency applied to them. In the majority of cases, it is best to just have one node per cluster, allowing for latency control between all of the nodes.

How is it all calculated?

Given a node number X and a serverId of Y, Let Z be the cluster number, I be the generated IP in big-endian, and the earlier mentioned variables applied

Z = floor(X / (2^D - 3))) I = (A * 2^(B+C+D) ) + ( Y * 2^(B+C) ) + (Z * 2^C) + (X % (2^D - 3) + 2)

if Z == (2^C - 1) then I = I - 2

Explanation

First get the cluster the node is in

Then construct the IP one segment at a time through addition

Due to the restrictions, each piece will fit neatly into place without overlap

Finally, check if it is not the last cluster on the server,

add 1 to the ip address if it is not the last cluster.

Example

Given a node number(X) of 2 and a serverId(Y) of 3

Given the IP Scheme of __A__ = 10, __B__ = 8, __C__ = 14, __D__ = 2

__Z__ = floor(2/(2^2 - 3))
__Z__ = 2

It is going to be in cluster 2

Now, for the construction of the IP

Visually, it can be represented as

IP = AAAAAAAA BBBBBBBB CCCCCCCC CCCCCCDD

The values are simply placed inside the bit space of the IP address as represented, with the exception of the D bits, which needs to be calculated

calculate this number as (2 % (2^2 - 3) + 2) or 2

Then since (__Z__ != (2^__C__ - 1)) 2 != 16383, the value remains 2

Finally, construct IP

Part A = 00001010
Part B = 00000011
Part C = 00000000000010
Part D = 10
IP = 00001010 00000011 00000000000010 10
   = 00001010 00000011 00000000 00001010
   = 10       3        0        10

The gateway is calculated in a similar way, except take Part D to always equal 1

Gateway IP = 00001010 00000011 00000000000010 01
           = 00001010 00000011 00000000 00001001
           = 10       3        0        9

Finally the subnet is 32 - D

Resulting in

IP = 10.3.0.10
Gateway IP = 10.3.0.9
Subnet = 10.3.0.8/30

Blockchain Specific Parameters

Geth (Go-Ethereum)

Note: Any configuration option can be left out, and this entire section can even be null, the example contains all of the defaults

Options

  • networkId: The network id
  • difficulty: The initial difficulty set in the genesis.conf file
  • initBalance: The initial balance for the accounts
  • maxPeers: The maximum number of peers for each node
  • gasLimit: The initial gas limit
  • homesteadBlock: Set in genesis.conf
  • eip155Block: Set in genesis.conf
  • eip158Block: Set in genesis.conf

Example (using defaults)

{
    "chainId":15468,
    "networkId":15468,
    "difficulty":100000,
    "initBalance":100000000000000000000,
    "maxPeers":1000,
    "gasLimit":4000000,
    "homesteadBlock":0,
    "eip155Block":0,
    "eip158Block":0
}

Syscoin (RegTest)

Options

  • rpcUser: The username credential

  • rpcPass: The password credential

  • masterNodeConns: The number of connections to set up for the master nodes

  • nodeConns: The number of connections to set up for the normal nodes

  • percentMasternodes: The percentage of the network consisting of master nodes

  • options: Options to set enabled for all nodes

  • senderOptions: Options to set enabled for senders

  • receiverOptions: Options to set enabled for receivers

  • mnOptions: Options to set enabled for master nodes

  • extras: Extra options to add to the config file for all nodes

  • senderExtras: Extra options to add to the config file for senders

  • receiverExtras: Extra options to add to the config file for receivers

  • mnExtras: Extra options to add to the config file for master nodes

Example (using defaults)

{
    "rpcUser":"username",
    "rpcPass":"password",
    "masterNodeConns":25,
    "nodeConns":8,
    "percentMasternodes":90,
    "options":[
        "server",
        "regtest",
        "listen",
        "rest"
    ],
    "senderOptions":[
        "tpstest",
        "addressindex"
    ],
    "mnOptions":[],
    "receiverOptions":[
        "tpstest"
    ],
    "extras":[],
    "senderExtras":[],
    "receiverExtras":[],
    "mnExtras":[]
}
You can’t perform that action at this time.