Skip to content

Commit

Permalink
[CE-145] Fix documentation structure and links
Browse files Browse the repository at this point in the history
* Re-struct the documentation
* Add necessary links to help explore quickly
* Fix several places of documentation content

Change-Id: I7eaeecc823d6949218a9b1b7652c37705d2a5849
Signed-off-by: Baohua Yang <yangbaohua@gmail.com>
  • Loading branch information
yeasy committed Sep 23, 2017
1 parent a552dd2 commit 93429cd
Show file tree
Hide file tree
Showing 13 changed files with 159 additions and 87 deletions.
3 changes: 1 addition & 2 deletions docs/dashboard.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Dashboard

By default, the dashboard will listen on port `8080` at the Master Node.
By default, the dashboard will listen on port `8080` at the Master Node, and operators can login in with default `admin:pass` credential.

The left panel gives quick link to various functions, including `overview`, `system status`, `Hosts`, `Active Clusters`, `Inused Clusters`, `Release History` and `About`.

Expand Down Expand Up @@ -71,5 +71,4 @@ Filter out those running chains that are occupied by users.

Record all the user releasing chain history.


<a rel="license" href="http://creativecommons.org/licenses/by/4.0/"><img alt="Creative Commons License" style="border-width:0" src="https://i.creativecommons.org/l/by/4.0/88x31.png" /></a><br />This work is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>.
Binary file added docs/imgs/deploy_arch.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
48 changes: 28 additions & 20 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,44 +3,52 @@ Welcome to Hyperledger Cello

Hyperledger Cello is a blockchain provision and operation system, which helps manage blockchain networks.

![Typical Scenario](imgs/scenario.png)

Hyperledger Cello is designed with the following features:

* Manage the lifecycle of blockchains, e.g., create/start/stop/delete/keep health automatically.
* Support customized (e.g., size, consensus) blockchains request, currently we mainly support [Hyperledger fabric](https://github.com/hyperledger/fabric).
* Support native Docker host, swarm or Kubernetes as the worker nodes. More supports on the way.
* Support bare-metal, virtual machine, native [Docker](https://www.docker.com/) host, swarm or [Kubernetes](https://kubernetes.io/) as the worker nodes. More supports on the way.
* Support heterogeneous architecture, e.g., X86, POWER and Z, from bare-metal servers to virtual machine clouds.
* Extend with monitor, log, health and analytics features by employing additional components.

Using Cello, every Blockchain application developer can:
Using Cello, Blockchain application developers can:

* Build up a Blockchain as a Service (BaaS) platform quickly from the scratch.
* Provision customizable Blockchains instantly, e.g., a Hyperledger fabric network v1.0.
* Maintain a pool of running blockchain networks on top of baremetals, Virtual Clouds (e.g., virtual machines, vsphere Clouds), Container clusters (e.g., Docker, Swarm, Kubernetes).
* Maintain a pool of running blockchain networks on top of baremetals, virtual clouds (e.g., virtual machines, vsphere Clouds), container clusters (e.g., Docker, Swarm, Kubernetes).
* Check the system status, adjust the chain numbers, scale resources... through dashboards.

A typical usage scenario is illustrated as:

![Typical Scenario](imgs/scenario.png)
## [Getting Started](tutorial)

## Getting Started
For new users, it is highly recommended to read the [Tutorial for Beginners](tutorial) first.

For new users, it is highly recommended to read the [tutorial](tutorial) first.

## Operational Guideline
* [New User Tutorial](tutorial)
* [Installation Steps](installation)
* [Terminologies List](terminology)
## Operation Guideline
* [Terminology and Concept](terminology)
* [Installation](installation)
* [Adoption Scenarios](scenario)
* [Production Configuration](production_config)
* [Configuration for Production](production_config)
* [Manage Cello services](service_management)

## Development Guideline
* [How to contribute](CONTRIBUTING)
## Contribute to the Project
* [How to Contribute](CONTRIBUTING)
* [Coding Style](code_style)
* [PEP8 Style Guide](https://www.python.org/dev/peps/pep-0008/)
* [Develop react js](reactjs)

## Design Documentation
* [Architecture Design](arch)
* [Database Model](db)
* [API Design](api/restserver_v2)
* [Wikipage](https://wiki.hyperledger.org/projects/cello)
* [Develop react js](reactjs)
* [Coding Style](code_style)
* [Pep8 style guide](https://www.python.org/dev/peps/pep-0008/)

## Communication Channels

For additional helps, feel free to take the following channels:

* [Wikipage](https://wiki.hyperledger.org/projects/cello): Lots of information and documentation about the project.
* [Jira Board](https://jira.hyperledger.org/projects/CE/issues): Find development status, report bug, or help contribute code.
* [Mail List](mailto:hyperledger-cello@lists.hyperledger.org): General discussions with Cello project.
* [Rocket.Chat channels](https://chat.hyperledger.org/channel/cello): Real-time technical conversations.

<a rel="license" href="http://creativecommons.org/licenses/by/4.0/"><img alt="Creative Commons License" style="border-width:0" src="https://i.creativecommons.org/l/by/4.0/88x31.png" /></a><br />This work is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>.
32 changes: 12 additions & 20 deletions docs/installation.md
Original file line number Diff line number Diff line change
@@ -1,35 +1,27 @@
# Installation

*Here we describe the setups for development usage. If you want to deploy Cello for production, please also refer to the [Production Configuration](production_config.md).*
Cello follows a typical Master-Worker architecture. There are two types of Nodes in the cluster.

Cello follows a typical Master-Worker architecture. Hence there will be two types of Nodes.
* Master Node: Holds [Cello services](service_management.md) to manage (e.g., create/delete) the chains inside Work Nodes. Usually, Master Node will provide Web dashboard (port `8080`) and RESTful api (port `80`). It is recommended to use Linux (e.g., Ubuntu 14.04+) or MacOS;
* Worker Node: Nodes to hold blockchains. Take Docker Host or Swarm Cluster for example, the Docker daemon should be accessible (typically at port `2375`) from the Master Node.

* Master Node: Manage (e.g., create/delete) the chains inside Work Nodes, with Web dashboard listening on port `8080` and RESTful api on port `80`;
* Worker Node: Chain providers, now support Docker Host or Swarm Cluster. The Docker service should be accessible from port `2375` from the Master Node.
![Deployment topology](imgs/deploy_arch.png)

![Deployment topology](imgs/deployment_topo.png)
## Master Node

For each Node, it is suggested as a Linux-based (e.g., Ubuntu 14.04+) server/vm:
See [Installation on Master Node](installation_master.md).

## Worker Node

Currently we support Docker Host or Swarm Cluster as Worker Node. More types will be added soon.

`Docker Host`: See [Installation on Worker Docker Node](installation_worker_docker.md).
`Docker Swarm `: TODO.
`Kubernetes`: TODO.
`vSphere`: TODO.

## Master Node

See [Installation on Master Node](installation_master.md).

### Make Command

A Makefile is provided to help setup the master node, please refer [make_support](/make_support.md) page.
* `Docker Host`: See [Installation on Worker Docker Node](installation_worker_docker.md).
* `Docker Swarm `: See [Create a Docker Swarm](https://docs.docker.com/engine/swarm/swarm-tutorial/create-swarm/).
* `Kubernetes`: See [Kubernetes Setup](https://kubernetes.io/docs/setup/).
* `vSphere`: TODO.

## Special Configuration for Production

*Licensed under Creative Commons Attribution 4.0 International License
https://creativecommons.org/licenses/by/4.0/*
Here we describe the setups for development usage. If you want to deploy Cello for production, please also refer to the [Production Configuration](production_config.md).

<a rel="license" href="http://creativecommons.org/licenses/by/4.0/"><img alt="Creative Commons License" style="border-width:0" src="https://i.creativecommons.org/l/by/4.0/88x31.png" /></a><br />This work is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>.
2 changes: 2 additions & 0 deletions docs/installation_master.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,3 +81,5 @@ By default, it also loads the `config.py` file as the configurations.
The mongo container will use local `/opt/cello/mongo` directory for persistent storage.

Please keep it safe by backups or using more high-available solutions.

<a rel="license" href="http://creativecommons.org/licenses/by/4.0/"><img alt="Creative Commons License" style="border-width:0" src="https://i.creativecommons.org/l/by/4.0/88x31.png" /></a><br />This work is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>.
4 changes: 3 additions & 1 deletion docs/installation_worker_docker.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,4 +74,6 @@ Make sure ip forward is enabled, you can simply run the follow command.
```sh
$ sysctl -w net.ipv4.ip_forward=1
```
And check the os iptables config, to make sure host ports are open (e.g., 2375, 7050~10000)
And check the os iptables config, to make sure host ports are open (e.g., 2375, 7050~10000)

<a rel="license" href="http://creativecommons.org/licenses/by/4.0/"><img alt="Creative Commons License" style="border-width:0" src="https://i.creativecommons.org/l/by/4.0/88x31.png" /></a><br />This work is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>.
14 changes: 10 additions & 4 deletions docs/make_support.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
## Operations Supported
### make
## Supported Make Commands

The following opperations are supported by make command. Prepend the following commands with make.
The following opperations are supported by [Make](https://en.wikipedia.org/wiki/Makefile) command. Prepend the following commands with make.

These commands should be run in `cello` directory, for example: `/cello$ make all`.

### all
Default to run check. Runs check.
Default to run the testing cases.

### build-js
Builds js files for react.

### check
CI checking. Runs the following commands. This runs the following commands for you.

```sh
$ tox
$ make start && sleep 10 && make stop
Expand All @@ -21,13 +21,15 @@ $ make start && sleep 10 && make stop
### clean
Cleans up the environment and removes temp files like .cache, .tox, .pyc.
It runs the following commands for you.

```sh
$ rm -rf .tox .cache *.egg-info
$ find . -name "*.pyc" -o -name "__pycache__" -exec rm -rf "{}" \;
```

### doc
Starts a doc server locally. It runs the following commands for you.

```sh
$ pip install mkdocs
$ mkdocs serve
Expand All @@ -38,6 +40,7 @@ Show help.

### image-clean
Cleans up all cello related docker images. It runs the following commands for you.

```sh
$ docker images | grep "cello-" | awk '{print $1}' | xargs docker rmi -f
$ docker rmi $(docker images -f dangling=true -q)
Expand All @@ -51,12 +54,14 @@ Shows logs of all services.

### setup-master
Sets up the master node. Run on Master node. It runs the following command for you.

```sh
$ cd scripts/master_node && bash setup.sh
```

### setup-worker
Sets up the worker node. Run on Worker node. It runs the following commands for you.

```sh
$ cd scripts/worker_node && bash setup.sh
```
Expand All @@ -66,6 +71,7 @@ Redeploys specified service(s). To redeploy `Dashboard` service, use: `make rede

### start
Starts all services. Runs following command for you.

```sh
$ docker-compose up -d --no-recreate
```
Expand Down
15 changes: 11 additions & 4 deletions docs/monitoring.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,24 @@
#Monitoring services
The monitoring services build in real time an archive of observations obtained from the analisys of the communication flows of the CELL application services.

The monitoring services build in real time an archive of observations obtained from the analisys of the communication flows of the Cello application services.

The observations could be checked in real time (directly in memory) against attacks, anomalies or errors.

The observations, once processed in real time, are stored into a persistent archive for statistic analisys and reports generation.

### Config the port to be monitored
Admin can configure the tcp ports to be monitored by the monitoring services, by adding the following two lines:
* # the monitoring services will have to monitor the following ports
* PORTS_TO_BE_MONITORED = list(PEER_SERVICE_PORTS.items()) + list(CA_SERVICE_PORTS.items())

```bash
# the monitoring services will have to monitor the following ports
PORTS_TO_BE_MONITORED = list(PEER_SERVICE_PORTS.items()) + list(CA_SERVICE_PORTS.items())
to the configuration file: /src/common/utils.py
```

### Config the monitoring level
Admin can add to the file /src/monitoring/config.py the following flags:
* MONITOR_DB=<file name or network path where to store the persistent archive>

* MONITOR_DB=[file name or network path where to store the persistent archive]
* MONITOR_LEVEL=FULL | SIMPLE | NONE

<a rel="license" href="http://creativecommons.org/licenses/by/4.0/"><img alt="Creative Commons License" style="border-width:0" src="https://i.creativecommons.org/l/by/4.0/88x31.png" /></a><br />This work is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>.
6 changes: 3 additions & 3 deletions docs/production_config.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# Production Configurations
Reference system configuration in production environment.

In order to optimize the system performance in production environment, these system configurations can be set.

## `/etc/sysctl.conf`

Expand Down Expand Up @@ -31,9 +32,8 @@ Then, need to run `sysctl -p` for enabling.
* soft stack 32768
* hard stack 32768
```
Log

## Other Consideration
## Other Configurations

* Use the code from `release` branch.
* Configuration: Set all parameters to production, including image, compose, and application.
Expand Down
20 changes: 10 additions & 10 deletions docs/scenario.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Scenarios

## Admin
## Admin Scenario
After start up, Cello provides a dashboar for administrators, which listens on localhost:8080.

The default login user name and password are `admin:pass`, you can modify this by changing the variables `USERNAME` and `PASSWORD` in the `nginx` section of the [docker-compose file](../docker-compose.yml).
Expand All @@ -16,19 +16,19 @@ Admin can also delete a host from the resource pool if it has no running chains.
### Config a host
Admin can manually update the host configuration, including:

* name: Human readable name alias.
* capacity: Maximum chain number on that host.
* schedulable: Whether to distribute chains on that host to users.
* autofill: Whether to keep host with running chains to its capacity.
* log_type: local or syslog.
* `name`: Human readable name alias.
* `capacity`: Maximum chain number on that host.
* `schedulable`: Whether to distribute chains on that host to users.
* `autofill`: Whether to keep host with running chains to its capacity.
* `log_type`: local or syslog.

### Operate a host

Admin can run several operations on a host, including:

* fill: Fill the host with chains to its capacity.
* clean: Clean up the free chains on that host.
* reset: Re-setup a host, e.g., cleaning useless docker containers.
* `fill`: Fill the host with chains to its capacity.
* `clean`: Clean up the free chains on that host.
* `reset`: Re-setup a host, e.g., cleaning useless docker containers.

### Add/Delete chains
Admin can also manually add some specific chain to a host, or delete one.
Expand All @@ -39,7 +39,7 @@ When the autofill box is checked on a host, then watchdog will automatically kee

e.g., if the capacity of one host is set to 10, then the host will be filled with 10 chains quickly. When 2 chains are broken, they will be replaced by healthy ones soon.

## Chain users
## Users Scenario

### apply a cluster

Expand Down
29 changes: 29 additions & 0 deletions docs/service_management.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Service Management

Main Cello services are running in the Master Node.

## Services

After starting cello services using `make start`, there will generate several service containers as the following:

```bash
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
fdf4b8465d14 yeasy/nginx "/bin/bash /tmp/do..." 12 seconds ago Up 11 seconds 0.0.0.0:80->80/tcp, 0.0.0.0:8080->8080/tcp, 443/tcp nginx
80c3962867ff mongo:3.2 "docker-entrypoint..." 12 seconds ago Up 11 seconds 127.0.0.1:27017-27018->27017-27018/tcp mongo
91df95a11229 cello-dashboard "python dashboard.py" 12 seconds ago Up 11 seconds 8080/tcp dashboard
051efd511066 cello-watchdog "python watchdog.py" 12 seconds ago Up 11 seconds watchdog
a66bb112a21f cello-restserver "python restserver.py" 12 seconds ago Up 12 seconds 80/tcp restserver
```

* `nginx`: [Nginx](https://nginx.org) is used as a reverse proxy to improve web performance.
* `mongo`: [MongoDB](https://www.mongodb.com) is used as the backend database.
* `dashboard`: Provides the admin dashboard using [Flask](http://flask.pocoo.org/).
* `watchdog`: Monitors the status of the system (e.g., chains' health).
* `restserver`: Core engine to do the provision, orchestration and management tasks.

## Make Command

A [Makefile](https://en.wikipedia.org/wiki/Makefile) is provided to help setup and manage the master node, please refer to the [make_support](make_support.md) page.

<a rel="license" href="http://creativecommons.org/licenses/by/4.0/"><img alt="Creative Commons License" style="border-width:0" src="https://i.creativecommons.org/l/by/4.0/88x31.png" /></a><br />This work is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>.

0 comments on commit 93429cd

Please sign in to comment.