Devcon4 Workshop
- Intro
- Prerequisites
- Initial Setup
- Exercises
- Administrative Tooling
- Docker (containerization)
- Ansible (configuration management)
- Packer (image builder)
- Terraform (cloud deployment)
- DigitalOcean (IaaS platform)
- Infrastructure Components
- Reverse Proxy
- Key-value Store and Service Discovery
- Ethereum Node
- Ethereum Applications
- IPFS
Welcome to the Architecting with Ethereum workshop, presented at Devcon IV in Prague on October 30, 2018.
CryptoKube is an open source software stack for hosting peer-to-peer cryptographic applications. It currently consists of many separate modules (primarily Ansible, Terraform, and Docker). The ultimate goal is to provide a turnkey Kubernetes implementation (still under development).
Visit CryptoKube.io for the latest details.
In this workshop we build a basic Ethereum application stack using components from the CryptoKube public repos.
We begin by introducing the adminstrative tooling and the major components of the stack. Then we conduct a series of exercises to demonstrate the concepts in realistic use cases. Each exercise builds on previous exercises, and each aims to introduce one major administrative concept and one major P2P crypto node concept.
It is possible to run the exercises locally, although it is recommended to use the provided management host image for greatest environment consistency.
"It's up to us. We cannot take for granted that the future will be better, and that means we need to work to create it today."
- Peter Thiel, "Zero to One"
- Experience: Familiarity with Ethereum and remote Linux administration
- Software: SSH client & web browser
- Assets: Digital Ocean account w/SSH & API keys
TODO
The management host is a 2GB droplet running Ubuntu Server 18.04 LTS. To provision the management host, run TODO
It comes preconfigured with the following components:
- Nginx+LetsEncrypt
- GoCD
- Ansible
- Terraform
- Packer
- Virtualbox
- 00: Docker Light Client
- 01: Light Client (cloud)
- 02: Full Node
- 03: HAProxy
- 04: Ethereum Application Server
We'll primarily be using Terraform, Ansible, terraform-inventory, Docker, and Git.
HashiCorp Terraform enables you to safely and predictably create, change, and improve infrastructure. It is an open source tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.
Overview:
- Easily describe your infrastructure as code
- Version control your resources, allowing rollback to previous state
- Uses declarative syntax (HCL), fully JSON compatible but extended for easier human consumption
Major Concepts:
- Configuration: text files with
.tfextension, describes infrastructure and sets variables - State:
terraform.tfstate, maps real world resources to your configuration, and keeps track of metadata - Providers: responsible for understanding API interactions and exposing resources, example: DigitalOcean, Docker, Consul
- Modules: self-contained packages configurations that are managed as a group, used to create reusable components
Basic Commands
To view a list of available commands, run terraform with no arguments. For details about a command, run terraform <command> -h
terraform init- Initialize a new or existing Terraform configuration (install plugins, perform minimal validation)terraform plan- Generate and show an execution planterraform show- Inspect Terraform state or planterraform apply- Builds or changes infrastructureterraform destroy- Destroy Terraform-managed infrastructure
Links
Ansible is an open source software that automates software provisioning, configuration management, and application deployment. -wikipedia
- Ansible User Guide (latest)
- Ansible Galaxy - repository of roles
External Training
- RedHat DO007 Ansible Essentials: Simplicity in Automation Technical Overview
- Linux Academy: Ansible Quick Start
- How To Test Ansible Roles with Molecule on Ubuntu 18.04
- How to Manage Multistage Environments with Ansible
Docker is a computer program that performs operating-system-level virtualization also known as containerization. -wikipedia
Basic Commands
To view a list of available commands, run docker with no arguemnts. For details about a command, run docker <command> -h
- Containers:
docker ps- List containersdocker logs- Fetch the logs of a containerdocker kill- Kill one or more running containersdocker rm- Remove one or more containers
- Images:
docker images- List imagesdocker rmi- Remove one or more images
Links
HashiCorp Packer is a tool for building images for cloud platforms, virtual machines, containers and more from a single source configuration.
DigitalOcean is an Infrastructure as a Service (IaaS) platform that aims to be "The simplest cloud platform for developers & teams."
- DigitalOcean Product Documenation
- The Navigator's Guide to DigitalOcean
- Use Terraform with DigitalOcean
Load balancing Load balancing improves the distribution of workloads across multiple computing resources, such as computers, a computer cluster, network links, central processing units, or disk drives. -wikipedia
High availability High availability is a characteristic of a system, which aims to ensure an agreed level of operational performance, usually uptime, for a higher than normal period. -wikipedia
- DigitalOcean tutorials
- NGINX Admin Guide
- Nginx glossary: What is Load Balancing? How Load Balancers Work
- Consul Docs
- Consul service discovery (using brianshumate/ansible-consul role)
- IPFS Docs
- Docker Hub: ipfs/go-ipfs (github)
- IPFS blog: Run IPFS in a Docker Container
- Ethereum Wiki
- Ethereum Yellow Paper
- Book: Mastering Ethereum by Andreas M. Antonopoulos, Gavin Wood
| Service | Protocol | Port | Interface |
|---|---|---|---|
| HTTP RPC | TCP | 8545 | private |
| HTTP WS | TCP | 8546 | private |
| P2P | TCP | 30303 | public |
| Node discovery | UDP | 30301 | public |
Here are the most commonly-used networks:
| Net ID | Chain ID | Description | Consensus | Client(s) |
|---|---|---|---|---|
| 1 | 1 | Ethereum mainnet | PoW | All |
| 3 | 1 | Ropsten testnet | PoW | All |
| 4 | 1 | Rinkeby tesnet | PoA | Geth |
| 42 | 1 | Kovan testnet | PoA | Parity |
| 1 | 61 | Ethereum Classic mainnet | PoW | All |
| 2 | 1 | Ethereum Classic testnet | PoW | All |
Links
- EIP-155: List of Chain ID's
- Comparison of the Different Testnets (links to testnet faucets)
- Explaining the Genesis Block in Ethereum (important to understand for private networks, which are not addressed further in this workshop)
| Name | Headers | Transactions | State | Notes |
|---|---|---|---|---|
| Light client | All | On-demand | On-demand | suitable for mobile & embedded applications |
| Full node | All | All | Pruned | suitable for most server applications |
| Archive node | All | All | All | necessary only for deep block exploration |
| Bootstrap node | All | All | Pruned | required for private blockchains |
- Docker Hub: ethereum/client-go
- Geth wiki
- Command Line Options
- Connecting to the network (geth does not work well with a config file, so use command-line options for everything)
- Running in Docker
- JSON-RPC reference
- Parity Config Generator
- Docker Hub: parity/parity
- Parity Wiki:
- Parity Skeleton DApp
- Ethereum Network Stats
- Etherchain light
- smart-contract-watch
- BlockScout - excellent Terraform setup for AWS