Tutorial: Setting up the server environment
This tutorial covers how to set up the cloud server environment for your wireless sensor network. We use Amazon Web Services (AWS) to host a time-series database (Influxdb) that receives and stores data transmitted from our nodes in the field. At the end of this tutorial, we will have set up a server environment capable of facilitating two-way communication between our nodes and a database.
The first step in setting up the server environment is to create an Amazon Elastic Compute Cloud (EC2) instance. An instance is a virtual computing environment that allows you to, among other things, host a web-accessible database. More about what EC2 is can be found here, in Amazon's documentation.
Launching an EC2 instance requires that you first set up an AWS account, which can be done here. Running an EC2 instance costs money, and as part of the sign up process, you will have to enter credit card information. At the time of writing, AWS offers one year of access to free-tier services for new accounts, so you have some time to experiment and orient yourself with the technology before you start racking up charges.
AWS offers step-by-step instructions on how to set up an EC2 instance here. Make sure that you complete the prerequisite steps covered in Setting Up with Amazon EC2 as well.
Notes:
- Creating your AWS account: If there is any trouble verifying your card details (e.g. you are travelling and your bank locks down you card), your account will not activate.
- IAM Users: If you are a lab group, you will want to pay attention to how IAM users and permissions work. It is standard practice to create one AWS account for a lab and add a new user for each lab member who uses AWS. Good security practices dictate that admin permission should only be given to those users who really need it while giving other users more limited access.
- AMI Choice: To run Influxdb, you will need to choose a Linux-based Amazon Machine Image (AMI) when launching your template. Depending on which AMI you choose, your Influxdb installation experience will be slightly different. Amazon Linux will work with the RedHat & Centos download instructions, while the Ubuntu Linux AMI will work with the Ubuntu & Debian download instructions. Aside from small differences like these, it doesn't really matter which AMI you choose.
- Instance Type: Our up-and-running sensor network runs on a t2.small instance, but we recommend sticking to the free-tier t2.micro instance while you are still figuring things out.
- Naming your instance: naming instances can be done in the Instances tab by clicking the edit arrow next to the blank space in the "Name" column. We recommend naming your instance something short but descriptive. This will make it slightly easier to keep track of the instance when doing things like attaching an EBS volume.
To facilitate running an Influxdb database, we will need to partition and attach some storage space using the AWS service Elastic Block Store (EBS). Your EC2 instance is already EBS backed, meaning its root volume is an EBS volume. This volume only provides 8Gb storage, however, which is not enough for our purposes. To enable a separate storage volume, we need to create an EBS volume and attach it to our EC2 instance. Follow Amazon's guide to create an EBS volume.
Notes:
- Size of volume: If you don't know how much storage space you need, 8 GiB is a fine starting point.
- Availability zone: It is really important that your EBS volume is in the same availability zone as your EC2 instance, otherwise you will not be able to attach them.
Next, follow the steps to attach an EBS volume to your EC2 instance.
Notes:
- In the attach volume window, your desired EC2 instance should appear from a drop down menu under 'instances'. Instances are listed by their instance ID's, but if you named your instance, that will be shown in parenthesis.
- It is Linux convention to name your first attached volume "/dev/sdf", your second "/dev/sdg", and so on until "/dev/sdp"
Now that you have launched your instance, it is time to connect to it. Amazon documentation on how to do this can be found here. Just like before, make sure you complete the prerequisite steps first!
Notes:
- Having chosen a Linux-based AMI, we will be communicating with our instance via SSH. This will require using a command line interface (CLI).
- If you are using a Mac or Linux computer, you can SSH (secure shell) into your instance through terminal. If you are using a Windows computer, you will need to download a terminal emulator (the most widely used is PuTTY) and use that to SSH into your instance.
- If you are unfamiliar with the Linux command line, this free ebook - The Linux Command Line by William Shotts - serves as a good overview and reference.
- A really common error: Make sure your private key permissions are 0400, not 0777. If the permissions are wrong, you will get an Error: Unprotected Private Key File message. You can fix this error with the following line of code ("/path/my-key-pair.pem" is replaced by the actual file path and name of your key pair .pem file).
chmod 400 /path/my-key-pair.pem
- On your first time connecting, you may be asked to update yum by typing "sudo yum update". Do this, then type 'y' when prompted to accept changes.
In step 2 we attached the storage volume, but this is (sadly) not enough. We have created the basic AWS resources we need, but these resources are not yet operational. Mounting is the process of connecting a storage device to a particular part of a computer's file system. This is a step that happens automatically when you plug in a USB drive to your computer, but must be done manually for an EC2 instance. Mounting will allow us to specify the specific part of the directory tree that our EBS volume will connect to a.k.a. the file path you will have to type to access files stored in the volume.
Amazon documentation on how to mount your volume can be found here. Alternately, you can follow the steps below to mount your EBS volume:
Connect to your EC2 instance and use the command lsblk to check whether your volume is exposed (attached) to the instance. This command will return a list of all block devices connected to your instance. You should see your instance listed as "/dev/xvdf" or similar, depending on what you named it. As Amazon warns us, "Newer Linux kernels may rename your devices to /dev/xvdf through /dev/xvdp internally, even when the device name entered here (and shown in the details) is /dev/sdf through /dev/sdp."
Once confirming your EBS volume is attached, execute the following commands:
#### Enable elastic block store at mount point /data
#### This assumes you have an EMPTY elastic block store attached at /sdf
#### Do not do this if the elastic block store has data on it!
sudo mkfs -t ext4 /dev/xvdf
sudo mkdir /data
sudo mount /dev/xvdf /data
sudo cp /etc/fstab /etc/fstab.orig
Now we need to edit the /etc/fstab file. The easiest way to do this is to use vim, a built-in command line text editor. If you don't know anything about vim, you might want to look at The Linux Command Line Chapter 12, which offers a "Gentle Introduction" to vi, the predecessor of vim (for our purposes, vi and vim are interchangeable). For your reference, below we have listed all of the vim commands you will need here:
- use A to begin editing the next line of the document
- tabs separate the clauses we are about to input
- when done typing, press esc to exit editing mode
- exit vim using :wq (this also saves the document).
Open the /etc/fstab file in vim.
sudo vim /etc/fstab
Add the following line to enable mount on boot:
/dev/xvdf /data ext4 defaults,nofail,nobootwait 0 2
Save and exit out of vim. Now we are ready to mount the volume based on the configuration we just input to /etc/fstab.
sudo mount -a
Notes:
- Depending on the AMI (operating system template) you set up your EC2 instance as way back in step 1, your /etc/fstab file might already have some configuration parameters in it. Just add our /dev/xvdf line in as the last line.
We now have our server environment set up on the AWS end, it is time to install Influxdb! Got to https://portal.influxdata.com/downloads/ and follow the instructions to install on a Linux system. Remember, your AMI choice determines which set of instructions you should follow. Amazon Linux will work with the RedHat & Centos download instructions, while the Ubuntu Linux AMI will work with the Ubuntu & Debian download instructions.
Once downloaded, you need to configure the influxdb directory with the following commands:
#### Set up influxdb write directory
sudo mkdir /data/influxdb
sudo chown influxdb:influxdb /data/influxdb
sudo mkdir /data/influxdb/data
sudo mkdir /data/influxdb/wal
sudo mkdir /data/influxdb/meta
sudo chown influxdb:influxdb /data/influxdb/data
sudo chown influxdb:influxdb /data/influxdb/wal
sudo chown influxdb:influxdb /data/influxdb/meta
We need to use vim once again to make a few edits in the influxdb config file:
sudo vim /etc/influxdb/influxdb.conf
Make the following changes:
"""
[meta]
dir = "/data/influxdb/meta"
...
[data]
dir = "/data/influxdb/data"
wal-dir = "/data/influxdb/wal"
"""
Start the influx service with:
sudo service influxdb start
... and add the influxdb configuration path to your environment variables:
INFLUXDB_CONFIG_PATH="/etc/influxdb/influxdb.conf"
Now reboot the server (exit and log in to AWS again) and run influxdb for the first time:
influxd
This brings you into the Influxdb command line interface. Having already SSH-ed into the AWS command line, you are now in a CLI (influx) within a CLI (AWS) within a CLI (your terminal)! Typing "exit" will pop you up a level. To log out of everything and close your terminal, type "exit" three times.
Congratulations! You now have successfully set up the server environment.