Nimbus Cumulus on a Raspberry Pi 3 with Raspbian (Debian Jessie 8.0)
The Raspberry Pi 3 single board computer from the Raspberry Pi Foundation has a 1.2 GHz quad-core ARMv8 processor, 1 GByte main memory and a 10/100 Mbps Ethernet interface.
This installation tutorial explains the installation of a Raspberry Pi 3 device from scratch and the configuration of s3cmd on your computer.
This implies that you have already an installation of s3cmd on the system you want to use for the interaction with the Cumulus storage service.
$ wget https://downloads.raspberrypi.org/raspbian_lite/images/raspbian_lite-2017-03-03/2017-03-02-raspbian-jessie-lite.zip
$ unzip 2017-03-02-raspbian-jessie-lite.zip
Check which one is the correct device! If you use an internal card reader, it is often /dev/mmcblk0.
# dd bs=1M if=2017-03-02-raspbian-jessie-lite.img of=/dev/sdb && sync
Default login of this image is pi/raspberry. To become user root, execute sudo su.
Older Raspian versions started the SSH server by default. Because of security reasons (as explained here), all Raspian versions since version 2016-11-25 (see the release notes) have the SSH server disabled by default. To get the SSH server automativally activated during boot time, create a file ssh with any content (or just an empty file) inside the boot partition (it is the first partition) of the micro SD card.
# mount /dev/sdb1 /media/
# touch /media/ssh
# umount /media/
Insert the micro SD card into the Raspberry Pi computer, connect it with the ethernet cable and the micro USB cable for power suppy and switch on the power supply. The operating system will try to fetch network configuration by using DHCP on the Ethernet interface per default. If you activated the SSH server, you can now log in via SSH.
$ sudo raspi-config
This is not required for running Nimbus Cumulus, but the Raspberry Pi 3 has just 1 GB of main memory which is not much at all and there is no need to waste 48 MB.
A part of the main memory (just 1 GB) is assigned to the GPU. A pure server does not need a GPU at all. The share can me specified via the raspi-config tool. The minimum value is 16 MB, which is more useful compared with the dafault value (64 MB).
After the new value is specified and after a reboot, the new value should be visible:
# vcgencmd get_mem gpu
gpu=16M
This is not required for running Nimbus Cumulus, but it is always useful to configure the operating system properly
$ sudo dpkg-reconfigure tzdata
$ cat /etc/timezone
Europe/Berlin
It is also possible to specify the time zone via the raspi-config tool.
This is not required for running Nimbus Cumulus, but it is always useful to have the correct time on a computer
$ sudo apt-get update && sudo apt-get install -y ntp ntpdate
Now the time sould be synchronized with several NTP servers.
$ ntpq -p
remote refid st t when poll reach delay offset jitter
==============================================================================
*time3.hs-augsbu 129.69.1.153 2 u - 128 377 31.891 7.194 1.053
-hotel.zq1.de 235.106.237.243 3 u 113 128 377 38.442 0.272 0.896
+bandersnatch.ro 213.239.154.12 3 u 75 128 377 28.497 6.777 3.761
-ntp.janetzki.eu 46.4.77.168 3 u 90 128 377 35.747 -0.316 27.235
+fritz.box 194.25.134.196 3 u 62 128 377 0.368 4.667 0.674
Check the time and date:
# date -R
Sat, 01 Apr 2017 14:57:33 +0200
Because we don't need the entire Nimbus IaaS functionality, we will only install, configure and start the Cumulus storage service
1) Install some packages
Some of them are required for the installation of Cumulus. Some are just nice to have.
$ sudo apt-get install -y curl htop joe nmap virtualenv sqlite3 git
$ sudo apt-get install -y python-dateutil python-pysqlite2 build-essential
$ sudo apt-get install -y libssl-dev ant-optional python-dev openjdk-7-jdk
Important !!! With this version of Debian Jessie, you need to create a missing link manually. Otherwise you will get an error No module named _sysconfigdata_nd.
$ sudo ln -s /usr/lib/python2.7/plat-*/_sysconfigdata_nd.py /usr/lib/python2.7/
2) Create a user nimbus
$ sudo adduser nimbus
3) Become user nimbus and jump the its home directory
$ su nimbus
$ cd ~
4) Fetch the nimbus source code and decompress it
$ curl -O http://www.nimbusproject.org/downloads/nimbus-iaas-2.10.1-src.tar.gz
$ tar xvzf nimbus-iaas-2.10.1-src.tar.gz
5) Create a new folder for Cumulus and install the storage service into it
$ mkdir ~/cumulus
$ ./nimbus-iaas-2.10.1-src/cumulus/cumulus-install.sh ~/cumulus
6) Insert the search path into the bashrc file and restart the bash
$ echo "export CUMULUS_HOME=~/cumulus" > ~/.bashrc
$ bash
7) Start Cumulus
$ ~/cumulus/bin/cumulus &
Now the service should be running and accessible via port 8888
$ nmap localhost
Starting Nmap 6.47 ( http://nmap.org ) at 2017-04-01 15:14 CEST
Nmap scan report for localhost (127.0.0.1)
Host is up (0.00093s latency).
Other addresses for localhost (not scanned): 127.0.0.1
Not shown: 998 closed ports
PORT STATE SERVICE
22/tcp open ssh
8888/tcp open sun-answerbook
Nmap done: 1 IP address (1 host up) scanned in 0.28 seconds
8) Create a new user inside the nimbus service
$ ~/cumulus/bin/cumulus-add-user <your_email_address>@server.com
ID : 8OBgXSnj32JeLryk3KntF
password : I0V3ImtqVe7xpgysvaQuKAUZFpxDWpLGtMP3yBSmWW
quota : None
canonical id : f53235f8-165c-12e9-beb3-c827ebcead46
Just these lines need to be modified:
access_key = 8OBgXSnj32JeLryk3KntF
host_base = <the_ip_of_your_raspberry>:8888
host_bucket = <the_ip_of_your_raspberry>:8888
secret_key = I0V3ImtqVe7xpgysvaQuKAUZFpxDWpLGtMP3yBSmWW
use_https = False
Now s3cmd should work properly with the Cumulus service.
$ s3cmd mb s3://TESTBUCKET
Bucket 's3://TESTBUCKET/' created
$ s3cmd ls
2017-04-01 11:09 s3://TESTBUCKET
Important !!! When using Cumulus, all bucket names need to be encoded entirely in upper case letters. Otherwise you will just get this error message:
$ s3cmd mb s3://testbucket
WARNING: Retrying failed request: /?location (500 (InternalError): We encountered an internal error. Please try again.)
WARNING: Waiting 3 sec...
The config file of Cumulus is here:
~/cumulus/etc/cumulus.ini
Inside this file, it is among others specified...
- the port number of cumulus (
8888) - the local folder, which is used to store the buckets and objects (
/home/nimbus/cumulus/posixdata) - the hostname
- the logfile (
/home/nimbus/cumulus/log/cumulus.log) - the installation directory (
/home/nimbus/cumulus) - if HTTPS shall be used