[f] Run Tezos with blockchain data stored on a usb external storage unit and create a swap file on a SSD

---

This is a mandatory setup to run Tezos on the mainnet network with RPI3.

Use of a SSD is highly recommended (see section 05/10/19 below here)

---

The default locations of the blockchain data are ./.tezos-node/store and ./.tezos-node/context. With RPI3 Tezos performance increases dramatically when the data are stored in an external USB hard drive. Here is the info to make a smooth transition. Plug in the external drive and type fdisk -l to find out the location of the device. Assuming it is located in /dev/sda, the first step is to create a partition, as root/administrator, type:

fdisk /dev/sda
then use option n to create a new partition /dev/sda1. The new partition needs to be formatted (ext4 format, for better performance) with the command:

mkfs.ext4 /dev/sda1

Now it can be mounted, assuming the mounting point is /media/hd, the command is:

mount -o noatime /dev/sda1 /media/hd

Alternatively the disk can be automounted on boot by editing the following line to /etc/fstab:

/dev/sda1 /media/hd ext4 defaults,noatime 0 2

Since the storage unit has to be accessed by the <user>, privilege of the storage location needs to be changed from root to <user>:

chown <user> /media/hd
chgrp <user> /media/hd

Make sure the external drive is tuned for best performance as describe below here in the next section, it makes a huge difference!

The existing blockchain data in the default location should be copied in the new external unit:
cp -a /home/<user>/.tezos-node /media/hd

and the default data folders ./.tezos-node/store and ./.tezos-node/context should be deleted, e.g. rm -rf /home/<user>/.tezos-node/store.

It is easier to keep the config.json file in the original location ./tezos-node (otherwise the new location has to be provided running the program tezos-node using the flag --config-file=<new location>). The following line should be added to the file config.json after the first curly bracket to define the new path of the folders that store the blockchain data:

{
"data-dir": "/media/hd/.tezos-node",
......

---

03/27/2019
A dramatic improvement of the RPI3 performance running Tezos can be achieved by tuning the external HDD I/O. Assuming that the changes outlined in the UPDATES post 01/04/2019 (revised 01/27/19) have been already applied, the following steps should be added:

In the next steps the external HDD is labelled sda and is mounted on /media/hd, first unmount the HDD, as a root type:
umount /media/hd

then type:
echo 8192 > /sys/block/sda/device/max_sectors
(default max_sectors is 240)
and mount back the external HDD:
mount -o noatime /dev/sda1 /media/hd

then add the following changes:
echo 4096 > /sys/block/sda/queue/max_sectors_kb
echo 8 > /sys/block/sda/queue/nr_requests
echo 0 > /sys/block/sda/queue/read_ahead_kb
(default values are: 120 for max_sectors_kb; 128 for nr_requests and 128 for read_ahead_kb)

and
echo noop > /sys/block/sda/queue/scheduler

Now the Tezos node on the mainnet can be started and the baker and endorser daemons should be launched after the node has been synced. Tezos will keep running very smoothly with no delay or performance loss.

The above modifications are non permanent, at the next reboot of the RPI3 the default values are restored automatically. The udev daemon can create permanent new rules for the USB device, more on this below here.

---

05/10/2019
With the release of the new linux kernel 5.x.x, there are different scheduler options available (to find which kernel is loaded, as root type uname -r).

To see the available scheduler:
cat /sys/block/sda/queue/scheduler
we choose "none":
echo none > /sys/block/sda/queue/scheduler

then increase the max size of the logical sectors:
echo 16384 > /sys/block/sda/device/max_sectors
and the actual size:
echo 8192 > /sys/block/sda/device/queue/max_sectors_kb

the parameter "nr_request is now by default set to 1, no need to change it. Just decrease the default value of read_ahead_kb:
echo 0 > /sys/block/sda/device/queue/read_ahead_kb

These changes are still non permanent, they need to be set again after each restart of the RPI3 (see below here for adding permanent changes).

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Should we use an external SSD to store Tezos data in /.tezos-node? ABSOLUTELY YES!

Even though RPI3 has only USB2 ports, R/W on the SSD is x~5-7 faster than on an average HDD.
The disk tuning outlined above here still applies.

Make sure also that the parameter "rotational" is set to 0 (for SSD):
cat /sys/block/sda/queue/rotational
in case the value is 1, it should be changed:
echo 0 > /sys/block/sda/queue/rotational

---

06/06/19
To make permanent changes to the external disk parameters, in /lib/udev/rules.d create a file with name 60-sda-scheduler.rules and add the following line (for an SSD external device):

ACTION=="add|change", KERNEL=="sda", RUN+="/bin/sh -c 'echo 16384 > /sys/block/sda/device/max_sectors'", ATTR{queue/rotational}="0", ATTR{queue/scheduler}="none", ATTR{queue/read_ahead_kb}="0", RUN+="/bin/sh -c 'echo 8192 > /sys/block/sda/queue/max_sectors_kb'"

With an external HDD, replace 0 with 1 in ATTR{queue/rotational} above here.

Use the following command to make sure the changes are correct:
udevadm info --name=/dev/sda --attribute-walk

To activate the change either reboot the machine or type:
udevadm control --reload
udevadm trigger

---

With an external SSD it is much better (faster) to have the swap space on the external drive. Even though it is barely used while running Tezos, compilation of Tezos can be painful on RPI3, having the swap memory allocated on the SDD mitigates the hassle. Side note, during the recent protocol upgrade (004-Pt24m4xi), large portion of the swap on the SSD was used during the transition, but the node on the RPI3 stayed synced all the time.

The SSD must be mounted on boot (see the instruction above here), in addition the original swap on the sd card should be removed. In /etc/fstab comment the line that looks like this (to comment the line add #):
# UUID=a3249530-b6dc-41a4-b213-f85856a7ccb0 swap swap defaults 0 0

To add 2GB of swap assuming that the ssd is mounted on /media/hd, as root type:
dd if=/dev/zero of=/media/hd/swapfile bs=1024 count=2097152

change the permission:
chmod 600 /media/hd/swapfile

then setup the swap and bring it up:
mkswap /media/hd/swapfile
swapon /media/hd/swapfile

finally add the following line in /etc/fstab to make the change permanent on boot:
/media/hd/swapfile swap swap defaults 0 0

Using the command free -m one can see that the total swap should be 2048MB. With the additional 120MB of swap created by zram (see page [b]) the total swap space is 2168MB.