geli
geli(8) is one of the most powerful block device-layer disk encryption system available in FreeBSD, which protects our data against cold storage attacks. geli(8) encrypts our data so that a skilled intruder cannot see sensitive documents, or modify our data without us noticing that a modification has taken place.
Instead of implementing geli(8) or any other FreeBSD supported encryption system in AppJail, I have preferred to create this howto. Encryption is a beautiful word, but it represents new problems that we need to accept before using it. We need to know where to store our keys and/or passphrases, our encrypted data, how to manage our keys and/or passphrases, how and when to decrypt the data (by booting the system or sometime later?), backups of our keys and/or passphrases, we need to be careful with our passphrases, a weak passphrase is a bad joke. Also if we use an external device (e.g. USB) to store our passphrases and/or keys, we need to be careful as such a device can be lost, stolen, etc., which means our data is lost.
geli(8) (or any other encryption system supported by FreeBSD) is not difficult to implement in AppJail, but implementing this kind of flexibility for each user is a challenge and the base tools are not difficult to use. AppJail is flexible and tries to adapt to your needs, so in these cases it is preferable to teach how to combine AppJail with other powerful FreeBSD tools instead of implementing each user case.
A passphrase is not the same as a password and should not be used in the same way as a key. A passphrase can be a single word, of course, like a password, but it is a weak passphrase and should not be used in a real environment.
A passphrase should not be limited to its length and can contain any character, and in a real environment it should be long with special characters, spaces and probably with words that are not used in your country. You should learn a passphrase, probably using a mnemonic technique.
A key should be generated using a program, not handwritten. You can use dd(1) to get a random key from the random device.
geli(8) is only part of the pie, not the whole pie, and as I mentioned, it protects you against cold storage attacks. At some point you need your data online, so you need to implement a better security mechanism and policy for this case to reduce the gap.
At this point we need to think our decisions very careful. For this howto I will use a USB with a single partition encrypted with geli(8). The idea with this partition is to use it as a master device where we will store our keys. For the partition I will not use a key, but a passphrase. The idea is to decrypt this single partition at boot time, so that jails can easily decrypt their encrypted file where the filesystem resides.
We will use files to store our data, and the memory disk driver will allow us to access the underlying data. Of course, we need to attach this device like any other device before the jail starts and detach it after it stops, which can be easily achieved using an AppJail template and some scripts.
We need more automation, so I've created some scripts to create, mount and unmount encrypted files.
create.sh:
#!/bin/sh
main()
{
local geli_init_args=
local images=
local keylen=64
local master_keys=
local newfs_args=-U
local name=
local size=48
while getopts ":g:i:k:m:N:n:s:" _o; do
case "${_o}" in
g)
geli_init_args="${OPTARG}"
;;
i)
images="${OPTARG}"
;;
k)
keylen="${OPTARG}"
;;
m)
master_keys="${OPTARG}"
;;
N)
newfs_args="${OPTARG}"
;;
n)
name="${OPTARG}"
;;
s)
size="${OPTARG}"
;;
*)
usage
exit 1
;;
esac
done
if [ -z "${images}" -o -z "${master_keys}" -o -z "${name}" ]; then
usage
exit 1
fi
local dirname
local imagedir master_keys_dir
dirname=`dirname "${name}"`
if [ "${dirname}" != "." ]; then
imagedir="${images}/${dirname}"
master_keys_dir="${master_keys}/${dirname}"
name=`basename "${name}"`
else
imagedir="${images}"
master_keys_dir="${master_keys}"
fi
if [ ! -d "${imagedir}" ]; then
mkdir -p "${imagedir}" || exit $?
fi
if [ ! -d "${master_keys_dir}" ]; then
mkdir -p "${master_keys_dir}" || exit $?
fi
local keyfile="${master_keys_dir}/${name}.key"
local imagefile="${imagedir}/${name}.img"
echo "Creating key ..."
dd if=/dev/random of="${keyfile}" bs="${keylen}" count=1 status=progress || exit $?
echo "Creating image ..."
dd if=/dev/random of="${imagefile}" bs="1m" count="${size}" status=progress || exit $?
echo "Initializing ..."
local unit
unit=`mdconfig -a -t vnode -f "${imagefile}"` || exit $?
eval geli init -P -K "${keyfile}" ${geli_init_args} "${unit}" || exit $?
echo "Attaching ..."
geli attach -p -k "${keyfile}" "${unit}" || exit $?
echo "Wiping ..."
dd if=/dev/random of="/dev/${unit}.eli" bs=1m status=progress
echo "Formatting ..."
eval newfs ${newfs_args} "/dev/${unit}.eli" || exit $?
echo "Cleaning ..."
geli detach "${unit}.eli" || exit $?
mdconfig -du "${unit}" || exit $?
echo "Done."
}
usage()
{
echo "usage: create.sh [-g geli_init_args] [-k KEYLEN] [-N newfs_args] [-s SIZE] -m MASTER_KEYS -i IMAGES -n NAME"
}
main "$@"mount.sh:
#!/bin/sh
main()
{
local images=
local mountpoint=
local master_keys=
local name=
while getopts ":i:M:m:n:" _o; do
case "${_o}" in
i)
images="${OPTARG}"
;;
M)
mountpoint="${OPTARG}"
;;
m)
master_keys="${OPTARG}"
;;
n)
name="${OPTARG}"
;;
*)
usage
exit 1
;;
esac
done
if [ -z "${images}" -o -z "${mountpoint}" -o -z "${master_keys}" -o -z "${name}" ]; then
usage
exit 1
fi
local image="${images}/${name}.img"
if [ ! -f "${image}" ]; then
echo "${name}: image not found."
exit 1
fi
local keyfile="${master_keys}/${name}.key"
if [ ! -f "${keyfile}" ]; then
echo "${name}: key not found."
exit 1
fi
local unit
unit=`mdconfig -l -n -f "${image}"`
local unit_total
unit_total=`echo ${unit} | wc -w`
if [ ${unit_total} -gt 0 ]; then
echo "${name}: There are many memory disk instances for this image."
exit 1
fi
unit=`mdconfig -a -t vnode -f "${image}"` || exit $?
geli attach -p -k "${keyfile}" "${unit}" || exit $?
fsck -p "/dev/${unit}.eli" || exit $?
geli detach -l "${unit}" || exit $?
mount "/dev/${unit}.eli" "${mountpoint}" || exit $?
}
usage()
{
echo "usage: mount.sh -M MOUNTPOINT -m MASTER_KEYS -i IMAGES -n NAME"
}
main "$@"umount.sh:
#!/bin/sh
main()
{
local images=
local mountpoint=
local name=
while getopts ":i:m:n:" _o; do
case "${_o}" in
i)
images="${OPTARG}"
;;
m)
mountpoint="${OPTARG}"
;;
n)
name="${OPTARG}"
;;
*)
usage
exit 1
;;
esac
done
if [ -z "${images}" -o -z "${mountpoint}" -o -z "${name}" ]; then
usage
exit 1
fi
local image="${images}/${name}.img"
if [ ! -f "${image}" ]; then
echo "${name}: image not found."
exit 1
fi
umount "${mountpoint}" || exit $?
local unit
unit=`mdconfig -l -n -f "${image}"`
local unit_total
unit_total=`echo ${unit} | wc -w`
if [ ${unit_total} -eq 0 ]; then
echo "${name}: There is no memory disk allocated for this image."
exit 1
elif [ ${unit_total} -gt 1 ]; then
echo "${name}: There are many memory disk instances for this image."
exit 1
fi
# Remove trailing space.
unit=`echo ${unit}`
if [ -c "/dev/md${unit}.eli" ]; then
geli detach "md${unit}.eli" || exit $?
fi
mdconfig -du "${unit}" || exit $?
}
usage()
{
echo "usage: umount.sh -m MOUNTPOINT -i IMAGES -n NAME"
}
main "$@"I'll explain it better later, be patient.
Before we continue, we probably want to put garbage to the old data of our device.
dd if=/dev/random of=/dev/da0 bs=1m status=progressNow we can proceed to create the partition schema and the single partition.
# gpart create -s gpt da0
da0 created
# gpart show da0
=> 40 60628912 da0 GPT (29G)
40 60628912 - free - (29G)
# gpart add -t freebsd-ufs -a 1m -s 1g da0
da0p1 added
# gpart show da0
=> 40 60628912 da0 GPT (29G)
40 2008 - free - (1.0M)
2048 2097152 1 freebsd-ufs (1.0G)
2099200 58529752 - free - (28G)
Initialize the device using geli init.
# geli init -b -a HMAC/SHA384 -e AES-XTS -l 256 da0p1
Enter new passphrase:
Reenter new passphrase:
Metadata backup for provider /dev/da0p1 can be found in /var/backups/da0p1.eli
and can be restored with the following command:
# geli restore /var/backups/da0p1.eli /dev/da0p1
-
-b: To try to decrypt the partition at boot time. -
-a HMAC/SHA384: We will useHMAC/SHA384as the authentication algorithm, although it is optional. -
-e AES-XTS: The encryption algorithm.AES-XTSis the default, but I prefer to say it explicitly. -
-l 256: Data Key length to use with the given encryption algorithm.
To format the encrypted partition, we must first attach it.
# geli attach da0p1
Enter passphrase:
# newfs -j /dev/da0p1.eli
/dev/da0p1.eli: 512.0MB (1048568 sectors) block size 32768, fragment size 4096
using 4 cylinder groups of 128.00MB, 4096 blks, 16384 inodes.
with soft updates
newfs: can't read old UFS1 superblock: read error from block device: Integrity check failedIf you decide to use authentication, you will encounter the above error. To fix this problem, we need to write the encrypted partition using dd(1).
# dd if=/dev/random of=/dev/da0p1.eli bs=1m status=progress
dd: /dev/da0p1.eli: short write on character device.098s, 1822 kB/s
dd: /dev/da0p1.eli: end of device
512+0 records in
511+1 records out
536870400 bytes transferred in 294.383959 secs (1823708 bytes/sec)If you are wondering why dd(1) has written only 512MiB on a 1GiB partition, here is the explanation:
-a aalgo Enable data integrity verification (authentication) using the given algorithm. This will reduce the size of storage available and also reduce speed. For example, when using 4096 bytes sector and HMAC/SHA256 algorithm, 89% of the original provider storage will be available for use. Currently supported algorithms are: HMAC/SHA1, HMAC/RIPEMD160, HMAC/SHA256, HMAC/SHA384 and HMAC/SHA512. If the option is not given, there will be no authentication, only encryption. The recommended algorithm is HMAC/SHA256.
Source: man 8 geli
Run newfs(8) again to format the encrypted partition, create a directory where to mount it and mount it.
# newfs -j /dev/da0p1.eli
/dev/da0p1.eli: 512.0MB (1048568 sectors) block size 32768, fragment size 4096
using 4 cylinder groups of 128.00MB, 4096 blks, 16384 inodes.
with soft updates
super-block backups (for fsck_ffs -b #) at:
192, 262336, 524480, 786624
Using inode 4 in cg 0 for 4194304 byte journal
newfs: soft updates journaling set
# mkdir /media/master-keys
# mount /dev/da0p1.eli /media/master-keysSuch a partition must be mounted every time our system is started, so put an entry in fstab(5):
echo "/dev/da0p1.eli /media/master-keys ufs rw,late 0 3" >> /etc/fstabYou can reboot your system to check if everything is OK.
It's time to explain in detail what scripts do. Of course, you need to decide where to put them. I chose /usr/local/scripts/geli4jails.
Note
The term image has nothing to do with AppJail images.
Create, initialize and format the encrypted file. This script also creates the key file.
-
-g(optional): Arguments used bygeli init. -
-k(default:64): Key length in bytes. The default is64bytes, so512bits. -
-N(default:-U): Arguments used bynewfs(8) -
-s(default:48): Image size in Mebibytes. -
-m(mandatory): Directory where the keys are stored. -
-i(mandatory): Directory where the images are stored. -
-n(mandatory): Image name. This will be concatenated with the master key directory and the image directory. The key filename will have the suffix.keyand the image filename will have the suffix.img.
Create a memory disk to attach the encrypted device and mount it in the given directory.
-
-M(mandatory): Directory where the device will be mounted. -
-m(mandatory): Directory where the keys are stored. -
-i(mandatory): Directory where the images are stored. -
-n(mandatory): Image name.
Unmount the mounted device in the given directory, detach the encrypted device and remove the memory disk instance.
-
-m(mandatory): Directory where the device is mounted. -
-i(mandatory): Directory where the images are stored. -
-n(mandatory): Image name.
We have to choose where to place the encrypted files. I chose /var/vnode-backed.d.
Using create.sh we can easily create an encrypted file. Of course, an encrypted file without data is useless, I will put an index.html file to explain to my customers that their web sites are encrypted [1]. To accomplish the last step we will use mount.sh to mount the encrypted file to a temporary directory. After doing our tasks we will use umount.sh to unmount it.
# /usr/local/scripts/geli4jails/create.sh -g "-a HMAC/SHA256" -N "-n" -m /media/master-keys -i /var/vnode-backed.d -n www04/usr_local_www_darkhttpd
Creating key ...
1+0 records in
1+0 records out
64 bytes transferred in 0.001864 secs (34328 bytes/sec)
Creating image ...
49283072 bytes (49 MB, 47 MiB) transferred 5.122s, 9621 kB/s
48+0 records in
48+0 records out
50331648 bytes transferred in 5.145210 secs (9782235 bytes/sec)
Initializing ...
Metadata backup for provider md0 can be found in /var/backups/md0.eli
and can be restored with the following command:
# geli restore /var/backups/md0.eli md0
Attaching ...
Wiping ...
dd: /dev/md0.eli: short write on character device17s, 2057 kB/s
dd: /dev/md0.eli: end of device
24+0 records in
23+1 records out
25165312 bytes transferred in 12.620492 secs (1994004 bytes/sec)
Formatting ...
/dev/md0.eli: 24.0MB (49144 sectors) block size 32768, fragment size 4096
using 4 cylinder groups of 6.00MB, 192 blks, 768 inodes.
super-block backups (for fsck_ffs -b #) at:
192, 12480, 24768, 37056
Cleaning ...
Done.
# /usr/local/scripts/geli4jails/mount.sh -M /tmp/www -m /media/master-keys -i /var/vnode-backed.d -n www04/usr_local_www_darkhttpd
# mdconfig -lv
md0 vnode 48M /var/vnode-backed.d/www04/usr_local_www_darkhttpd.img
# ls /dev/md0.eli
/dev/md0.eli
# mount | grep -F md0.eli
/dev/md0.eli on /tmp/www (ufs, local)
# cat << "EOF" > /tmp/www/index.html
> Your website is encrypted, muahahahaha!!!!!!
> EOF
# /usr/local/scripts/geli4jails/umount.sh -m /tmp/www -i /var/vnode-backed.d -n www04/usr_local_www_darkhttpdWe should have already created an encrypted file using create.sh for this step. Basically we will use an AppJail template to put the exec.prestart parameter to mount the encrypted file and exec.poststop to unmount the encrypted file.
template.conf:
$scriptsdir: /usr/local/scripts/geli4jails
$master_keys: /media/master-keys
$imagesdir: /var/vnode-backed.d
$image: ${name}/usr_local_www_darkhttpd
$mountpoint: usr/local/www/darkhttpd
exec.prestart+: "appjail cmd local ${name} ${scriptsdir}/mount.sh -M ${mountpoint} -m ${master_keys} -i ${imagesdir} -n ${image}"
exec.start: "/bin/sh /etc/rc"
exec.stop: "/bin/sh /etc/rc.shutdown jail"
exec.poststop+: "appjail cmd local ${name} ${scriptsdir}/umount.sh -m ${mountpoint} -i ${imagesdir} -n ${image}"
mount.devfs
Note
We use appjail cmd local to use a path relative to the jail directory.
We have already done all the steps, so we can proceed to create our jail, which in this case will be a web site hosted using darkhttpd.
# appjail makejail \
-j www04 \
-f gh+AppJail-makejails/darkhttpd \
-o virtualnet=":<random> default" \
-o nat \
-o template=$PWD/template.conf
# service appjail-dns restart && service dnsmasq restart # To update the DNS hostnames
# as quickly as possible, but
# this is not required if you
# do not have DNS enabled.
...
# fetch -qo - http://www04 # Of course, I use the DNS hostname, but
# you should use the IP address if you
# don't have DNS enabled in AppJail.
Your website is encrypted, muahahahaha!!!!!![1] Don't do this unless you want to explain to your customers what kind of ransomware has infected your servers.