Complemented chapter 23.2.2 scsicrypto-sd; Added chapter C.4.6 bscrypto

Signed-off-by: Joerg Steffens <joerg.steffens@bareos.com>

manuals/en/main/bareos.sty

@@ -153,6 +153,7 @@
 \newcommand{\user}[1]{\path|#1|}
 \newcommand{\package}[1]{\path|#1|}
 \newcommand{\parameter}[1]{\path|#1|}
+\newcommand{\pluginevent}[1]{\path|#1|}
 \newcommand{\argument}[1]{\textit{#1}}
 \newcommand{\registrykey}[1]{\path|#1|}
 \newcommand{\variable}[1]{\path|#1|}

manuals/en/main/plugins.tex

@@ -237,6 +237,176 @@ \subsection{scsicrypto-sd}
 
 This plugin is part of the \package{bareos-storage} package.
 
+\subsubsection{LTO Hardware Encryption - General}
+\label{LTOHardwareEncryptionGeneral}
+
+Modern tape-drives, for example LTO (from LTO4 onwards) support hardware encryption.
+There are several ways of using encryption with these drives. The following three types of key management are available for encrypting drives. The transmission of the keys to the volumes is accomplished by either of the three:
+
+\begin{itemize}
+ \item A backup application that supports Application Managed Encryption (AME)
+ \item A tape library that supports Library Managed Encryption (LME)
+ \item A Key Management Appliance (KMA).
+\end{itemize}
+
+We added support for Application Managed Encryption (AME) scheme, where on labeling a crypto key is generated for a volume and when the volume is mounted, the crypto key is loaded. When finally the volume is unmounted, the key is cleared from the memory of the Tape Drive using the SCSI SPOUT command set.
+
+If you have implemented Library Managed Encryption (LME) or a Key Management Appliance (KMA), there is no need to have support from Bareos on loading and clearing the encryption keys, as either the Library knows the per volume encryption keys itself, or it will ask the KMA for the encryption key when it needs it. For big installations you might consider using a KMA, but the Application Managed Encryption implemented in Bareos should also scale rather well and have a low overhead as the keys are only loaded and cleared when needed.
+
+\subsubsection{The bscrypto tool}
+
+For information regarding the \command{bscrypto} tool, see chapter \nameref{bscrypto}.
+
+\subsubsection{The scsicrypto-sd plugin}
+
+The \command{scsicrypto-sd} hooks into the \pluginevent{unload}, \pluginevent{label read}, \pluginevent{label write} and \pluginevent{label verified} events for loading and clearing the key. It checks whether it it needs to clear the drive by either using an internal state (if it loaded a key before) or by checking the state of a special option that first issues an encrytion status query. If there is a connection to the director and the volume information is not available, it will ask the director for the data on the currently loaded volume. If no connection is available, a cache will be used which should contain the most recently mounted volumes. If an encryption key is available, it will be loaded into the drive's memory.
+
+\subsubsection{Changes in the director}
+
+The director has been extended with additional code for handling hardware data encryption. The extra keyword "encrypt" on the label of a volume will force the director to generate a new semi-random passphrase for the volume, which will be stored in the database as part of the media information.
+
+A passphrase is always stored in the database base64-encoded. When a so called Key Encryption Key is set in the config of the director, the passphrase is first wrapped using RFC3394 key wrapping and then base64-encoded. By using key wrapping, the keys in the database are safe against people sniffing the info, as the data is still encrypted using the Key Encryption Key (which in essence is just an extra passphrase of the same length as the volume passphrases used).
+
+When the storage daemon needs to mount the volume, it will ask the director for the volume information and that protocol is extended with the exchange of the base64-wrapped encryption key (passphrase). The storage daemon provides an extra config option in which it records the Key Encryption Key of the particular director, and as such can unwrap the key sent into the original passphrase.
+
+As can be seen from the above info we don't allow the user to enter a passphrase, but generate a semi-random passphrase using the openssl random functions (if available) and convert that into a readable ASCII stream of letters, numbers and most other characters, apart from the quotes and space etc. This will produce much stronger passphrases than when requesting the info from a user. As we store this information in the database, the user never has to enter these passphrases.
+
+The volume label is written in unencrypted form to the volume, so we can always recognize a Bareos volume. When the key is loaded onto the drive, we set the decryption mode to mixed, so we can read both unencrypted and encrypted data from the volume. When no key or the wrong key has been loaded, the drive will give an IO error when trying to read the volume.
+For disaster recovery you can store the Key Encryption Key and the content of the wrapped encryption keys somewhere safe and the bscrypto tool together with the scsicrypto-sd plugin can be used to get access to your volumes, in case you ever lose your complete environment.
+
+If you don't want to use the scsicrypto-sd plugin when doing DR and you are only reading one volume, you can also set the crypto key using the bscrypto tool. Because we use the mixed decryption mode, in which you can read both encrypted and unencrypted data from a volume, you can set the right encryption key before reading the volume label.
+
+If you need to read more than one volume, you better use the scsicrypto-sd plugin with tools like bscan/bextract, as the plugin will then auto-load the correct encryption key when it loads the volume, similiarly to what the storage daemon does when performing backups and restores.
+
+The volume label is unencrypted, so a volume can also be recognized by a non-encrypted installation, but it won't be able to read the actual data from it. Using an encrypted volume label doesn't add much security (there is no security-related info in the volume label anyhow) and it makes it harder to recognize either a labeled volume with encrypted data or an unlabeled new volume (both would return an IO-error on read of the label.)
+
+\subsubsection{SCSI crypto setup}
+%
+The initial setup of SCSI crypto looks something like this:
+%
+\begin{itemize}
+ \item Generate a Key Encryption Key e.g.
+\begin{commands}{}
+bscrypto -g -
+\end{commands}
+\end{itemize}
+
+Some security levels need to be increased for the storage daemon to be able to use the low level SCSI interface for setting and getting the encryption status on a tape device.
+
+The following additional security is needed for the following operating systems:
+
+\paragraph{Security Setup}
+
+\subparagraph{Linux (SG\_IO ioctl interface):} The user running the storage daemon needs the following additional capabilities:
+
+\begin{itemize}
+ \item \parameter{CAP_SYS_RAWIO} (See capabilities(7))
+ \begin{itemize}
+  \item On older kernels you might need \parameter{CAP_SYS_ADMIN}. Try \parameter{CAP_SYS_RAWIO} first and if that doesn't work try \parameter{CAP_SYS_ADMIN}
+ \end{itemize}
+ \item If you are running the storage daemon as another user than root (which has the \parameter{CAP_SYS_RAWIO} capability), you need to add it to the current set of capabilities.
+ \item If you are using systemd, you could add this additional capability to the CapabilityBoundingSet parameter.
+ \begin{itemize}
+  \item For systemd add the following to the bareos-sd.service: \parameter{Capabilities=cap_sys_rawio+ep}
+ \end{itemize}
+\end{itemize}
+
+You can also set up the extra capability on bscrypto and bareos-sd by running the following commands:
+
+\begin{commands}{}
+setcap cap_sys_rawio=ep bscrypto
+setcap cap_sys_rawio=ep bareos-sd
+\end{commands}
+
+Check the setting with
+
+\begin{commands}{}
+getcap -v bscrypto
+getcap -v bareos-sd
+\end{commands}
+
+\command{getcap} and \command{setcap} are part of libcap-progs.\\
+
+\subparagraph{Solaris (USCSI ioctl interface):} The user running the storage daemon needs the following additional privileges:
+
+\begin{itemize}
+ \item \parameter{PRIV_SYS_DEVICES} (See privileges(5))
+\end{itemize}
+
+If you are running the storage daemon as another user than root (which has the \parameter{PRIV_SYS_DEVICES} privilege), you need to add it to the current set of privileges.
+This can be set up by setting this either as a project for the user, or as a set of extra privileges in the SMF definition starting the storage daemon. The SMF setup is the cleanest one.
+
+For SMF make sure you have something like this in the instance block:
+
+\begin{bconfig}{}
+<method_context working_directory=":default"> <method_credential user="bareos" group="bareos" privileges="basic,sys_devices"/> </method_context>
+\end{bconfig}
+
+\paragraph{Changes in bareos-sd.conf}
+
+\begin{itemize}
+ \item The entry \parameter{Key Encryption Key} has to be put into the \file{bareos-sd.conf} below the director entry in the config for the specific director you are creating the config for, e.g.
+ \begin{bconfig}{}
+  Key Encryption Key = "<passphrase>"
+ \end{bconfig}
+ \item Enable the loading of storage daemon plugins by setting the plugin dir in the bareos-sd.conf, e.g.
+ \begin{bconfig}{}
+  Plugin Directory = <path_to_sd_plugins>
+ \end{bconfig}
+ \item Enable the SCSI encryption option in the device configuration section of the drive in the bareos-sd.conf, e.g.
+ \begin{bconfig}{}
+  Drive Crypto Enabled = Yes
+ \end{bconfig}
+ \item If you want the plugin to probe the encryption status of the drive when it needs to clear a pending key, enable the Query Crypto Status option in the device configuration section of the drive in the bareos-sd.conf e.g.
+ \begin{bconfig}{}
+  Query Crypto Status = Yes
+ \end{bconfig}
+\end{itemize}
+
+\paragraph{Changes in bareos-dir.conf}
+
+\begin{itemize}
+ \item Put the Key Encryption Key into the bareos-dir.conf under the director config item named \parameter{Key Encryption Key}, e.g.
+ \begin{bconfig}{}
+ Key Encryption Key = "<passphrase>"
+ \end{bconfig}
+ \item restart sd and dir
+ \item Label a volume with the encrypt option, e.g.
+ \begin{bconfig}{}
+  label slots=1-5 barcodes encrypt
+ \end{bconfig}
+\end{itemize}
+
+For Disaster Recovery (DR) you need the following information:
+
+\begin{itemize}
+ \item Actual bareos-sd.conf with config options enabled as described above, including, among others, a definition of a director with the Key Encryption Key used for creating the encryption keys of the volumes.
+ \item The actual keys used for the encryption of the volumes.
+\end{itemize}
+
+This data needs to be availabe as a so called crypto cache file which is used by the plugin when no connection to the director can be made to do a lookup (most likely on DR).
+
+Most of the times the needed information, e.g. the bootstrap info, is available on recently written volumes and most of the time the encryption cache will contain the most recent data, so a recent copy of the bareos-sd.<portnr>.cryptoc file in the workingdir is enough most of the time. You can also save the info from database in a safe place and use bscrypto to populate this info (VolumeName<tab>EncryptKey) into the crypto cache file used by bextract and bscan. You can use bscrypto with the following flags to create a new or update an existing crypto cache file e.g.:
+
+\begin{commands}{}
+ bscrypto -p /var/lib/bareos/bareos-sd.<portnr>.cryptoc
+\end{commands}
+
+\begin{itemize}
+ \item A valid BSR file containing the location of the last safe of the database makes recovery much easier. Adding a post script to the database save job could collect the needed info and make sure its stored somewhere safe.
+ \item Recover the database in the normal way e.g. for postgresql:
+ \begin{commands}{}
+ bextract -D <director_name> -c bareos-sd.conf -V <volname> \ /dev/nst0 /tmp -b bootstrap.bsr
+ /usr/lib64/bareos/create_bareos_database
+ /usr/lib64/bareos/grant_bareos_privileges
+ psql bareos < /tmp/var/lib/bareos/bareos.sql
+ \end{commands}
+\end{itemize}
+
+Or something similar (change paths to follow where you installed the software or where the package put it.)
+
+\textbf{Note:} As described at the beginning of this chapter, there are different types of key management, AME, LME and KMA. If the Library is set up for LME or KMA, it probably won't allow our AME setup and the scsi-crypto plugin will fail to set/clear the encryption key. To be able to use AME you need to "Modify Encryption Method" and set it to something like "Application Managed". If you decide to use LME or KMA you don't have to bother with the whole setup of AME which may for big libraries be easier, although the overhead of using AME even for very big libraries should be minimal.
+
 \subsection{scsitapealert-sd}
 \index[general]{Plugin!scsitapealert-sd}
 

manuals/en/main/programs.tex

@@ -201,13 +201,19 @@ \subsection{bat}
 \index[general]{Command!bat}
 \label{bat}
 
+The Bacula/Bareos Administration Tool (\command{bat}) is a native GUI for Bareos. Please note that, although \command{bat} is still a part of Bareos and still receives maintenance bugfixes, the main development effort will be spent on bareos-webui. For \sinceVersion{dir}{bat vs. bareos-webui}{15.2.0} we therefore encourage the use of Bareos Webui, which will, in the long term, replace \command{bat}. By now Bareos Webui does not have all the features \command{bat} offers.
 
+\subsection{bareos-webui}
+\index[general]{Command!bareos-webui}
+\label{bareos-webui}
+
+For further information regarding the Bareos Webui, please refer to the chapter \nameref{}.
 
 \section{Volume Utility Commands}
     \index[general]{Volume Utility Tools}
     \index[general]{Tools!Volume Utility}
 
-\warning{If you using Bareos with non-default block sizes defined in the pools, it might be neccessary to specifiy the \configdirective{maximum block level} also in the storage device resource, see \ilink{Direct access to Volumes with non-default blocksizes}{direct-access-to-volumes-with-non-default-blocksizes}.}
+\warning{If you use Bareos with non-default block sizes defined in the pools, it might be neccessary to specifiy the \configdirective{maximum block level} also in the storage device resource, see \ilink{Direct access to Volumes with non-default blocksizes}{direct-access-to-volumes-with-non-default-blocksizes}.}
 
 \subsection{bls}
 \label{bls}
@@ -1159,8 +1165,35 @@ \subsubsection{btape Commands}
 
 You can change the block size in the Storage Daemon configuration file.
 
+\subsection{bscrypto}
+\label{bscrypto}
+\index[general]{bscrypto}
+\index[general]{Command!bscrypto}
 
+\command{bscrypto} is used in the process of encrypting tapes (see also \nameref{LTOHardwareEncryptionGeneral}). The storage daemon and the btools (\command{bls}, \command{bextract}, \command{bscan}, \command{btape}, \command{bextract}) will use a so called storage daemon plugin to perform the setting and clearing of the encryption keys. To bootstrap the encryption support and for populating things like the crypto cache with encryption keys of volumes that you want to scan, you need to use the bscrypto tool.
+The bscrypto tool has the following capabilities:
 
+\begin{itemize}
+ \item Generate a new passphrase
+ \begin{itemize}
+  \item to be used as a so called Key Encryption Key (KEK) for wrapping a passphrase using RFC3394 key wrapping with aes-wrap\\- or -
+  \item for usage as a clear text encryption key loaded into the tape drive.
+ \end{itemize}
+ \item Base64-encode a key if requested
+ \item Generate a wrapped passphrase which performs the following steps:
+ \begin{itemize}
+  \item generate a semi random clear text passphrase
+  \item wrap the passphrase using the Key Encryption Key using RFC3394
+  \item base64-encode the wrapped key (as the wrapped key is binary, we always need to base64-encode it in order to be able to pass the data as part of the director to storage daemon protocol
+ \end{itemize}
+ \item show the content of a wrapped or unwrapped keyfile.\\This can be used to reveal the content of the passphrase when a passphrase is stored in the database and you have the urge to change the Key Encryption Key. Normally it is unwise to change the Key Encryption Key, as this means that you have to redo all your stored encryption keys, as they are stored in the database wrapped using the Key Encryption Key available in the config during the label phase of the volume.
+ \item Clear the crypto cache on the machine running the bareos-sd, which keeps a cache of used encryption keys, which can be used when the bareos-sd is restarted without the need to connect to the bareos-dir to retrieve the encryption keys.
+ \item Set the encryption key of the drive
+ \item Clear the encryption key of the drive
+ \item Show the encryption status of the drive
+ \item Show the encryption status of the next block (e.g. volume)
+ \item Populate the crypto cache with data
+\end{itemize}
 
 \section{Other Programs}