LVM #83

contrib/lvm-snapshot/README.md

@@ -0,0 +1,54 @@
+lvm-snap.pre-apply
+
+Description: creates an LVM snapshot on the logical volume underneath root filesystem, if applicable.
+
+# Script Details
+
+Language: bash
+Supported OS: CENTOS>=6 RHEL>=6 Fedora>=26 Ubuntu>=16.04 Debian>=9
+Additional setup required: NO
+Dependencies: util-linux, lvm2
+
+# Forewords of caution
+This script is not going through each and every possible checks to ensure a snapshot revert will be successful. It was tested in different scenario and will do a reasonable amount of checks, halting on possible issues, but we do not guarantee that a revert will always be successful and/or without issues. We strongly recommand to do your own testing before using this on anything you hold dear.
+
+# Description
+
+This script executes the following steps:
+
+* Check if lvm is in use for the root filesystem
+* Check if /bin, /etc, /lib, /lib64, /sbin, /usr, /var are on a separate filesystem => halt if confirmed
+* Check if there is a reasonable amount of free unallocated space in the VG (default = 20 %)
+* Check if there are already snapshots created (we do not allow multiple snapshots to be created by default)
+* If all checks are passed then create a new snapshot with a descriptive name associating it with auter (format: <snap_root_auter_YYYY-MM-DD_HHMMSS>)
+* Set up automatic removal of the snapshot after a number of days (default = 3). If the removal fails, modify /etc/motd to alert about the failed removal on next login.
+
+# Pre-requisites and dependencies
+
+* The script is using command **findmnt** to help determine if root filesystem '/' is on an LVM logical volume. You need util-linux package to use this program.
+* You **should not** make use of this script if your root filesystem '/' **is not** on an LVM logical volume, otherwise it will fail with a non-zero code exit that will prevent auter to run.
+* LVM command-line tools need to be installed on the system (lvcreate / lvremove /lvs /vgs). This should need require extra installing if you have lvm2 package installed.
+
+Variables at the beginning of the script may be altered to adjust to your needs.
+
+
+# Files and explainations
+
+This should be a list of files that should be included in the directory and an expaination of what each file does
+^ File ^ Purpose ^
+| lvm-snap.pre-apply | handles all steps: lvm checks, snapshot creation and scheduled removal |
+
+# Reverting a snapshot
+If you decide to revert the root filesystem to the time of the snapshot, you will need to do the following.
+
+**WARNING**: you will lose all data modified on the logical volume since the snapshot. This is **not** reversible.
+
+1. Merge the LVM snapshot with its origin to revert:
+```
+lvconvert --mergesnapshot <LV_VG>/<LV_SNAP>
+[output]
+  Can't merge until origin volume is closed.
+  Merging of snapshot centos/snap_root_auter_2018-01-08_092517 will occur on next activation of centos/root.
+```
+1. **If** your /boot directory resides on an separate filesystem, you **need** to update the grub default file to load the second entry (ie set up `GRUB_DEFAULT=1` in /etc/default/grub -- this will match previously installed kernel. If in doubt, look back into the script's log file and compare with list of grub entries in /boot/grub2/grub.conf . Keep in mind grub starts numbering at 0.)
+1. Reboot the device to validate.

contrib/lvm-snapshot/lvm-snap.pre-apply

@@ -0,0 +1,154 @@
+#!/bin/bash
+
+# Context:  auter is a yum-cron type package which implements automatic updates on an
+# individual server with features such as predownloading packages and reboots. 
+#
+# This script executes the following steps:
+
+# * Check if lvm is in use for the root filesystem
+# * Check if there is a reasonable amount of free unallocated space in the VG
+# * Check if there are already snapshots created (we do not allow multiple snapshots to be created by default)
+# * If all checks are passed then create a new snapshot with a descriptive name associating it with auter
+# * Set up automatic removal of the snapshot after a number of days (default = 3). If the removal fails, modify 
+#   /etc/motd to alert about the failed removal on next login.
+# 
+# Copyright 2018 Rackspace, Inc.
+# This script is provided as a courtesy and is not officially supported by Rackspace.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may not use
+# this file except in compliance with the License.  You may obtain a copy of the
+# License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software distributed
+# under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
+# CONDITIONS OF ANY KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations under the License.
+#
+
+
+############
+## Variables
+############
+
+# Minimum free space on the VG in percentage to allow snapshot creation.
+# This will also define maximum snapshot size.
+# If the snapshot consumes all its reserved space, you won't be able to revert, 
+# so please exerce caution when changing this.
+readonly MIN_PERC_AVAIL=20
+# Number of days to keep the LVM snapshot running. The longer we keep it, the more 
+# space it is likely to consume.
+readonly DAYS=3
+# Where to log messages
+readonly LOGDIR=/var/log/auter-lvm
+readonly LOGFILE=${LOGDIR}/$(basename $0).log
+# Should we create new snapshot on root logical volume if one already exists? (0 = NO; 1 = YES)
+readonly MULTI_SNAP=0
+# Name of the snapshot
+readonly LV_SNAP="snap_root_auter_$(date +%Y-%m-%d_%H%M%S)"
+
+# Journal fonction
+jnl() {
+  echo "$(date +%c)  $1" | tee -a ${LOGFILE}
+}
+
+# Check if a submitted path is on separate filesystem
+check_fs_separate() {
+  local myfs="$1"
+
+  MY_FS=$(findmnt -n -l -o SOURCE ${myfs} 2>/dev/null) && { jnl "ERROR: ${myfs} is part of a different filesystem, this could be dramatic in case of a snapshot revert. Aborting!"; exit 2; } || jnl "INFO: ${myfs} is not on a separate filesystem, continuing..."
+}
+
+# Check that root is on an LVM volume and init variables
+check_root_lvm() {
+  ROOT_FS=$(findmnt -n -l -o SOURCE /)
+  jnl "INFO: Root filesystem info: ${ROOT_FS}"
+
+  # Try to get LVM info
+  readonly LV_INFO=$(lvs --noheadings -o lv_name,vg_name,lv_size --units m --nosuffix ${ROOT_FS} 2>/dev/null) || { jnl "ERROR: The root filesystem is NOT LVM, quitting..."; exit 2; }
+
+  readonly LV_ROOT=$(echo "$LV_INFO" | awk '{ print $1 }')
+  readonly LV_VG=$(echo "$LV_INFO" | awk '{ print $2 }')
+  readonly LV_SIZE=$(echo "$LV_INFO" | awk '{ match($3, /(^[0-9]+)\.[0-9]+/, res); print res[1] }')
+
+  jnl "INFO: Root filesystem identified on LV ${LV_ROOT} belonging to VG ${LV_VG}"
+
+  # Checking for 'core' FHS directories mounted on a separate FS. Halting if found 
+  for i in /etc /lib /lib64 /sbin /usr /var
+  do
+    check_fs_separate ${i}
+  done
+  #USR_FS=$(findmnt -n -l -o SOURCE /usr 2>/dev/null) && { jnl "ERROR: /usr is part of a different filesystem, this could be dramatic in case of a snapshot revert. Aborting!"; exit 2; } || jnl "INFO: /usr is not on a separate filesystem, continuing..."
+
+  # Is /boot on another FS/LV?
+  BOOT_FS=$(findmnt -n -l -o SOURCE /boot 2>/dev/null) && jnl "WARN: /boot is on a different filesystem, you will need to amend grub configuration to point to current kernel in case of a revert LVM snapshot on the root volume." || jnl "INFO: /boot is not on a separate filesystem"
+}
+
+# If conditions are met, creating a new LVM snapshot
+create_snapshot() {
+  # Do we have enough space?
+  SPACE_LEFT=$(vgs --noheading -o vg_free --units m --nosuffix ${LV_VG} | awk '{ match($1, /(^[0-9]+)\.[0-9]+/, res); print res[1] }') 
+  jnl "INFO: There is ${SPACE_LEFT} MB available free space on VG"
+
+  SPACE_NEED=$(expr $MIN_PERC_AVAIL \* $LV_SIZE / 100)
+  jnl "INFO: We need ${SPACE_NEED} MB available free space on VG"
+
+  if [ $SPACE_LEFT -lt $SPACE_NEED ]
+  then
+    jnl "ERROR: Not enough free space on VG $LV_VG ( $SPACE_LEFT < $SPACE_NEED). Quitting..."
+    exit 2
+  fi
+
+  # Is there any snapshot on-going for our VG?
+  SNAP_LIST=$(lvs -S 'lv_attr =~ ^s' --noheading -o origin --rows ${LV_VG})
+
+  # Testing if multiple snapshots on the VG are authorised
+  if [ ${MULTI_SNAP} -eq 1 ]
+  then
+    jnl "WARN: We are authorising multiple snapshots."
+  else
+    [[ "${SNAP_LIST}" =~ "${LV_ROOT}" ]] && { jnl "ERROR: VG ${LV_VG} already contains snapshots on ${SNAP_LIST}. Aborting..."; exit 2; } || jnl "INFO: No previous snapshot detected. Steady as she goes..."
+  fi
+
+  # Everything checks out
+  jnl "INFO: Creating snapshot ${LV_SNAP}"
+  lvcreate -n $LV_SNAP -l ${MIN_PERC_AVAIL}%ORIGIN -s /dev/${LV_VG}/${LV_ROOT} >> $LOGFILE 1>&1 && jnl "INFO: Snaphot created successfully." || { jnl "ERROR: Could not create snapshot.";exit 2; }
+
+  # Write down kernel version in case of a revert is needed, for extra information
+  jnl "INFO: Kernel version running: $(uname -r)."
+  [ -n "$BOOT_FS" ] && jnl "WARN: You should ensure to keep this version installed during snapshot lifespan to be able to revert snapshot and boot to this kernel."
+} 
+
+# Scheduling automatic removal of the snapshot
+remove_snapshot() {
+  jnl "INFO: Setting removal of the snapshot in $DAYS days."
+
+  # set up snapshot removal with system logging
+  at "now +$DAYS days" <<EOF
+/sbin/lvremove -y ${LV_VG}/${LV_SNAP} && logger "INFO: LVM snapshot ${LV_SNAP} has been removed" || { logger "ERROR: LVM snapshot removal of ${LV_SNAP} did not return exit code 0";echo -e "\nWARNING: auter snapshot ${LV_SNAP} could not be removed.\n**Please remove the snapshot manually**.\nOnce ok, please remove this line from /etc/motd" >> /etc/motd; }
+EOF
+}
+
+##########################################################
+##
+##           MAIN 
+##
+##########################################################
+
+# Check for root user
+[ $UID -eq 0 ] || { echo "ERROR: Requires root privileges"; exit 1; }
+
+# Create log dir if needed
+[ -d ${LOGDIR} ] || mkdir -p ${LOGDIR}
+
+# Is root FS an LVM volume?
+check_root_lvm
+
+# Run snapshot after checking it is ok to do so
+create_snapshot
+
+## set up at task for snapshot remove in $DAYS
+remove_snapshot
+
+exit 0