Add logic for upgrading datadir (#34)

* Upgrading and data directory version checking

MySQL and MariaDB use versions that consist of three numbers X.Y.Z (e.g. 5.6.23).
For version changes in Z part, the server's binary data format stays compatible and thus no
special upgrade procedure is needed. For upgrades from X.Y to X.Y+1, consider doing manual
steps as described at
https://mariadb.com/kb/en/library/upgrading-from-mariadb-101-to-mariadb-102/

Skipping versions like from X.Y to X.Y+2 or downgrading to lower version is not supported;
the only exception is ugrading from MariaDB 5.5 to MariaDB 10.0.

**Important**: Upgrading to a new version is always risky and users are expected to make a full
back-up of all data before.

A safer solution to upgrade is to dump all data using `mysqldump` or `mysqldbexport` and then
load the data using `mysql` or `mysqldbimport` into an empty (freshly initialized) database.

Another way of proceeding with the upgrade is starting the new version of the `mysqld` daemon
and run `mysql_upgrade` right after the start. This so called in-place upgrade is generally
faster for large data directory, but only possible if upgrading from the very previous version,
so skipping versions is not supported.

This container detects whether the data needs to be upgraded using `mysql_upgrade` and
we can control it by setting `MYSQL_DATADIR_ACTION` variable, which can have one or more of the following values:

 * `upgrade-warn` -- If the data version can be determined and the data come from a different version
   of the daemon, a warning is printed but the container starts. This is the default value.
   Since historically the version file `mysql_upgrade_info` was not created, when using this option,
   the version file is created if not exist, but no `mysql_upgrade` will be called.
   However, this automatic creation will be removed after few months, since the version should be
   created on most deployments at that point.
 * `upgrade-auto` -- `mysql_upgrade` is run at the beginning of the container start, when the local
   daemon is running, but only if the data version can be determined and the data come
   with the very previous version. A warning is printed if the data come from even older
   or newer version. This value effectively enables automatic upgrades,
   but it is always risky and users should still back-up all the data before starting the newer container.
   Set this option only if you have very good back-ups at any moment and you are fine to fail-over
   from the back-up.
 * `upgrade-force` -- `mysql_upgrade --force` is run at the beginning of the container start, when the local
   daemon is running, no matter what version of the daemon the data come from.
   This is also the way to create the missing version file `mysql_upgrade_info` if not present
   in the root of the data directory; this file holds information about the version of the data.

There are also some other actions that you may want to run at the beginning of the container start,
when the local daemon is running, no matter what version of the data is detected:

 * `optimize` -- runs `mysqlcheck --optimize`. It optimizes all the tables.
 * `analyze` -- runs `mysqlcheck --analyze`. It analyzes all the tables.
 * `disable` -- nothing is done regarding data directory version.

Multiple values are separated by comma and run in-order, e.g. `MYSQL_DATADIR_ACTION="optimize,analyze"`.

10.0/root/usr/share/container-scripts/mysql/README.md

@@ -260,6 +260,60 @@ Such a directory `sslapp` can then be mounted into the container with -v,
 or a new container image can be built using s2i.
 
 
+Upgrading and data directory version checking
+---------------------------------------------
+
+MySQL and MariaDB use versions that consist of three numbers X.Y.Z (e.g. 5.6.23).
+For version changes in Z part, the server's binary data format stays compatible and thus no
+special upgrade procedure is needed. For upgrades from X.Y to X.Y+1, consider doing manual
+steps as described at
+https://mariadb.com/kb/en/library/upgrading-from-mariadb-55-to-mariadb-100/
+
+Skipping versions like from X.Y to X.Y+2 or downgrading to lower version is not supported;
+the only exception is ugrading from MariaDB 5.5 to MariaDB 10.0.
+
+**Important**: Upgrading to a new version is always risky and users are expected to make a full
+back-up of all data before.
+
+A safer solution to upgrade is to dump all data using `mysqldump` or `mysqldbexport` and then
+load the data using `mysql` or `mysqldbimport` into an empty (freshly initialized) database.
+
+Another way of proceeding with the upgrade is starting the new version of the `mysqld` daemon
+and run `mysql_upgrade` right after the start. This so called in-place upgrade is generally
+faster for large data directory, but only possible if upgrading from the very previous version,
+so skipping versions is not supported.
+
+This container detects whether the data needs to be upgraded using `mysql_upgrade` and
+we can control it by setting `MYSQL_DATADIR_ACTION` variable, which can have one or more of the following values:
+
+ * `upgrade-warn` -- If the data version can be determined and the data come from a different version
+   of the daemon, a warning is printed but the container starts. This is the default value.
+   Since historically the version file `mysql_upgrade_info` was not created, when using this option,
+   the version file is created if not exist, but no `mysql_upgrade` will be called.
+   However, this automatic creation will be removed after few months, since the version should be
+   created on most deployments at that point.
+ * `upgrade-auto` -- `mysql_upgrade` is run at the beginning of the container start, when the local
+   daemon is running, but only if the data version can be determined and the data come
+   with the very previous version. A warning is printed if the data come from even older
+   or newer version. This value effectively enables automatic upgrades,
+   but it is always risky and users should still back-up all the data before starting the newer container.
+   Set this option only if you have very good back-ups at any moment and you are fine to fail-over
+   from the back-up.
+ * `upgrade-force` -- `mysql_upgrade --force` is run at the beginning of the container start, when the local
+   daemon is running, no matter what version of the daemon the data come from.
+   This is also the way to create the missing version file `mysql_upgrade_info` if not present
+   in the root of the data directory; this file holds information about the version of the data.
+
+There are also some other actions that you may want to run at the beginning of the container start,
+when the local daemon is running, no matter what version of the data is detected:
+
+ * `optimize` -- runs `mysqlcheck --optimize`. It optimizes all the tables.
+ * `analyze` -- runs `mysqlcheck --analyze`. It analyzes all the tables.
+ * `disable` -- nothing is done regarding data directory version.
+
+Multiple values are separated by comma and run in-order, e.g. `MYSQL_DATADIR_ACTION="optimize,analyze"`.
+
+
 Changing the replication binlog_format
 --------------------------------------
 Some applications may wish to use `row` binlog_formats (for example, those built

10.1/Dockerfile.fedora

@@ -42,7 +42,7 @@ EXPOSE 3306
 # This image must forever use UID 27 for mysql user so our volumes are
 # safe in the future. This should *never* change, the last test is there
 # to make sure of that.
-RUN INSTALL_PKGS="rsync tar gettext hostname bind-utils groff-base shadow-utils mariadb-server policycoreutils" && \
+RUN INSTALL_PKGS="rsync tar gettext hostname bind-utils groff-base shadow-utils mariadb mariadb-server policycoreutils" && \
     dnf install -y --setopt=tsflags=nodocs $INSTALL_PKGS && \
     rpm -V $INSTALL_PKGS && \
     dnf clean all && \

10.1/root/usr/share/container-scripts/mysql/README.md

@@ -260,6 +260,60 @@ Such a directory `sslapp` can then be mounted into the container with -v,
 or a new container image can be built using s2i.
 
 
+Upgrading and data directory version checking
+---------------------------------------------
+
+MySQL and MariaDB use versions that consist of three numbers X.Y.Z (e.g. 5.6.23).
+For version changes in Z part, the server's binary data format stays compatible and thus no
+special upgrade procedure is needed. For upgrades from X.Y to X.Y+1, consider doing manual
+steps as described at
+https://mariadb.com/kb/en/library/upgrading-from-mariadb-100-to-mariadb-101/
+
+Skipping versions like from X.Y to X.Y+2 or downgrading to lower version is not supported;
+the only exception is ugrading from MariaDB 5.5 to MariaDB 10.0.
+
+**Important**: Upgrading to a new version is always risky and users are expected to make a full
+back-up of all data before.
+
+A safer solution to upgrade is to dump all data using `mysqldump` or `mysqldbexport` and then
+load the data using `mysql` or `mysqldbimport` into an empty (freshly initialized) database.
+
+Another way of proceeding with the upgrade is starting the new version of the `mysqld` daemon
+and run `mysql_upgrade` right after the start. This so called in-place upgrade is generally
+faster for large data directory, but only possible if upgrading from the very previous version,
+so skipping versions is not supported.
+
+This container detects whether the data needs to be upgraded using `mysql_upgrade` and
+we can control it by setting `MYSQL_DATADIR_ACTION` variable, which can have one or more of the following values:
+
+ * `upgrade-warn` -- If the data version can be determined and the data come from a different version
+   of the daemon, a warning is printed but the container starts. This is the default value.
+   Since historically the version file `mysql_upgrade_info` was not created, when using this option,
+   the version file is created if not exist, but no `mysql_upgrade` will be called.
+   However, this automatic creation will be removed after few months, since the version should be
+   created on most deployments at that point.
+ * `upgrade-auto` -- `mysql_upgrade` is run at the beginning of the container start, when the local
+   daemon is running, but only if the data version can be determined and the data come
+   with the very previous version. A warning is printed if the data come from even older
+   or newer version. This value effectively enables automatic upgrades,
+   but it is always risky and users should still back-up all the data before starting the newer container.
+   Set this option only if you have very good back-ups at any moment and you are fine to fail-over
+   from the back-up.
+ * `upgrade-force` -- `mysql_upgrade --force` is run at the beginning of the container start, when the local
+   daemon is running, no matter what version of the daemon the data come from.
+   This is also the way to create the missing version file `mysql_upgrade_info` if not present
+   in the root of the data directory; this file holds information about the version of the data.
+
+There are also some other actions that you may want to run at the beginning of the container start,
+when the local daemon is running, no matter what version of the data is detected:
+
+ * `optimize` -- runs `mysqlcheck --optimize`. It optimizes all the tables.
+ * `analyze` -- runs `mysqlcheck --analyze`. It analyzes all the tables.
+ * `disable` -- nothing is done regarding data directory version.
+
+Multiple values are separated by comma and run in-order, e.g. `MYSQL_DATADIR_ACTION="optimize,analyze"`.
+
+
 Changing the replication binlog_format
 --------------------------------------
 Some applications may wish to use `row` binlog_formats (for example, those built

10.2/Dockerfile.fedora

@@ -1,4 +1,4 @@
-FROM registry.fedoraproject.org/f26/s2i-core:latest
+FROM registry.fedoraproject.org/f27/s2i-core:latest
 
 # MariaDB image for OpenShift.
 #
@@ -42,7 +42,7 @@ EXPOSE 3306
 # This image must forever use UID 27 for mysql user so our volumes are
 # safe in the future. This should *never* change, the last test is there
 # to make sure of that.
-RUN INSTALL_PKGS="rsync tar gettext hostname bind-utils groff-base shadow-utils mariadb-server policycoreutils" && \
+RUN INSTALL_PKGS="rsync tar gettext hostname bind-utils groff-base shadow-utils mariadb mariadb-server policycoreutils" && \
     dnf install -y --setopt=tsflags=nodocs $INSTALL_PKGS && \
     rpm -V $INSTALL_PKGS && \
     dnf clean all && \

10.2/root/usr/share/container-scripts/mysql/README.md

@@ -260,6 +260,60 @@ Such a directory `sslapp` can then be mounted into the container with -v,
 or a new container image can be built using s2i.
 
 
+Upgrading and data directory version checking
+---------------------------------------------
+
+MySQL and MariaDB use versions that consist of three numbers X.Y.Z (e.g. 5.6.23).
+For version changes in Z part, the server's binary data format stays compatible and thus no
+special upgrade procedure is needed. For upgrades from X.Y to X.Y+1, consider doing manual
+steps as described at
+https://mariadb.com/kb/en/library/upgrading-from-mariadb-101-to-mariadb-102/
+
+Skipping versions like from X.Y to X.Y+2 or downgrading to lower version is not supported;
+the only exception is ugrading from MariaDB 5.5 to MariaDB 10.0.
+
+**Important**: Upgrading to a new version is always risky and users are expected to make a full
+back-up of all data before.
+
+A safer solution to upgrade is to dump all data using `mysqldump` or `mysqldbexport` and then
+load the data using `mysql` or `mysqldbimport` into an empty (freshly initialized) database.
+
+Another way of proceeding with the upgrade is starting the new version of the `mysqld` daemon
+and run `mysql_upgrade` right after the start. This so called in-place upgrade is generally
+faster for large data directory, but only possible if upgrading from the very previous version,
+so skipping versions is not supported.
+
+This container detects whether the data needs to be upgraded using `mysql_upgrade` and
+we can control it by setting `MYSQL_DATADIR_ACTION` variable, which can have one or more of the following values:
+
+ * `upgrade-warn` -- If the data version can be determined and the data come from a different version
+   of the daemon, a warning is printed but the container starts. This is the default value.
+   Since historically the version file `mysql_upgrade_info` was not created, when using this option,
+   the version file is created if not exist, but no `mysql_upgrade` will be called.
+   However, this automatic creation will be removed after few months, since the version should be
+   created on most deployments at that point.
+ * `upgrade-auto` -- `mysql_upgrade` is run at the beginning of the container start, when the local
+   daemon is running, but only if the data version can be determined and the data come
+   with the very previous version. A warning is printed if the data come from even older
+   or newer version. This value effectively enables automatic upgrades,
+   but it is always risky and users should still back-up all the data before starting the newer container.
+   Set this option only if you have very good back-ups at any moment and you are fine to fail-over
+   from the back-up.
+ * `upgrade-force` -- `mysql_upgrade --force` is run at the beginning of the container start, when the local
+   daemon is running, no matter what version of the daemon the data come from.
+   This is also the way to create the missing version file `mysql_upgrade_info` if not present
+   in the root of the data directory; this file holds information about the version of the data.
+
+There are also some other actions that you may want to run at the beginning of the container start,
+when the local daemon is running, no matter what version of the data is detected:
+
+ * `optimize` -- runs `mysqlcheck --optimize`. It optimizes all the tables.
+ * `analyze` -- runs `mysqlcheck --analyze`. It analyzes all the tables.
+ * `disable` -- nothing is done regarding data directory version.
+
+Multiple values are separated by comma and run in-order, e.g. `MYSQL_DATADIR_ACTION="optimize,analyze"`.
+
+
 Changing the replication binlog_format
 --------------------------------------
 Some applications may wish to use `row` binlog_formats (for example, those built

root-common/usr/bin/run-mysqld

@@ -3,6 +3,9 @@
 export_vars=$(cgroup-limits); export $export_vars
 source ${CONTAINER_SCRIPTS_PATH}/common.sh
 set -eu
+if [[ -v DEBUG_IGNORE_SCRIPT_FAILURES ]]; then
+  set +e
+fi
 
 export_setting_variables
 

root-common/usr/bin/run-mysqld-master

@@ -2,9 +2,13 @@
 #
 # This is an entrypoint that runs the MySQL server in the 'master' mode.
 #
+
 export_vars=$(cgroup-limits); export $export_vars
 source ${CONTAINER_SCRIPTS_PATH}/common.sh
 set -eu
+if [[ -v DEBUG_IGNORE_SCRIPT_FAILURES ]]; then
+  set +e
+fi
 
 export_setting_variables
 

root-common/usr/bin/run-mysqld-slave

@@ -2,9 +2,13 @@
 #
 # This is an entrypoint that runs the MySQL server in the 'slave' mode.
 #
+
 export_vars=$(cgroup-limits); export $export_vars
 source ${CONTAINER_SCRIPTS_PATH}/common.sh
 set -eu
+if [[ -v DEBUG_IGNORE_SCRIPT_FAILURES ]]; then
+  set +e
+fi
 
 export_setting_variables
 
@@ -51,6 +55,7 @@ process_extending_files ${APP_DATA}/mysql-init/ ${CONTAINER_SCRIPTS_PATH}/init/
 
 # Restart the MySQL server with public IP bindings
 shutdown_local_mysql
+
 unset_env_vars
 log_volume_info $MYSQL_DATADIR
 log_info 'Running final exec -- Only MySQL server logs after this point'

root-common/usr/share/container-scripts/mysql/common.sh

@@ -39,6 +39,7 @@ function export_setting_variables() {
     export MYSQL_INNODB_LOG_FILE_SIZE=${MYSQL_INNODB_LOG_FILE_SIZE:-$((MEMORY_LIMIT_IN_BYTES*15/1024/1024/100))M}
     export MYSQL_INNODB_LOG_BUFFER_SIZE=${MYSQL_INNODB_LOG_BUFFER_SIZE:-$((MEMORY_LIMIT_IN_BYTES*15/1024/1024/100))M}
   fi
+  export MYSQL_DATADIR_ACTION=${MYSQL_DATADIR_ACTION:-upgrade-warn}
 }
 
 # this stores whether the database was initialized from empty datadir
@@ -63,9 +64,9 @@ function unset_env_vars() {
 function wait_for_mysql() {
   pid=$1 ; shift
 
-  while [ true ]; do
+  while true; do
     if [ -d "/proc/$pid" ]; then
-      mysqladmin --socket=/tmp/mysql.sock ping &>/dev/null && log_info "MySQL started successfully" && return 0
+      mysqladmin $admin_flags ping &>/dev/null && log_info "MySQL started successfully" && return 0
     else
       return 1
     fi
@@ -99,6 +100,11 @@ function initialize_database() {
   mysql_install_db --rpm --datadir=$MYSQL_DATADIR --basedir=''
   start_local_mysql "$@"
 
+  # Running mysql_upgrade creates the mysql_upgrade_info file in the data dir,
+  # which is necessary to detect which version of the mysqld daemon created the data.
+  # Checking empty file should not take longer than a second and one extra check should not harm.
+  mysql_upgrade ${admin_flags}
+
   if [ -v MYSQL_RUNNING_AS_SLAVE ]; then
     log_info 'Initialization finished'
     return 0
@@ -223,3 +229,58 @@ function process_extending_config_files() {
     fi
   done <<<"$(get_matched_files "$custom_dir" "$default_dir" '*.cnf' | sort -u)"
 }
+
+# Converts string version to the integer format (5.5.33 is converted to 505,
+# 10.1.23-MariaDB is converted into 1001, etc.
+function version2number() {
+  local version_major=$(echo "$1" | grep -o -e '^[0-9]*\.[0-9]*')
+  printf %d%02d ${version_major%%.*} ${version_major##*.}
+}
+
+# Converts the version in format of an integer into major.minor
+function number2version() {
+  local numver=${1}
+  echo $((numver / 100)).$((numver % 100))
+}
+
+# Prints version of the mysqld that is currently available (string)
+function mysqld_version() {
+  ${MYSQL_PREFIX}/libexec/mysqld -V | awk '{print $3}'
+}
+
+# Returns version from the daemon in integer format
+function mysqld_compat_version() {
+  version2number $(mysqld_version)
+}
+
+# Returns version from the datadir in the integer format
+function get_datadir_version() {
+  local datadir="$1"
+  local upgrade_info_file=$(get_mysql_upgrade_info_file "$datadir")
+  [ -r "$upgrade_info_file" ] || return
+  local version_text=$(cat "$upgrade_info_file" | head -n 1)
+  version2number "${version_text}"
+}
+
+# Returns name of the file in the datadir that holds version information about the data
+function get_mysql_upgrade_info_file() {
+  local datadir="$1"
+  echo "$datadir/mysql_upgrade_info"
+}
+
+# Writes version string of the daemon into mysql_upgrade_info file
+# (should be only used when the file is missing and only during limited time;
+# once most deployments include this version file, we should leave it on
+# scripts to generate the file right after initialization or when upgrading)
+function write_mysql_upgrade_info_file() {
+  local datadir="$1"
+  local version=$(mysqld_version)
+  local upgrade_info_file=$(get_mysql_upgrade_info_file "$datadir")
+  if [ -f "$datadir/mysql_upgrade_info" ] ; then
+    echo "File ${upgrade_info_file} exists, nothing is done."
+  else
+    log_info "Storing version '${version}' information into the data dir '${upgrade_info_file}'"
+    echo "${version}" > "${upgrade_info_file}"
+    mysqld_version >"$datadir/mysql_upgrade_info"
+  fi
+}

root-common/usr/share/container-scripts/mysql/helpers.sh

@@ -2,6 +2,10 @@ function log_info {
   echo "---> `date +%T`     $@"
 }
 
+function log_warn {
+  echo "---> `date +%T`     Warning: $@"
+}
+
 function log_and_run {
   log_info "Running $@"
   "$@"
@@ -21,4 +25,7 @@ function log_volume_info {
     shift
   done
   set -e
+  if [[ -v DEBUG_IGNORE_SCRIPT_FAILURES ]]; then
+    set +e
+  fi
 }