Skip to content

Upgrade metacat more intelligently #14

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Feb 6, 2020
Merged

Upgrade metacat more intelligently #14

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1772e8f
Issue #134: Pass a flag to upgrade manually
helbashandy Jan 9, 2020
5a2a503
Adds upgrade check
helbashandy Jan 23, 2020
abdf3f0
Refactor upgrade logic to exit
vchendrix Feb 6, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,14 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
python-bcrypt \
vim \
netcat \
libxml2-utils \
net-tools \
telnetd \
procps \
&& rm -rf /var/lib/apt/lists/* \
&& cp /tmp/metacat.war /tmp/metacat-index.war /usr/local/tomcat/webapps \
&& cat /tmp/catalina.properties >> /usr/local/tomcat/conf/catalina.properties


COPY apply_config.py /usr/local/bin/
RUN ln -s usr/local/bin/apply_config.py / # backwards compat

Expand Down
154 changes: 118 additions & 36 deletions docker-entrypoint.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,10 @@ if [ "$1" = 'bin/catalina.sh' ]; then
METACAT_WAR=${METACAT_DIR}.war
METACATUI_BASE_SKIN_PATH=/usr/local/tomcat/webapps/catalog/style/skins/metacatui

# The following constant defines the last version before the metacat version that supports the upgrade status
# ability in metacat node capabilities. This is used to enable the script to learn when to use the node capability
UPGRADE_STATUS_ABILITY_PRE_VERSION=2.12.2

# Expand the metacat-index.war
if [ ! -d webapps/metacat-index ];
then
Expand Down Expand Up @@ -43,14 +47,13 @@ if [ "$1" = 'bin/catalina.sh' ]; then
DEFAULT_PROPERTIES_FILE=${METACAT_DIR}/WEB-INF/metacat.properties
APP_PROPERTIES_FILE=${APP_PROPERTIES_FILE:-/config/app.properties}
METACATUI_CUSTOM_SKINS_PATH=/config/skins
METACAT_VERSION_CONFIGURED=`grep application.metacatVersion $DEFAULT_PROPERTIES_FILE | sed 's/[^:]*=//'`


# Look for the metacat ui skin directory
if [ -d ${METACATUI_CUSTOM_SKINS_PATH} ];
then



echo
echo '**********************************************************'
echo "Synchronizing skins from ${METACATUI_CUSTOM_SKINS_PATH} "
Expand Down Expand Up @@ -236,8 +239,9 @@ if [ "$1" = 'bin/catalina.sh' ]; then
echo "checking upgrade/initialization status"
echo '**************************************'
echo
sleep 10

while ! nc -z localhost 8080; do
sleep 0.1
done


# Login to Metacat Admin and start a session (cookie.txt)
Expand All @@ -248,10 +252,12 @@ if [ "$1" = 'bin/catalina.sh' ]; then
echo '**************************************'
echo

# Login to Metacat Admin and start a session (cookie.txt)
curl -v -X POST \
--data "loginAction=Login&configureType=login&processForm=true&password=${ADMINPASS}&username=${ADMIN}" \
--cookie-jar /tmp/cookie.txt http://localhost:8080/${METACAT_APP_CONTEXT}/admin > /tmp/login_result.txt 2>&1


# Test the the admin logged in successfully
[ -f /tmp/login_result.txt ] && [ $(grep "User logged in as:" /tmp/login_result.txt| wc -l) -eq 1 ] || (echo "Administrator not logged in!!" && [ ! $DEBUG -eq 1 ] && grep "<message>" /tmp/login_result.txt && exit -4)

Expand All @@ -261,52 +267,128 @@ if [ "$1" = 'bin/catalina.sh' ]; then
echo '**************************************'
echo

## If the DB needs to be updated run the migration scripts
DB_CONFIGURED=`grep "configureType=database" /tmp/login_result.txt | wc -l`
if [ $DB_CONFIGURED -ne 0 ];
then

# Run the database initialization to create or upgrade tables
# /${METACAT_APP_CONTEXT}/admin?configureType=database must have an authenticated session, then run
curl -X POST --cookie /tmp/cookie.txt \
--data "configureType=database&processForm=true" \
http://localhost:8080/${METACAT_APP_CONTEXT}/admin > /dev/null 2>&1
if [ "${MANUAL_UPGRADE}" != 1 ] ; then

# Validate the database should be configured
curl -X POST --cookie /tmp/cookie.txt \
--data "configureType=configure&processForm=false" \
http://localhost:8080/${METACAT_APP_CONTEXT}/admin > /dev/null 2>&1
## If the DB needs to be updated run the migration scripts
DB_CONFIGURED=`grep "configureType=database" /tmp/login_result.txt | wc -l`
if [ $DB_CONFIGURED -ne 0 ];
then

sleep 10
# Run the database initialization to create or upgrade tables
# /${METACAT_APP_CONTEXT}/admin?configureType=database must have an authenticated session, then run
curl -X POST --cookie /tmp/cookie.txt \
--data "configureType=database&processForm=true" \
http://localhost:8080/${METACAT_APP_CONTEXT}/admin > /dev/null 2>&1

# Validate the database should be configured
curl -X POST --cookie /tmp/cookie.txt \
--data "configureType=configure&processForm=false" \
http://localhost:8080/${METACAT_APP_CONTEXT}/admin > /dev/null 2>&1



# Check if configured version is 2.12.3 or above as it's the starting version that supports
# upgrade_status field.
if [ "$METACAT_VERSION_CONFIGURED" = "`echo -e "$METACAT_VERSION_CONFIGURED\n$UPGRADE_STATUS_ABILITY_PRE_VERSION" | \
sort --version-sort | tail -n1`" ] && \
[ "$METACAT_VERSION_CONFIGURED" != "$UPGRADE_STATUS_ABILITY_PRE_VERSION" ]; then

# Wait until the upgrade is done.
NEXT_WAIT_TIME=0
UPGRADED=0
until [ $UPGRADED -eq 1 ]; do

sleep $(( NEXT_WAIT_TIME++ ))

# Wait for total of 5 minutes (Sigma 24) seconds before timeout and
# increase wait time every time gradually.
if [ $NEXT_WAIT_TIME -eq 24 ]; then
echo "****************************************************************************"
echo "******************************** ERROR *************************************"
echo "ERROR: The metacat upgrade is taking too long."
echo " You may try to pass the MANUAL_UPGRADE variable in the docker-compose with 1 to upgrade "
echo " metacat manually"
echo "****************************************************************************"
echo "****************************************************************************"
exit -1
fi

# Get Node capabilities from metacat
NODE_CAPABILITIES_XML=`curl --insecure -X GET http://localhost:8080/catalog/d1/mn/v2/node`
UPGRADE_STATUS=`xmllint --xpath '//property[@key="upgrade_status"]/text()' - <<< $NODE_CAPABILITIES_XML`
CURRENT_METACAT_VERSION=`xmllint --xpath '//property[@key="metacat_version"]/text()' - <<< $NODE_CAPABILITIES_XML`

if [ "$UPGRADE_STATUS" = "success" ]; then
echo "***************************************************************************"
echo "************************* Upgrade successful ******************************"
echo "Metacat now finished the upgrade."
echo " Metacat upgraded version is ${CURRENT_METACAT_VERSION} "
echo "****************************************************************************"
echo "****************************************************************************"
UPGRADED=1
fi
done

else

# if this is an older version than 2.12.3, sleep for 10 seconds until the upgrade finishes
sleep 10

fi

# stop tomcat and ignore exit signal
/bin/catalina.sh stop > /dev/null 2>&1 || true

echo
echo '**************************************'
echo "Waiting for Tomcat to stop "
echo "after upgrade/initialization"
echo '**************************************'
echo

# Give time for tomcat to stop
echo
echo '**************************************'
echo "Waiting for Tomcat to stop before"
echo "restarting after upgrade/initialization"
echo '**************************************'
echo
sleep 10
# Shutdown tomcat background process taking hold of port 8080 and put it in the background
# Saving tomcat PID in a file to be read by the shutdown script
echo `ps h -C java -o "%p:%a" | grep catalina | cut -d: -f1` > /tmp/tomcat.pid
CATALINA_PID=/tmp/tomcat.pid ./bin/catalina.sh stop 120 -force > /dev/null 2>&1 &

echo
echo '**************************************'
echo "Waiting for ports 8080,8009,5701 to be released..."
echo '**************************************'

# Wait for ports to be released (metacat, hazelcast and apr )
# NOTE: This does not perfectly tell us that tomcat is down but
# it is our best guess. The catalina.sh script does not stop the
# tomcat process which makes it difficult to determine when tomcat is down.
while nc -z localhost 8080 || nc -z localhost 8009 || nc -z localhost 5701; do
sleep 0.1
done

# Start tomcat
$@ > /dev/null 2>&1
echo
echo '***********************************'
echo "Upgraded/Initialized the metacat DB"
echo " RESTART the CONTAINER "
echo '***********************************'
echo
exit
else
echo
echo '**************************************'
echo "Metacat is already configured"
echo '**************************************'
echo
fi

echo
echo '***********************************'
echo "Upgraded/Initialized the metacat DB"
echo '***********************************'
echo
else

echo
echo '**************************************'
echo "Metacat is already configured"
echo '**************************************'
echo "Metacat is configured to be upgraded manually"
echo "Please go to http://localhost:8080/${METACAT_APP_CONTEXT}/admin to upgrade metacat"
echo '**************************************'
echo '**************************************'
echo

fi

# Remove the session cookie
Expand Down