Backup
Silent119 edited this page May 1, 2021
·
14 revisions
The following page describes an example of a possible backup script for your GitBucket installation. Both the Windows and bash scripts need to be run on the machine running GitBucket.
Notice: This script is primarily for demonstrating what all should be included in a backup. It is not regularly updated or maintained, and as such, shouldn't be relied on in mission-critical situations.
The backup of a GitBucket installation should be consistent between the database state and the state of the git repositories.
Of course the most important to keep is probably the git repositories, but hey if you need sometime to recover it would be cool that all PRs, issues, references and so on are in synch with the backups of the repositories no?
The script minimises the time between the database backup and a clean stage of all repositories by doing the following steps:
- clone all repositories into a backup folder (creating only non already existing ones)
- update all repositories
- make a database backup
- update again all repositories
bash backup.sh [-v] GITBUCKET_HOME BACKUP_FOLDER GITBUCKET_ROOT_URL GITBUCKET_API_TOKEN
- -v: Verbose, when used the script will output more about what it is doing. Optional
- GITBUCKET_HOME: Full path to your GitBucket data folder. By default it might be
~/.gitbucket - BACKUP_FOLDER: Full path to the folder into which you would like the backup to be done
- GITBUCKET_ROOT_URL: Full root URL of your GitBucket installation. (see PR-845)
- GITBUCKET_API_TOKEN: API token, used to authenticate the database backup request
Below is a basic script that runs backup.sh in an msys2/msysgit environment (Such as the one included with the Sourcetree git client).
@ECHO OFF
CD /D %~dp0
REM Set your paths here
SET GitBucket_Path=
SET Backup_Path=
SET Gitbucket_API_Token=
SET Server_Root=
REM Use this only if msys isn't part of PATH already
REM SET PATH=C:\path\to\msys\bin;%PATH%
REM Coerce windows paths into linux ones
SET GitBucket_Path=%GitBucket_Path:\=/%
SET Backup_Path=%Backup_Path:\=/%
REM If these are on a drive other than C, change both letters in 'C:=/c'
SET GitBucket_Path=%GitBucket_Path:C:=/c%
SET Backup_Path=%Backup_Path:C:=/c%
bash backup.sh %GitBucket_Path% %Backup_Path% %Server_Root% %Gitbucket_API_Token%#!/bin/bash
DEBUG=0
if [ "$1" = "-v" ]
then
DEBUG=1
shift
fi
GITBUCKET_DATA=$1
BACKUP_FOLDER=$2
GITBUCKET_URL=$3
GITBUCKET_TOKEN=$4
GITBUCKET_DB_BACKUP_URL=$3/api/v3/plugins/database/backup
gitbucketRepositoriesFolder=$GITBUCKET_DATA/repositories
repositoriesFolderNameSize=${#gitbucketRepositoriesFolder}
repositoriesBackupFolder=$BACKUP_FOLDER/repositories
##
## trace allows to print messages on the console if -v argument (verbose) has been given to the program
##
function trace {
if [ "$DEBUG" = "1" ]
then
echo "$@"
fi
}
##
## create a git mirror clone of a repository. If the clone already exists, the operation is skipped
## arg1: source of the repository to be cloned
## arg2: full folder name of the clone
##
function createClone {
source=$1
dest=$2
if [ ! -d $dest ]
then
trace " cloning $source into $dest"
git clone --mirror $source $dest > /dev/null 2>&1
else
trace " $dest already exists, skipping git clone operation"
fi
}
##
## update a clone folder, the update is down toward the latest state of its default remote
##
function updateRepository {
currentFolder=$(pwd)
cd $1
trace " updating $1"
git remote update > /dev/null
cd $currentFolder
}
# Perform some initializations
if [ ! -d $repositoriesBackupFolder ]
then
mkdir -p $repositoriesBackupFolder > /dev/null
fi
#
# To keep integrity as its maximum possible, the database export and git backups must be done in the shortest possible timeslot.
# Thus we will:
# - clone new repositories into backup directory
# - update them all a first time, so that we gather gib updates
# - export the database
# - update all repositories a second time, this time it should be ultra-fast
#
# First let's be sure that all existing directories have a clone
# as clone operation can be heavy let's do it now
repositories=$(find $gitbucketRepositoriesFolder -name *.git -print)
echo "Starting clone process"
for fullRepositoryFolderPath in $repositories
do
repositoryFolder=${fullRepositoryFolderPath:$repositoriesFolderNameSize}
mirrorFolder=$repositoriesBackupFolder$repositoryFolder
createClone $fullRepositoryFolderPath $mirrorFolder
done;
echo "All repositories, cloned"
echo "Update repositories: phase 1"
#
# Then let's update all our clones
#
for fullRepositoryFolderPath in $repositories
do
repositoryFolder=${fullRepositoryFolderPath:$repositoriesFolderNameSize}
mirrorFolder=$repositoriesBackupFolder$repositoryFolder
updateRepository $mirrorFolder
done;
echo "Update repositories: phase 1, terminated"
#
# Export the database
#
if [ "$GITBUCKET_DB_BACKUP_URL" != "" ]
then
echo "Database backup"
echo "HTTP POST $GITBUCKET_DB_BACKUP_URL"
db_backup_zip_name=$(curl --silent --show-error --fail -X POST -H "Authorization: token $GITBUCKET_TOKEN" $GITBUCKET_DB_BACKUP_URL)
db_backup_zip=$GITBUCKET_DATA/backup/$db_backup_zip_name
echo "Copying $db_backup_zip to $BACKUP_FOLDER"
cp $db_backup_zip $BACKUP_FOLDER
else
echo "No database URL provided, skipping database backup"
fi
#
# Export the GitBucket configuration
#
echo "Configuration backup"
cp $GITBUCKET_DATA/gitbucket.conf $BACKUP_FOLDER > /dev/null
#
# Export the GitBucket data directory (avatars, ...)
#
echo "Avatars backup"
tar -cf $BACKUP_FOLDER/data.tar $GITBUCKET_DATA/data > /dev/null 2>&1
gzip -f $BACKUP_FOLDER/data.tar > /dev/null
#
# Export the gist directory
#
echo "Gist backup"
tar -cf $BACKUP_FOLDER/gist.tar $GITBUCKET_DATA/gist > /dev/null 2>&1
gzip -f $BACKUP_FOLDER/gist.tar > /dev/null
#
# Then let's do a final update
#
echo "Update repositories: phase 2"
for fullRepositoryFolderPath in $repositories
do
repositoryFolder=${fullRepositoryFolderPath:$repositoriesFolderNameSize}
mirrorFolder=$repositoriesBackupFolder$repositoryFolder
updateRepository $mirrorFolder
done;
echo "Update repositories: phase 2, terminated"
echo "Update process ended, backup available under: $BACKUP_FOLDER"