Development

If you want to help out with development, this page will describe how you can contribute.

Project structure

.github/scripts/                      # contains scripts related to continuous integration/deployment
.husky/                               # contains hook scripts to prevent user mistakes when committing/pushing
docker/                               # contains dockerfiles and compose files
lib/                                  # contains most of the actual code
  backup/                             # contains the functions used by the backup.sh script
  common/                             # contains most of the functions used by the different scripts
  entrypoint/                         # contains the functions used by the entrypoint.sh script
  prune/                              # contains the functions used by the prune.sh script
  store/                              # contains the functions used by the store.sh script
tests/                                # contains the testing suite
  specs/                              # contains the test specifications (ordered by tested functions mostly)
samples/                              # contains docker-compose files to demonstrate functionality

Here are some key files you need to take note of:

.github/scripts/test.sh               # the script runner (called by run-tests.sh)
docker/docker-compose.test.yml        # the compose file used for running tests
docker/docker-compose.test.alpine.yml # the compose file used for running tests
docker/docker-compose.yml             # the compose file used for trying out new functionality
docker/Dockerfile                     # the Dockerfile used for building images and running tests
lib/common/_common.sh                 # functions that are used by most functions/scripts
lib/common/_constants.sh              # useful constants that can be used in e.g. time calculations
lib/entrypoint/cronjobs.sh            # this is where cron jobs are setup during startup
lib/entrypoint/permissions.sh         # this is where we verify we have the correct permissions
lib/entrypoint/settings.sh            # this is where we list settings on startup -- update if you add more settings
tests/assertions.sh                   # test assertions -- update if you need more assertions
tests/common.sh                       # common test functionality
tests/run-tests.sh                    # entrypoint for running tests
tests/specs/__helpers.sh.             # functions that are run before each test -- update if you need more setup for all tests

When developing, you might want to create the following folders:

backups/                              # used by some test compose files for storing backups
data/                                 # used by some test compose files for storing data
lts/                                  # used by some test compose files for long-term storage

Make sure you (the main user with uid:gid of 1000:1000) have read & write access to these folders.

Install hooks

Run yarn in the project root to install the pre-commit and pre-push hooks.

These will protect you from writing bad commit messages and pushing to the wrong branches.

You are now ready for development.

Development

The development process looks a lot like this:

1. Write a new function in one of the lib/common/ files.
2. Use that function in one of the functions in the different script folders.
3. Write a test that tests that functionality and run ./run-tests.sh (specify the test spec if you want to only run your own tests).
4. Improve the code, fix bugs.
5. Write more tests.

Etc...

When you're done, you might want to do a docker-compose up in the project root to test the functionality with a live image. Tweak the settings so that your new functionality is used.

Writing tests

The recommended way to develop is to write tests for your code.

To create a test, create a new file for your function at ./tests/specs/<function>_spec.sh

The file should look something like this:

#!/bin/bash

for f in "$APP_PATH"/common/*; do . "$f"; done

test__contains_numeric_date__returns_true_if_valid_date() {
  test_begin "contains_numeric_date returns true if valid date"

  # Arrange
  backup_filename="backup-test-backup-20140517223559.tgz"
  result=false

  # Act
  contains_numeric_date "$backup_filename" && result=true

  # Assert
  assert_true "$result"
}

All files begin with the shebang (#!/bin/bash) followed by the for-loop that includes the common functions.

This test assumes that the function (contains_numeric_date) is specified in some file in lib/common/.

Assertions

Here is a list of the currently available assertions:

Assertion	Description
assert_file_exists	File exists at given path
assert_file_does_not_exist	File does not exist at given path
assert_file_ends_with	File exists and filename ends with given string
assert_file_starts_with	File exists and filename starts with given string
assert_string_starts_with	String starts with given substring
assert_string_ends_with	String ends with given substring
assert_string_contains	String contains a given substring
assert_null	Given variable is null ("")
assert_false	Given variable is false ("false")
assert_true	Given variable is true ("true")
assert_equals	Given variable is equal to given value

If you need more assertions for your tests, write new assertions in ./tests/assertions.sh.

Take inspiration from how other assertions are written.

Running tests

To run the entire test suite, navigate to the project root and run ./run-tests.sh.

This will create a fresh docker container and run all tests contained in ./tests/specs/.

If you only wish to run a certain set of tests (one spec file). Simply specify that spec as a parameter:

./run-tests.sh get_backups_spec

This will run all tests contained in ./tests/specs/get_backups_spec.sh.

If you only want to run one specific test, specify the name of that test function:

./run-tests.sh test__get_backups__does_not_list_sfv_files

This will run the test__get_backups__does_not_list_sfv_files() function in the ./tests/specs/get_backups_spec.sh file.

The test runner knows to look for defined functions when given a parameter that starts with test__ so you don't need to know where a test is located, just what the test function is called. This is useful when running the entire test suite and you get a list of failed tests at the end of the test session.

Test tips

Some advice when writing tests:

• Create one spec per function you test.
• Use the Arrange/Act/Assert test pattern.
• Use the absolut paths of built-in programs (/bin/echo /bin/rm /bin/touch etc) to avoid mistakenly using a mock function.
• Invoke a subshell for functions that will throw errors (wrap the function call in parentheses).
• Override functions that you want to mock -- it will be reset in the next test since all tests run in their own subshells.