Skip to content
GPDB Backup Utility
Go Shell Other
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
backup Add tests for materialized views Nov 11, 2019
backup_filepath
backup_history Fix warnings that were flagged by Goland IDE (#297) Oct 11, 2019
ci Set up gpadmin user when building gppkgs for ubuntu Nov 12, 2019
end_to_end Add tests for materialized views Nov 11, 2019
gppkg
helper Do not send sigterm to gpbackup_helpers on success Sep 11, 2019
integration Add tests for materialized views Nov 11, 2019
options Sort package imports (#316) Nov 7, 2019
plugins Enhance error logging in plugin test bench Aug 23, 2019
restore Sort package imports (#316) Nov 7, 2019
testutils Add tests for materialized views Nov 11, 2019
utils
.gitignore Don't place cross compiled binaries into $GOPATH/bin May 29, 2018
Gopkg.lock Fix make targets and reuse current scripts for the ubuntu job (#318) Nov 11, 2019
Gopkg.toml
LICENSE Update LICENSE and NOTICE file Feb 6, 2019
Makefile Fix make targets and reuse current scripts for the ubuntu job (#318) Nov 11, 2019
NOTICE Add copyright symbol and date range to NOTICE file Feb 6, 2019
README.md goimports related updates Jul 31, 2019
gometalinter.config Separate history into its own package Nov 9, 2018
gpbackup.go Fix warnings that were flagged by Goland IDE (#297) Oct 11, 2019
gpbackup_helper.go Add support for backing up to and restoring from a single data file Nov 20, 2017
gprestore.go
show_coverage.sh Add "coverage" target in makefile to display code coverage percentages Aug 11, 2017

README.md

Greenplum Backup

gpbackup and gprestore are Go utilities for performing Greenplum Database backups. They are still currently in active development.

Pre-Requisites

The project requires the Go Programming language version 1.8 or higher. Follow the directions here for installation, usage and configuration instructions.

Downloading

go get github.com/greenplum-db/gpbackup/...

This will place the code in $GOPATH/github.com/greenplum-db/gpbackup.

Building and installing binaries

Make the gpbackup directory your current working directory and run:

make depend
make build

The build target will put the gpbackup and gprestore binaries in $HOME/go/bin.

This will also attempt to copy gpbackup_helper to the greenplum segments (retrieving hostnames from gp_segment_configuration). Pay attention to the output as it will indicate whether this operation was successful.

make build_linux and make build_mac are for cross compiling between macOS and Linux

make install_helper will scp the gpbackup_helper binary (used with -single-data-file flag) to all hosts

Validation and code quality

Test setup

Required for Greenplum Database 6 or higher, several tests require the dummy_seclabel Greenplum contrib module. This module exists only to support regression testing of the SECURITY LABEL statement. It is not intended to be used in production. Use the following commands to install the module.

pushd ~/workspace/gpdb/contrib/dummy_seclabel
    make install
    gpconfig -c shared_preload_libraries -v dummy_seclabel
    gpstop -ra
    gpconfig -s shared_preload_libraries | grep dummy_seclabel
popd

Test execution

NOTE: The integration and end_to_end tests require a running Greenplum Database instance.

To run all tests except end-to-end (linters, unit, and integration), use

make test

To run only unit tests, use

make unit

To run only integration tests (requires a running GPDB instance), use

make integration

To run end to end tests (requires a running GPDB instance), use

make end_to_end

We provide the following targets to help developers ensure their code fits Go standard formatting guidelines.

To run a linting tool that checks for basic coding errors, use

make lint

This target runs gometalinter.

Note: The lint target will fail if code is not formatted properly.

To automatically format your code and add/remove imports, use

make format

This target runs goimports and gofmt. We will only accept code that has been formatted using this target or an equivalent gofmt call.

Running the utilities

The basic command for gpbackup is

gpbackup --dbname <your_db_name>

The basic command for gprestore is

gprestore --timestamp <YYYYMMDDHHMMSS>

Run --help with either command for a complete list of options.

Cleaning up

To remove the compiled binaries and other generated files, run

make clean

More Information

The Greenplum Backup wiki for this project has several articles providing a more in-depth explanation of certain aspects of gpbackup and gprestore.

How to Contribute

We accept contributions via Github Pull requests only.

Follow the steps below to contribute to gpbackup:

  1. Fork the project’s repository.
  2. Run go get github.com/greenplum-db/gpbackup/... and add your fork as a remote.
  3. Run make depend to install required dependencies
  4. Create your own feature branch (e.g. git checkout -b gpbackup_branch) and make changes on this branch.
    • Follow the previous sections on this page to setup and build in your environment.
    • Add new tests to cover your code. We use Ginkgo and Gomega for testing.
  5. Run make format, make test, and make end_to_end in your feature branch and ensure they are successful.
  6. Push your local branch to the fork (e.g. git push <your_fork> gpbackup_branch) and submit a pull request.

Your contribution will be analyzed for product fit and engineering quality prior to merging. Note: All contributions must be sent using GitHub Pull Requests.

Your pull request is much more likely to be accepted if it is small and focused with a clear message that conveys the intent of your change.

Overall we follow GPDB's comprehensive contribution policy. Please refer to it here for details.

Code Formatting

We use goimports to format go code. See https://godoc.org/golang.org/x/tools/cmd/goimports The following command formats the gpbackup codebase excluding the vendor directory and also lists the files updated.

goimports -w -l $(find . -type f -name '*.go' -not -path "./vendor/*")

Troubleshooting

Dummy Security Label module is not installed or configured

If you see errors in many integration tests (below), review the Validation and code quality [Test setup](#Test setup) section above:

SECURITY LABEL FOR dummy ON TYPE public.testtype IS 'unclassified';
      Expected
          <pgx.PgError>: {
              Severity: "ERROR",
              Code: "22023",
              Message: "security label provider \"dummy\" is not loaded",

Tablespace already exists

If you see errors indicating the test_tablespace tablespace already exists (below), execute psql postgres -c 'DROP TABLESPACE test_tablespace' to cleanup the environment and rerun the tests.

    CREATE TABLESPACE test_tablespace LOCATION '/tmp/test_dir'
    Expected
        <pgx.PgError>: {
            Severity: "ERROR",
            Code: "42710",
            Message: "tablespace \"test_tablespace\" already exists",
You can’t perform that action at this time.