WishBuilder-CLI

WishBuilder-CLI is a continuous-integration pipeline that tests and merges pull requests for WishBuilder.

Purpose

WishBuilder is an open source project that provides biology-related datasets to Geney, a Web application that makes the data easier to filter and query for research. WishBuilder allows users to submit code which gathers and reformats data pulled from public Web servers into a consistent format described on the project wiki.

When a WishBuilder researcher prepares scripts for a given dataset, the researcher submits a pull request to the WishBuilder GitHub repository. WishBuilder-CLI checks for these pull requests, executes the scripts in a Docker container, tests that the ouput data are consistent with the WishBuilder requirements, and add datasets to Geney.

Using WishBuilder-CLI

WishBuilder-CLI tests code within a Docker container. It also uses Docker Compose. You must have these tools installed to execute WishBuilder-CLI. They can be installed on many different operating systems.

The docker-compose.yaml contains configuration settings for executing WishBuilder-CLI. These settings may differ, depending on who is executing WishBuilder-CLI and where they are executing it. This repository has an example docker-compose.yaml.

After you have created a custom docker-compose.yaml file, you can begin executing WishBuilder-CLI using this command:

docker-compose up -d

This will execute WishBuilder-CLI in "detached" mode. This means that it will continue executing even after you log out. If you would like to see the program's latest output, execute the following command:

docker-compose logs

If you would like to stop executing WishBuilder-CLI, execute the following command:

docker-compose down

Below is a summary of files used by WishBuilder-CLIf

• /app/ (WB_DIRECTORY)
  • testing/ (location for downloading and testing pull requests)
  • history.sql (a SQLite database that tracks pull requests that have been processed)
  • RawDatasets/ (stores TSV files that have been generated using WishBuilder code)
  • GeneyDatasets/ (stores Parquet and config files that have been generated by WishBuilder-CLI)

Re-executing a pull request

Sometimes you may wish to re-execute a pull request after it has failed, even though the WishBuilder code hasn't changed. To do this, create a text file called prs_to_test.txt and put the number of the pull request (as listed on GitHub) on a separate line. You'll also need to stop and start WishBuilder-CLI.