Skip to content
develop
Switch branches/tags
Go to file
Code

Latest commit

DEV-506: fix blocking issue on failed segment retry

When a worker fails to write its segment, the worker
would immediately exit, causing the main process to block
as it is waiting for all the child processes to read from the
work queue a final message.
e84ad87

Git stats

Files

Permalink
Failed to load latest commit information.

Codacy Badge Python 3 Updates pre-commit


GDC Data Transfer Tool (gdc-client)

Overview of the GDC Data Transfer Tool

The gdc-client provides several convenience functions over the GDC API which provides general download/upload via HTTPS.

Building the gdc-client

There is a bash script inside the ./bin directory of this repository named package that does most of the heavy lifting for building a single executable file of the gdc-client through PyInstaller. It will attempt to guess your operating system, based on uname, and build accordingly.

Building on Windows requires the installation of git and the use of the git-shell that comes bundled with it. This will provide enough Unix-like utility needed to run the bash script in this repository.

Instructions

# The script is currently location-dependant, so navigate to the bin directory.
cd bin
# Then just execute the package script. The result will be a zip file containing your executable.
./package

Executing unit tests

First install the Python package from source locally

virtualenv venv
source venv/bin/activate
pip install -r requirements.txt
python setup.py install
pip install -r dev-requirements.txt

Run tests

  • python -m pytest tests/

Run tests with coverage:

  • python -m pytest --cov=gdc_client --cov-branch --cov-report term tests/

Install pre-commit

This repository makes use of pre-commit for code formatting and secrets detecting. In order to make use of it, run the following command:

pip install -r dev-requirements.txt
pre-commit install

Note: This requires your dev environment to have Python 3.6 or higher.

Update secrets baseline for detect-secrets

We use detect-secrets to search for secrets being committed into the repo.

To update the .secrets.baseline file run

detect-secrets scan --update .secrets.baseline

.secrets.baseline contains all the string that were caught by detect-secrets but are not stored in plain text. Audit the baseline to view the secrets .

detect-secrets audit .secrets.baseline

Contributing

Read how to contribute here