GalaxyCloudRunner enables bursting of user jobs to remote compute resources for the Galaxy application. It provides a dynamic job runner that can be plugged into Galaxy.
GalaxyCloudRunner enables bursting of user jobs to remote compute resources for the Galaxy application. It provides several dynamic job rules that can be plugged into Galaxy, enabling Galaxy to submit jobs to remote cloud nodes.
How it works
The GalaxyCloudRunner provides a library of rules that can be plugged
into Galaxy through
job_conf.xml. Once configured, you can get your
jobs to be automatically routed to remote Pulsar nodes running on the
cloud. The GalaxyCloudRunner will discover what Pulsar nodes are
available by querying the CloudLaunch API.
Adding a new node is a simple matter of visiting the
CloudLaunch site and launching a new
Pulsar node on your desired cloud.
Getting started with the GalaxyCloudRunner is a simple process.
- First, install galaxycloudrunner into your Galaxy's virtual
pip install galaxycloudrunner.
- Add a job rule to Galaxy which will determine the Pulsar node to route to.
- Configure your
job_conf.xmlto use this rule.
- Launch as many Pulsar nodes as you need through CloudLaunch.
- Submit jobs as usual.
For detailed instructions, see: https://galaxycloudrunner.readthedocs.io/
Clone the source code repository and install the library with the dev dependencies.
git clone https://github.com/CloudVE/galaxycloudrunner.git cd galaxycloudrunner pip install --upgrade .[dev]
To build the HTML docs locally, run the following commands. The built site will
be available in
cd docs make html
Community contributions for any part of the project are welcome. If you have a completely new idea or would like to bounce your idea before moving forward with the implementation, feel free to create an issue to start a discussion.
Contributions should come in the form of a pull request. The code needs to be well documented and all methods have docstrings. We are largely adhering to the PEP8 style guide with 80 character lines, 4-space indentation (spaces instead of tabs), explicit, one-per-line imports among others. Please keep the style consistent with the rest of the project.
- Update any dependencies in
setup.pyand commit the changes.
- Increment the library version number in
galaxycloudrunner/__init__.pyas per semver rules.
- Add release notes to
CHANGELOG.rst, adding the most recent commit hash to the changelog, and make a commit. List of commits can be obtained using
git shortlog <last release hash>..HEAD
- Test the release with PyPI staging server, as described in
You will need to
pip install -U wheel twinefor this step.
- Release to PyPI
# Remove stale files or wheel might package them rm -r build dist python setup.py sdist bdist_wheel twine upload -r pypi dist/galaxycloudrunner-*
- Tag release and make a GitHub release. Alternatively, push the current
commits and make a release directly on GitHub to include the release
changelog in the tag body.
git tag -a v0.3.0 -m "Release 0.3.0" git push git push --tags
- Increment version number in
<current-version>+devto indicate the development cycle; commit, and push the changes.