eHive is a system for running computation pipelines on distributed computing resources - clusters, farms or grids. This repository is the implementation of eHive's Meadow interface for the SGE job scheduler (Sun Grid Engine, now known as Oracle Grid Engine).
This repository is versioned the same way as eHive itself, and both checkouts are expected to be on the same branch name to function properly.
version/2.4,version/2.5, etc. are stable branchs that work with eHive's branches of the same name. These branches are stable and only receive bugfixes.masteris the development branch and follows eHive'smaster. We primarily maintain eHive, so both repos may sometimes go out of sync until we upgrade the SGE module too
The module is continuously tested under SGE 8.1.9 using a Docker image of SGE (contributions from Matthieu Muffato, Robert Syme's and Steve Moss').
You can use the Docker image ensemblorg/ensembl-hive-sge, which contains all the dependencies and checkouts.
docker run -it ensemblorg/ensembl-hive-sge # run as normal user on your machine. Will start the image as sgeuser
prove -rv ensembl-hive-sge/t # run as "sgeuser" on the image. Uses sqlite
To test your own version of the code, you can use
scripts/ensembl-hive-sge/start_test_docker.sh.
The scriptwill start a new ensemblorg/ensembl-hive-sge container with
your own copies of ensembl-hive and ensembl-hive-sge mounted.
scripts/ensembl-hive-sge/start_test_docker.sh /path/to/your/ensembl-hive /path/to/your/ensembl-hive-sge name_of_docker_image
The last argument can be skipped and defaults to ensemblorg/ensembl-hive-sge.
This module has been written in collaboration between Lel Eory (University of Edinburgh) and Javier Herrero (University College London) based on the LSF.pm module. The Docker layer has been added by Matthieu Muffato (EMBL-EBI).
eHive is maintained by the Ensembl project. We (Ensembl) are only using Platform LSF to run our computation pipelines, and can only test SGE on the Docker image indicated above. Both Lel Eory and Javier Herrero have access to a "real" SGE cluster and are better positioned to answer SGE-specific questions.
There is eHive users' mailing list for questions, suggestions, discussions and announcements. To subscribe to it please visit this link