Python interface to Xenon 2.0
Python interface to the Xenon middleware library, v. 2.0. Xenon provides a simple programming interface to various pieces of software that can be used to access distributed compute and storage resources.
Clone this repository, and do:
pip install .
The code will appear on PyPI when it is ready for release.
The compiled documentation is hosted on Read the Docs. This includes a quick-start guide.
PyXenon ships with the Xenon-GRPC jar-file and command-line executable. If
these need upgrading, build them manually, following instructions at
Xenon-GRPC, and place the contents of the
build/install/xenon-grpc-shadow folder (
To generate the GRPC code, run
scripts/protoc.sh from the project root.
Unit tests all run against the local scheduler and the file adaptor for filesystems. To run them, just do:
$ pytest ./tests
For faster testing it may be useful to start the
manually; start it in a separate terminal as it may give useful output for
For integration testing, run the following docker container to test against remote slurm
docker run --detach --publish 10022:22 nlesc/xenon-slurm:17
An example of some code running against this container is in
import xenon from pathlib import Path import os xenon.init() # create a new job scheduler, using SSH to localhost to submit new jobs. with xenon.Scheduler.create( adaptor='ssh', location='localhost') as scheduler: # make a new job description. The executable must already be present on # the target host. target = Path('.') / 'stdout.txt' desc = xenon.JobDescription( executable='hostname', stdout=str(target.resolve())) # submit a job job = scheduler.submit_batch_job(desc) status = scheduler.wait_until_done(job, 1000) # read the standard output of the job. We can do this directly because # we ran on localhost, otherwise, we need to transfer the file first. with open(target) as f: print(f.read())
Contributions can be made using GitHub pull requests. To add a feature, first install the test requirements
pip install -U tox
and then run
until all tests succeed. The command checks against flake8 code standards and syntax errors on Python 3.5 and 3.6. Then commit, to make sure the change didn't break any code. The pull request will be evaluated in Travis.