Skip to content
master
Go to file
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
src
 
 
 
 
 
 
 
 
 
 
 
 

README.md

Stratocumulus

Deploy biggish cloud-based clusters.

Some usage instructions and pre-configured scripts are available at hammerlab/stratotemplate.

The Tests

The main test is in two pieces:

  • src/test/test_deployment.ml → uses the stratocumulus library to define a cluster deployment.
  • src/test/test_biokepi.ml → uses the deployed cluster to run a Biokepi workflow.

For the Biokepi submission to work Ketrew should be compiled with TLS support (opam install tls).

Cloud Requirements:

  • a “One-click NFS server”.
  • a host with gcloud installed and configured and password-less ~/.ssh/google_compute_engine ssh-keys ($GCLOUD_HOST below). (also make sure some basic utilities are there like unzip, wget, …)

The test is configured with environment variables:

# Ketrew host that has the `gcloud` tool:
export GCLOUD_HOST=ssh://MyGCloudSSHHost/tmp/KT

# (Optional) Ketrew binary for Ubuntu Xenial (Ketrew will be built if absent).
export KETREW_BIN="http://example.com/path/to/ketrew-xenial-x86_64"

# (Optional) A prefix for all the names creates (compute nodes and servers):
export NAME_PREFIX="my-first-deplyment"

# (Optional) The number of compute nodes in the cluster:
export NODES=12

# Authentication token for the Ketrew server to deploy:
export KETREW_TOKEN=kjkjdjedekj388787732edkde8ude909e39iki

# The NFS server:
export NFS_VM="stratocumulus-test-nfs-server-vm"
export NFS_PATH="/some-test-storage"

# The URL to download the GATK:
export GATK_JAR_URL="http://example.com/path/to/GenomeAnalysisTK_35.jar"

# (optional) make the 2nd test (NFS deployment) reuse an existing google disk:
export REUSE_DATA_DISK=more-persistent-disk

# (optional) ask for a Ketrewless cluster:
export WITH_KETREW_SERVER=no

Start the deployment workflow (a “deployer“ Ketrew server is assumed to be configured/running):

./deployment-test up {view,submit}

When the workflow is done and successful:

./deployment-test status

displays the URL to the WebUI of the Ketrew server; and

./deployment-test ketrew-config some-file.json

outputs a valid Ketrew client-configuration to some-file.json.

One can now run the Biokepi workflow:

KETREW_CONFIG=some-file.json ./biokepi-test go

About

DEPRECATED: we don't really maintain this any more, we use Coclobas:

Resources

License

Releases

No releases published

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •  

Languages

You can’t perform that action at this time.