Skip to content
CWL on Kubernetes
Python Common Workflow Language Other
Branch: master
Clone or download
dleehr Merge pull request #94 from Duke-GCB/cwl-v1.1
Upgrade cwltool, runs conformance tests for cwl 1.1
Latest commit 367e35c Aug 6, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.circleci Also install wheel Feb 7, 2019
calrissian
conformance
examples Sets revsort input volume mount to readOnly Jul 22, 2019
input-data Update input-data/revsort-single-no-docker.cwl Jan 28, 2019
tests Rewrites test to use manager that tests call order Aug 6, 2019
.dockerignore Adds basic Dockerfile Feb 4, 2019
.gitignore Add script to run tests with coverage Jan 18, 2019
Dockerfile Clarify rationale for home dir permissions Mar 7, 2019
LICENSE
README.md Remove specific test count from top-level README Jul 23, 2019
requirements.txt Update base cwltool to 1.0.20190621234233, regenerate requirements.txt Aug 2, 2019
setup.py
test-coverage.sh
test.sh Script to run test Jan 9, 2019

README.md

Calrissian

CWL on Kubernetes

CircleCI

Overview

Calrissian is a CWL implementation designed to run inside a Kubernetes cluster. Its goal is to be highly efficient and scalable, taking advantage of high capacity clusters to run many steps in parallel.

Cluster Requirements

Calrissian requires a Kubernetes or Openshift/OKD cluster, configured to provision PersistentVolumes with the ReadWriteMany access mode. Kubernetes installers and cloud providers don't usually include this type of storage, so it may require additional configuration.

Calrissian has been tested with NFS using the nfs-client-provisioner and with GlusterFS using OKD Containerized GlusterFS. Many cloud providers have an NFS offering, which integrates easily using the nfs-client-provisioner.

Scalability / Resource Requirements

Calrissian is designed to issue tasks in parallel if they are independent, and thanks to Kubernetes, should be able to run very large parallel workloads.

When running calrissian, you must provide a limit the the number of CPU cores (--max-cores) and RAM megabytes (--max-ram) to use concurrently. Calrissian will use CWL ResourceRequirements to track usage and stay within the limits provided. We highly recommend using accurate ResourceRequirements in your workloads, so that they can be scheduled efficiently and are less likely to be terminated or refused by the cluster.

CWL Conformance

Calrissian leverages cwltool heavily and most conformance tests for CWL v1.0. Please see conformance for further details and processes.

To view open issues related to conformance, see the conformance label on the issue tracker.

Setup

Please see examples for installation and setup instructions.

You can’t perform that action at this time.