Volcano is a batch system built on Kubernetes. It provides a suite of mechanisms that are commonly required by many classes of batch & elastic workload including: machine learning/deep learning, bioinformatics/genomics and other "big data" applications. These types of applications typically run on generalized domain frameworks like TensorFlow, Spark, PyTorch, MPI, etc, which Volcano integrates with.
Volcano builds upon a decade and a half of experience running a wide variety of high performance workloads at scale using several systems and platforms, combined with best-of-breed ideas and practices from the open source community.
- Intro: Kubernetes Batch Scheduling @ KubeCon 2019 EU
- Volcano 在 Kubernetes 中运行高性能作业实践 @ ArchSummit 2019
- Volcano：基于云原生的高密计算解决方案 @ Huawei Connection 2019
- Improving Performance of Deep Learning Workloads With Volcano @ KubeCon 2019 NA
- Session Batch Capability of Kubernetes Intro @ KubeCon 2019 NA
Quick Start Guide
- Kubernetes 1.12+ with CRD support
You can try volcano by one the following two ways.
Install with YAML files
Install volcano on a existing Kubernetes cluster.
kubectl apply -f https://raw.githubusercontent.com/volcano-sh/volcano/master/installer/volcano-development.yaml
Enjoy! Volcano will create the following resources in
NAME READY STATUS RESTARTS AGE pod/volcano-admission-5bd5756f79-dnr4l 1/1 Running 0 96s pod/volcano-admission-init-4hjpx 0/1 Completed 0 96s pod/volcano-controllers-687948d9c8-nw4b4 1/1 Running 0 96s pod/volcano-scheduler-94998fc64-4z8kh 1/1 Running 0 96s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/volcano-admission-service ClusterIP 10.98.152.108 <none> 443/TCP 96s NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/volcano-admission 1/1 1 1 96s deployment.apps/volcano-controllers 1/1 1 1 96s deployment.apps/volcano-scheduler 1/1 1 1 96s NAME DESIRED CURRENT READY AGE replicaset.apps/volcano-admission-5bd5756f79 1 1 1 96s replicaset.apps/volcano-controllers-687948d9c8 1 1 1 96s replicaset.apps/volcano-scheduler-94998fc64 1 1 1 96s NAME COMPLETIONS DURATION AGE job.batch/volcano-admission-init 1/1 48s 96s
Install from code
If you have no kubernetes cluster, try one click install from code base:
Community, discussion, contribution, and support
You can reach the maintainers of this project at:
Mailing List : https://groups.google.com/forum/#!forum/volcano-sh