Volcano is a batch system built on Kubernetes. It provides a suite of mechanisms that are commonly required by many classes of batch & elastic workload including: machine learning/deep learning, bioinformatics/genomics and other "big data" applications. These types of applications typically run on generalized domain frameworks like TensorFlow, Spark, PyTorch, MPI, etc, which Volcano integrates with.
Volcano builds upon a decade and a half of experience running a wide variety of high performance workloads at scale using several systems and platforms, combined with best-of-breed ideas and practices from the open source community.
Volcano is a sandbox project of the Cloud Native Computing Foundation (CNCF). Please consider joining the CNCF if you are an organization that wants to take an active role in supporting the growth and evolution of the cloud native ecosystem.
- Intro: Kubernetes Batch Scheduling @ KubeCon 2019 EU
- Volcano 在 Kubernetes 中运行高性能作业实践 @ ArchSummit 2019
- Volcano：基于云原生的高密计算解决方案 @ Huawei Connection 2019
- Improving Performance of Deep Learning Workloads With Volcano @ KubeCon 2019 NA
- Batch Capability of Kubernetes Intro @ KubeCon 2019 NA
Quick Start Guide
- Kubernetes 1.12+ with CRD support
You can try Volcano by one of the following two ways.
- For Kubernetes v1.16+ use CRDs under config/crd/bases (recommended)
- For Kubernetes versions < v1.16 use CRDs under config/crd/v1beta1 (deprecated)
Install with YAML files
Install Volcano on an existing Kubernetes cluster. This way is both available for x86_64 and arm64 architecture.
For x86_64: kubectl apply -f https://raw.githubusercontent.com/volcano-sh/volcano/master/installer/volcano-development.yaml For arm64: kubectl apply -f https://raw.githubusercontent.com/volcano-sh/volcano/master/installer/volcano-development-arm64.yaml
Enjoy! Volcano will create the following resources in
NAME READY STATUS RESTARTS AGE pod/volcano-admission-5bd5756f79-dnr4l 1/1 Running 0 96s pod/volcano-admission-init-4hjpx 0/1 Completed 0 96s pod/volcano-controllers-687948d9c8-nw4b4 1/1 Running 0 96s pod/volcano-scheduler-94998fc64-4z8kh 1/1 Running 0 96s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/volcano-admission-service ClusterIP 10.98.152.108 <none> 443/TCP 96s NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/volcano-admission 1/1 1 1 96s deployment.apps/volcano-controllers 1/1 1 1 96s deployment.apps/volcano-scheduler 1/1 1 1 96s NAME DESIRED CURRENT READY AGE replicaset.apps/volcano-admission-5bd5756f79 1 1 1 96s replicaset.apps/volcano-controllers-687948d9c8 1 1 1 96s replicaset.apps/volcano-scheduler-94998fc64 1 1 1 96s NAME COMPLETIONS DURATION AGE job.batch/volcano-admission-init 1/1 48s 96s
Install from code
If you don't have a kubernetes cluster, try one-click install from code base:
This way is only available for x86_64 temporarily.
Install monitoring system
If you want to get prometheus and grafana volcano dashboard after volcano installed, try following commands:
make TAG=latest generate-yaml kubectl create -f _output/release/volcano-monitoring-latest.yaml
Regular Community Meeting:
If you have any question, feel free to reach out to us in the following ways: