Skip to content
Stork - Storage Orchestration Runtime for Kubernetes
Branch: master
Clone or download
Latest commit 4a04e42 Jul 12, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github add pull request template Feb 24, 2019
cmd Remove storage cluster CRD and controller Jun 13, 2019
doc Update Rule CRD Jan 18, 2019
drivers Update vendor Jul 21, 2019
examples/groupvolumesnapshots/cassandra-e2e Add integration tests for group local and cloud snaps Jan 29, 2019
hack Fix codegen to use correct vendor directory Jul 27, 2018
images new logo Sep 26, 2017
pkg Update vendor Jul 21, 2019
specs Update permissions for all stork resources Jun 1, 2019
test/integration_test Add -t to test2json to add timestamps Jul 18, 2019
vendor Update vendor Jul 21, 2019
.gitignore Remove storage cluster CRD and controller Jun 13, 2019
.travis.yml Upload storkctl from master branch to master path instead of latest Apr 26, 2019
CODE_OF_CONDUCT.md Create CODE_OF_CONDUCT.md Mar 14, 2018
CONTRIBUTING.md spelling fixes, no content changes (#23) Jan 31, 2018
Dockerfile Update google cloud sdk and use python3 in container May 8, 2019
Dockerfile.cmdexecutor Add an executor CLI for running async commands in pods (#104) Jul 2, 2018
Gopkg.lock Update vendor Jul 21, 2019
Gopkg.toml Vendor update for new dependencies May 29, 2019
LICENSE initial skeleton framework Sep 6, 2017
Makefile Update staticcheck for integration test Jun 1, 2019
README.md Fixed a typo Feb 21, 2019
STYLEGUIDE.md spelling fixes, no content changes (#23) Jan 31, 2018
_config.yml Set theme jekyll-theme-cayman Dec 4, 2018
codecov.yml Add codecoverage and badges Mar 28, 2018
help-cmdexecutor.md Add an executor CLI for running async commands in pods (#104) Jul 2, 2018
help.md Add some metadata to docker container Mar 20, 2018

README.md

Build Status Go Report Card Docker Pulls Code Coverage

Stork - Storage Operator Runtime for Kubernetes

Stork is a Cloud Native storage operator runtime scheduler plugin. It translates a scheduler's orchestration decisions into someting that an external cloud native storage solution can act upon. By doing so, it extends Kubernetes with more stateful awareness of the underlying storage provider, it's capabilities and state.

Drawing

Stork is intended to allow storage operators such as Portworx, EMC-RexRay, and Kubernetes Local Storage to extend upon scheduler actions and allow for a storage-implementation specific orchestration actions around what the orchestrator is trying to do. The most basic example is when the scheduler is trying to spawn a container that is part of a pod - Stork will allow for the storage provider to specify an appropriate node on which that container needs to run such that it's data access is local to the runtime of the container. This is one of many orchestration scenarios that is adressed by this project.

Features

Hyper-Convergence

Stork can be used to co-locate pods with where their data is located. This is achieved by using a kubernetes scheduler extender. The scheduler is configured to use stork as an extender. So every time a pod is being scheduled, the scheduler will send filter and prioritize requests to stork. Stork will then check with the storage driver You can either configure the default kubernetes scheduler to communicate with stork or launch another instance of kube-scheduler.

Initializer (Experimental)

If you are not able to update the schedulerName for you applications to use stork, you can enable the app-initializer feature. This uses the Kubernetes AdmissionController Initializer feature to automatically update the scheduler to stork if your application (deployment or statefulset) is using volumes backed by the configured driver.

To enable the Initializer you need to:

Health Monitoring

Stork will monitor the health of the volume driver on the different nodes. If the volume driver on a node becomes unhealthy pods on that node using volumes from the driver will not be able to access their data. In this case stork will relocate pods on to other nodes so that they can continue running.

Volume Snapshots

Stork uses the external-storage project from kubernetes-incuabator to add support for snapshots.

Refer to Snapshots with Stork for instructions on creating and using snapshots with Stork.

Application consistent Snapshots

This feature allows you to specify pre and post rules that are run on the application pods before and after a snapshot is triggered. This allows users to perform actions like quiescing or flushing data from applications before a snapshot is taken and resume I/O after the snapshot is taken. The commands will be run in pods which are using the PVC being snapshotted.

Read Configuring application consistent snapshots for further details.

Building Stork

Stork is written in Golang. To build Stork:

# git clone git@github.com:libopenstorage/stork.git
# export DOCKER_HUB_REPO=myrepo
# export DOCKER_HUB_STORK_IMAGE=stork
# export DOCKER_HUB_STORK_TAG=latest
# make

This will create the Docker image $(DOCKER_HUB_REPO)/$(DOCKER_HUB_STORK_IMAGE):$(DOCKER_HUB_TAG).

Running Stork

Now that you have stork in a container image, you can just create a pod config for it and run it in your Kubernetes cluster. We do this via a deployment.

Create a Deployment

A Deployment manages a Replica Set which in turn manages the pods, thereby making stork resilient to failures. The deployment spec is defined in specs/stork-deployment.yaml. By default the deployment does the following

  • Uses the latest stable image of stork to start a pod. You can update the tag to use a specific version or use your own stork image.
  • Creates a service to provide an endpoint that can be used to reach the extender.
  • Creates a ConfigMap which can be used by a scheduler to communicate with stork.
  • Uses the Portworx (pxd) driver for stork.

Run Stork in your Kubernetes cluster

You can either update the default kube scheduler to use stork or start a new scheduler instance which can use stork. Once this has been deployed the scheduler can be used to schedule any pods with the added advantage that it will also try to optimize the storage requirements for the pod.

Start a new scheduler instance and configure it with Stork (recommended)

You might not always have access to your default scheduler to update it's config options. So the recommended way to start stork is to launch another instance of the scheduler and configure it to use stork

In order to run stork in your Kubernetes cluster, just create the deployment specified in the config above in a Kubernetes cluster:

# kubectl create -f stork-deployment.yaml

Verify that the stork pod is running:

# kubectl get pods --namespace=kube-system
NAME                              READY     STATUS    RESTARTS   AGE
....
stork-6dc5d66997-4rs2w            1/1       Running   1          27m
stork-6dc5d66997-fl8wr            1/1       Running   1          27m
stork-6dc5d66997-xvnbj            1/1       Running   1          27m
....

We will then start a new scheduler instance here and configure it to use stork. We will call the new scheduler 'stork'. This new scheduler instance is defined in specs/stork-scheduler.yaml. This spec starts 3 replicas of the scheduler.

You will need to update the version of kube scheduler that you want to use. This should be the same version as your kubernetes cluster. Example for Kubernetes v1.8.1 it would be:

image: gcr.io/google_containers/kube-scheduler-amd64:v1.8.1

You can deploy it by running the following command:

# kubectl create -f stork-scheduler.yaml

Verify that the scheduler pods are running:

# kubectl get pods --namespace=kube-system
NAME                              READY     STATUS    RESTARTS   AGE
....
stork-scheduler-9d6cb4546-gqdq2   1/1       Running   0          32m
stork-scheduler-9d6cb4546-k4z8t   1/1       Running   0          32m
stork-scheduler-9d6cb4546-tfkh4   1/1       Running   0          30m
....

Configure your default scheduler with Stork

When using stork with the default scheduler, stork needs to be run as a deamon set. This is to avoid a deadlock when trying to schedule the stork pods from the scheduler.

First create the stork daemonset defined in specs/stork-daemonset.yaml

# kubectl create -f stork-daemonset.yaml

Verify that the stork pod is running:

# kubectl get pods --namespace=kube-system
NAME                              READY     STATUS    RESTARTS   AGE
....
stork-6dc5d66997-4rs2w            1/1       Running   1          27m
stork-6dc5d66997-fl8wr            1/1       Running   1          27m
stork-6dc5d66997-xvnbj            1/1       Running   1          27m
....

To configure your default scheduler to use stork add the following arguments to the scheduler and restart the scheduler if required:

--policy-configmap=stork-config --policy-configmap-namespace=kube-system

You will also need to make sure that the kube-scheduler clusterrole has permissions to read config maps. If not, run the following command:

# kubectl edit clusterrole -n kube-system system:kube-scheduler

And add the following permissions:

- apiGroups: ['']
  resources: ['configmaps']
  verbs: ['get']

Specify the Stork scheduler for pods

In order to schedule a given pod using the Stork scheduler, specify the name of the scheduler in that pod spec:

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
   name: mysql-data
   annotations:
     volume.beta.kubernetes.io/storage-class: px-mysql-sc
spec:
   accessModes:
     - ReadWriteOnce
   resources:
     requests:
       storage: 2Gi
---
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
    name: px-mysql-sc
provisioner: kubernetes.io/portworx-volume
parameters:
   repl: "2"
---
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: mysql
spec:
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
    type: RollingUpdate
  replicas: 1
  template:
    metadata:
      labels:
        app: mysql
        version: "1"
    spec:
      schedulerName: stork
      containers:
      - image: mysql:5.6
        name: mysql
        env:
        - name: MYSQL_ROOT_PASSWORD
          value: password
        ports:
        - containerPort: 3306
        volumeMounts:
        - name: mysql-persistent-storage
          mountPath: /var/lib/mysql
      volumes:
      - name: mysql-persistent-storage
        persistentVolumeClaim:
          claimName: mysql-data

The above spec will create a mysql pod with a Portworx volume having 2 replicas. The pod will then get scheduled on a node in the cluster where one of the replicas is located. If one of those nodes does not have enough cpu or memory resources then it will get scheduled on any other node in the cluster where the driver (in this case Portworx) is running.

You can’t perform that action at this time.