Skip to content
Kubernetes GPU Mutating Webhook
Branch: master
Clone or download
Latest commit 0877900 Mar 8, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
deployment
.gitignore first commit Feb 8, 2019
Dockerfile
Gopkg.lock first commit Feb 8, 2019
Gopkg.toml
LICENSE
README.md Some fix Feb 8, 2019
build
main.go first commit Feb 8, 2019
webhook.go Fix bug Mar 8, 2019

README.md

GPU Mutating Webhook

This webhook fixes the problem that no GPU request container can see all GPUs.

Prerequisites

Kubernetes 1.9.0 or above with the admissionregistration.k8s.io/v1beta1 API enabled. Verify that by the following command:

kubectl api-versions | grep admissionregistration.k8s.io/v1beta1

The result should be:

admissionregistration.k8s.io/v1beta1

In addition, the MutatingAdmissionWebhook and ValidatingAdmissionWebhook admission controllers should be added and listed in the correct order in the admission-control flag of kube-apiserver.

Build

  1. Setup dep

    The repo uses dep as the dependency management tool for its Go codebase. Install dep by the following command:

go get -u github.com/golang/dep/cmd/dep
./build

Deploy

  1. Create namespace
kubectl apply -f deployment/namespace.yaml
  1. Create a signed cert/key pair and store it in a Kubernetes secret that will be consumed by sidecar deployment
./deployment/webhook-create-signed-cert.sh \
    --service gpu-mutating-webhook-svc \
    --secret gpu-mutating-webhook-certs \
    --namespace gpu-mutating
  1. Patch the MutatingWebhookConfiguration by set caBundle with correct value from Kubernetes cluster
cat deployment/mutatingwebhook.yaml | \
    deployment/webhook-patch-ca-bundle.sh > \
    deployment/mutatingwebhook-ca-bundle.yaml
  1. Deploy resources
kubectl apply -f deployment/deployment.yaml
kubectl apply -f deployment/service.yaml
kubectl apply -f deployment/mutatingwebhook-ca-bundle.yaml

Verify

  1. The sidecar inject webhook should be running
# kubectl get pods -n gpu-mutating
NAME                                               READY     STATUS    RESTARTS   AGE
gpu-mutating-webhook-deployment-7f55d45d64-mpkmk   1/1       Running   0          6m
# kubectl get deployment -n gpu-mutating
NAME                              DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
gpu-mutating-webhook-deployment   1         1         1            1           7m
  1. Deploy an app in Kubernetes cluster, take sleep app as an example
# cat <<EOF | kubectl create -f -
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: sleep
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: sleep
    spec:
      containers:
      - name: sleep
        image: tutum/curl
        command: ["/bin/sleep","infinity"]
        imagePullPolicy: 
EOF
  1. Verify container env injected
# kubectl get pods
NAME                                                READY     STATUS      RESTARTS   AGE
sleep-854884fd47-98xq7                              1/1       Running     0          7m
# kubectl describe pod sleep-854884fd47-98xq7 | grep NVIDIA_VISIBLE_DEVICES
     NVIDIA_VISIBLE_DEVICES:  none

LICENSE

This is created from kube-mutating-webhook-tutorial

You can’t perform that action at this time.