Skip to content


Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?


Failed to load latest commit information.


Maintenance PR slack

version Build Status codecov Go Report Card

Create checkpoint snapshots of the state of running pods for later off-line analysis.

kube-forensics allows a cluster administrator to dump the current state of a running pod and all its containers so that security professionals can perform off-line forensic analysis.

In the event of a security breach, members of the Security Team need to examine the state of the Pod and perform a detailed forensics analysis to determine the mode of attack. However, the business would like to terminate the Pod and get back to normal processing as quickly as possible. kube-forensics was developed to allow a cluster administrator to dump the state of a running Pod for offline analysis.

The forensics-controller-manager manages a PodCheckpoint custom resource definition (CRD). The PodCheckpoint resource runs a Kubernetes Job on the same node as the target pod and performs the equivalent of the following operations on the indicated pod/containers:

docker inspect
docker diff
docker export

In addition, it collects some meta-data about the target pod. The output is uploaded to the destination S3 bucket.


You must have cluster administrator access to deploy kube-forensics to a running cluster.

  1. Insure your KUBECONFIG and current context correctly points to the desired cluster.
  2. Checkout kube-forensics repository
  3. Change directory into the root of the repository
  4. Run make deploy

For example:

$ cd kube-forensics
$ make deploy
/Users/tekenstam/go/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
kubectl apply -f config/crd/bases configured
kustomize build config/default | kubectl apply -f -
namespace/forensics-system unchanged configured unchanged configured unchanged unchanged unchanged unchanged
service/forensics-controller-manager-metrics-service unchanged
deployment.apps/forensics-controller-manager unchanged

Usage example

Once the kube-forensics controller is installed, a PodCheckpoint spec can be submitted for processing.

Sample spec

Save the following yaml file to example.yaml and modify the destination, pod and namespace to valid values for your cluster.

kind: PodCheckpoint
  name: podcheckpoint-sample
  namespace: forensics-system
  destination: s3://my-bucket-123456789000-us-west-2
  subpath: forensics
  pod: bad-pod-1234567890-dead1
  namespace: default

Submit & Verify

$ kubectl apply -f ./config/samples/forensics_v1alpha1_podcheckpoint.yaml created

$ kubectl get -n forensics-system PodCheckpoint
NAME                   AGE
podcheckpoint-sample   33s

Check the state of the PodCheckpoint.

$ kubectl describe PodCheckpoint -n forensics-system podcheckpoint-sample
Name:         podcheckpoint-sample
Namespace:    forensics-system
Labels:       <none>
API Version:
Kind:         PodCheckpoint
  Creation Timestamp:  2019-08-14T23:19:13Z
  Generation:          2
  Resource Version:    595318
  Self Link:           /apis/
  UID:                 edbe3bd6-bee9-11e9-a5c6-0afa5b77e74c
  Destination:  s3://my-bucket-123456789000-us-west-2
  Namespace:    default
  Pod:          bad-pod-1234567890-dead1
  Subpath:      forensics
  Completion Time:  2019-08-14T23:19:13Z
    Last Probe Time:       2019-08-14T23:19:13Z
    Last Transition Time:  2019-08-14T23:19:13Z
    Message:               The specified Pod 'bad-pod-1234567890-dead1' was not found in the 'default' namespace.
    Reason:                NotFound
    Status:                True
    Type:                  Failed
  Start Time:              2019-08-14T23:19:13Z
Events:                    <none>

In the above output you can see the PodCheckpoint failed due to the Pod name not being found in the system.

Bucket Configuration

The S3 bucket indicated in the destination spec must allow the worker pod created by kube-forensics to put objects into the bucket. For example, you may use the nodes role of the cluster to provide the needed access.

    "Version": "2012-10-17",
    "Statement": [
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::<AWS_ACCOUNT>:role/nodes.<CLUSTER_NAME>.cluster.k8s.local"
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::kops-state-store-<AWS_ACCOUNT>-us-west-2/*"

Release History

  • 0.1.0
    • Release alpha version of kube-forensics


Please see

Developer Guide

Please see