☁️ 🔒 💪
kubeaudit is a command line tool to audit Kubernetes clusters for various
different security concerns: run the container as a non-root user, use a read
only root filesystem, drop scary capabilities, don't add new ones, don't run
privileged, ... You get the gist of it and more on that later. Just know:
kubeaudit makes sure you deploy secure containers!
Download a binary
Master will have newer features than the stable releases. If you need a newer feature not yet included in a release you can do the following to get kubeaudit:
go get -v github.com/Shopify/kubeaudit make make install
Now you can just call
kubeaudit with one of commands from here
Prerequisite: kubectl v1.12.0 or later
With kubectl v1.12.0 introducing easy pluggability of external functions, kubeaudit can be invoked as
kubectl audit just by
make pluginand having $GOPATH/bin available in your path.
- renaming the binary to
kubectl-auditand having it available in your path.
kubeaudit has three different modes for its audits:
- Cluster mode
If kubeaudit detects that it's running in a container,
kubeaudit cmdwill attempt to audit the cluster it's running in.
- Local config mode
If kubeaudit is not running in a container,
kubeaudit cmdwill audit the resources specified by your local kubeconfig (
$HOME/.kube/config) file. You can force kubeaudit to use a specific local config file with the switch
- Manifest mode
If you wish to audit a manifest file, use the command
kubeaudit -f/--manifest /path/to/manifest.yml
kubeaudit supports two different output types:
- just running
kubeauditwill log human readable output
- if run with
-j/--jsonit will log output json formatted so that its output can be used by other programs easily
kubeaudit has four different log levels
INFO, WARN, ERROR controlled by
-v/--verbose LEVEL and for those who counted and want to work on
- by default the debug level is set to
ERRORand will log
- if you only care about
ERRORset it to
- if you care about
WARNset it to
But wait! Which version am I actually running?
kubeaudit version will tell you.
I need help! Run
kubeaudit help every audit has its own help so you can run
kubeaudit help sc
Last but not least before we look at the audits:
audits against pods in all the phases (default Running Phase)
As humans we are lazy and
kubeaudit knows that so it comes with the functionality to
autofix workload manifests. Point it at your workload manifests and it will automagically fix everything so that manifests are as secure as it gets.
kubeaudit autofix -f path/to/manifest.yml
The manifest might end up a little too secure for the work it is supposed to do. If that is the case check out labels to opt out of certain checks.
kubeaudit has multiple checks:
- Audit all
- Audit security context
- Audit image
- Audit Service Accounts
- Audit network policies
- Audit resources
Runs all the above checks.
kubeaudit all ERRO RunAsNonRoot is not set, which results in root user being allowed! ERRO Default serviceAccount with token mounted. Please set automountServiceAccountToken to false WARN Privileged defaults to false, which results in non privileged, which is okay. ERRO Capability not dropped CapName=AUDIT_WRITE
Audit security contexts
The security context holds a couple of different security related
configurations. For convenience,
kubeaudit will always log the following
information when it creates a log:
kubeaudit command LOG KubeType=deployment Name=THEdeployment Namespace=deploymentNS
And for brevity, the information will not be shown in the commands below.
kubeaudit is able to check for the following fields in the security context:
kubeaudit will detect whether
readOnlyRootFilesystem is either not set
nil or explicitly set to
kubeaudit rootfs ERRO ReadOnlyRootFilesystem not set which results in a writable rootFS, please set to true ERRO ReadOnlyRootFilesystem set to false, please set to true
kubeaudit will detect whether the container is to be run as root:
kubeaudit nonroot ERRO RunAsNonRoot is set to false (root user allowed), please set to true! ERRO RunAsNonRoot is not set, which results in root user being allowed!
kubeaudit will detect whether
allowPrivilegeEscalation is either set to
nil or explicitly set to
kubeaudit allowpe ERRO AllowPrivilegeEscalation set to true, please set to false ERRO AllowPrivilegeEscalation not set which allows privilege escalation, please set to false
kubeaudit will detect whether the container is to be run privileged:
kubeaudit priv ERRO Privileged set to true! Please change it to false!
Since we want to make sure everything is intentionally configured correctly
kubeaudit warns about
privileged not being set:
kubeaudit priv WARN Privileged defaults to false, which results in non privileged, which is okay.
Docker comes with a couple of capabilities that shouldn't be needed and
therefore should be dropped.
kubeaudit will also complain about added capabilities.
If the capabilities field doesn't exist within the security context:
kubeaudit caps ERRO Capabilities field not defined!
When capabilities were added:
kubeaudit caps ERRO Capability added CapName=NET_ADMIN
holds a list of capabilities that we recommend be dropped, change it if you
want to keep some of the capabilities otherwise
kubeaudit will complain about
them not being dropped:
kubeaudit caps ERRO Capability not dropped CapName=AUDIT_WRITE
Audit container image tags
kubeaudit can check for image names and image tags:
If the image tag is incorrect an ERROR will issued
kubeaudit image -i gcr.io/google_containers/echoserver:1.7 ERRO Image tag was incorrect
If the image doesn't have a tag but an image of the name was found a WARNING will be created:
kubeaudit image -i gcr.io/google_containers/echoserver:1.7 WARN Image tag was missing
If the image was found with correct tag
kubeauditnotifies with an INFO message:
kubeaudit image -i gcr.io/google_containers/echoserver:1.7 INFO Image tag was correct
Audit Service Accounts
It audits against the following scenarios:
A default serviceAccount mounted with a token:
kubeaudit sat ERRO Default serviceAccount with token mounted. Please set AutomountServiceAccountToken to false
A deprecated service account:
kubeaudit sat WARN serviceAccount is a deprecated alias for ServiceAccountName, use that one instead DSA=DeprecatedServiceAccount
Audit network policies
kubeaudit np WARN Default allow mode on test/testing
Audit resources limits
It checks that every resource has a CPU and memory limit. See Kubernetes Resource Quotas for more information:
kubeaudit limits WARN CPU limit not set, please set it! WARN Memory limit not set, please set it!
kubeaudit can check the limits not to be exceeded.
kubeaudit limits --cpu 500m --memory 125Mi WARN CPU limit exceeded, it is set to 1 but it must not exceed 500m. Please adjust it! ! WARN Memory limit exceeded, it is set to 512Mi but it must not exceed 125Mi. Please adjust it!
It checks that AppArmor is enabled for all containers by making sure the following annotation exists on the pod. There must be an annotation for each container in the pod:
container.apparmor.security.beta.kubernetes.io/<container name>: <profile>
where profile can be "runtime/default" or start with "localhost/" to be considered valid.
If the AppArmor annotation is missing:
kubeaudit apparmor ERRO AppArmor annotation missing. Container=myContainer KubeType=pod Name=myPod Namespace=myNamespace
When AppArmor annotations are misconfigured:
kubeaudit apparmor ERRO AppArmor disabled. Annotation=container.apparmor.security.beta.kubernetes.io/myContainer Container=myContainer KubeType=pod Name=myPod Namespace=myNamespace Reason=badval
It checks that Seccomp is enabled for all containers by making sure one or both of the following annotations exists on the pod. If no pod annotation is used, then there must be an annotation for each container. Container annotations override the pod annotation:
# pod annotation seccomp.security.alpha.kubernetes.io/pod: <profile> # container annotation container.seccomp.security.alpha.kubernetes.io/<container name>: <profile>
where profile can be "runtime/default" or start with "localhost/" to be considered valid. "docker/default" is deprecated and will show a warning. It should be replaced with "runtime/default".
If the Seccomp annotation is missing:
kubeaudit seccomp ERRO Seccomp annotation missing. Container=myContainer KubeType=pod Name=myPod Namespace=myNamespace
When Seccomp annotations are misconfigured for a container:
kubeaudit seccomp ERRO Seccomp disabled for container. Annotation=container.seccomp.security.alpha.kubernetes.io/myContainer Container=myContainer KubeType=pod Name=myPod Namespace=myNamespace Reason=badval
When Seccomp annotations are misconfigured for a pod:
kubeaudit seccomp ERRO Seccomp disabled for pod. Annotation=seccomp.security.alpha.kubernetes.io/pod Container= KubeType=pod Name=myPod Namespace=myNamespace Reason=unconfined
Override labels give you the ability to have
kubeaudit allow certain audits to fail.
For example, if you want
kubeaudit to ignore the fact that
AllowPrivilegeEscalation was set to
true, you can add the following label:
spec: template: metadata: labels: apps: YourAppNameHere audit.kubernetes.io/allow-privilege-escalation: "YourReasonForOverrideHere"
Any label with a non-nil reason string will prevent
kubeaudit from throwing the corresponding error and issue a warning instead.
"true" (not case sensitive) will be displayed as
kubeaudit supports many labels:
allowPrivilegeEscalation to be set to
kubeaudit.allow.privilegeEscalation: "Superuser privileges needed" WARN Allowed setting AllowPrivilegeEscalation to true Reason="Superuser privileges needed"
privileged to be set to
kubeaudit.allow.privileged: "Privileged execution required" WARN Allowed setting privileged to true Reason="Privileged execution required"
Allows adding a capability or keeping one that would otherwise be dropped.
kubeaudit.allow.capability.chown: "true" WARN Capability allowed CapName=CHOWN Reason=Unspecified
kubeaudit.allow.runAsRoot: "Root privileges needed" WARN Allowed setting RunAsNonRoot to false Reason="Root privileges needed"
kubeaudit.allow.autmountServiceAccountToken: "True" WARN Allowed setting automountServiceAccountToken to true Reason=Unspecified
kubeaudit.allow.readOnlyRootFilesystemFalse: "Write permissions needed" WARN Allowed setting readOnlyRootFilesystem to false Reason="Write permissions needed"
Drop capabilities list
Allows configuring the audit against drop capabilities. Sane defaults are as follows:
# SANE DEFAULTS: capabilitiesToBeDropped: # https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities - SETPCAP #Modify process capabilities. - MKNOD #Create special files using mknod(2). - AUDIT_WRITE #Write records to kernel auditing log. - CHOWN #Make arbitrary changes to file UIDs and GIDs (see chown(2)). - NET_RAW #Use RAW and PACKET sockets. - DAC_OVERRIDE #Bypass file read, write, and execute permission checks. - FOWNER #Bypass permission checks on operations that normally require the file system UID of the process to match the UID of the file. - FSETID #Don’t clear set-user-ID and set-group-ID permission bits when a file is modified. - KILL #Bypass permission checks for sending signals. - SETGID #Make arbitrary manipulations of process GIDs and supplementary GID list. - SETUID #Make arbitrary manipulations of process UIDs. - NET_BIND_SERVICE #Bind a socket to internet domain privileged ports (port numbers less than 1024). - SYS_CHROOT #Use chroot(2), change root directory. - SETFCAP #Set file capabilities.
This can be overridden by using
-d flag and providing your own defaults in the yaml format as shown above.
- Create your own fork!
- Get the source:
go get github.com/Shopify/kubeaudit
- Go to the source:
- Add your forked repo as a fork:
git remote add fork https://github.com/you-are-awesome/kubeaudit
- Create your feature branch:
git checkout -b awesome-new-feature
- Run the tests to see everything is working as expected:
- Commit your changes:
git commit -am 'Adds awesome feature'
- Push to the branch:
git push fork
- Submit a PR