k8s-node-collector

The k8s-Node-collector is an open-source collector that gathers Node information (file system and process data) from Kubernetes nodes and outputs it in a JSON format.

Installation

git clone git@github.com:aquasecurity/k8s-node-collector.git
cd k8s-node-collector/cmd/node-collector
GOOS=linux GOARCH=arm64 go build -o node-collector main.go

Executing node-collector binary

Usage:
  node-collector [flags]
  node-collector [command]

Examples:
node-collector k8s [flags]

Available Commands:
  completion  Generate the autocompletion script for the specified shell
  help        Help about any command
  k8s         k8s-node-collector extract file system info from cluster Node

Flags:
  -c, --cluster-version string   cluser version. example 1.23.0
  -h, --help                     help for node-collector
      --kubelet-config string    kubelet config via api /api/v1/nodes/<>/proxy/configz encoded in base64
  -n, --node string              node name
  -o, --output string            Output format. One of table|json (default "json")
  -s, --spec-name string         spec name. example: k8s-cis
  -v, --spec-version string      spec version. example 1.23.0

./node-collector k8s

Executing Collector specifications

The node-collector executes a collector specification,example k8s-cis-1.23.0. Each specification must include:

name: any other platfrom used, example (k8s-cis, aks-cis, gke-cis and etc)
version: of the cis-benchmark it represent (example: 1.23.0)

for executing a specific spec need to pass the --spec-name k8s-cis and --spec-version 1.23.0 flags

If no collector spec has been specified. the node-collector will try to auto detect the matching spec by platform type and version as define in version_mapping data example:

k8s:
  - op: "="
    cluster_version: "1.21"
    spec: k8s-cis-1.21.0
  - op: ">"
    cluster_version: "1.21"
    spec: k8s-cis-1.23.0

you can use the cluster-version flag in case you do not know what cis spec is supported for you cluster. this option must be used in conjantion with spec-name flag and the matching spec version will be auto detected example:| --spec-name k8s-cis --cluster-version 1.23.1

In the example provided, there are two rules; the first matching rule will obtain the appropriate specification. Any native k8s cluser with version equal to 1.21 will obtain the k8s-cis-1.21.0 collector specification it no match found any native k8s cluser with version grather to 1.21 will obtain the k8s-cis-1.23.0

Adding new collector specifications

In order to add a new specifications, put a new yaml file to this path : .pkg/collector/config/specs/ with the following file naming convesion <platform-cis-spec_version> example: gke-cis-1.24.0

Each collector specification audit includes the following fields

---
version: "1.23.0"
name: aks-cis
title: Node Specification for AKS info collector
collectors:
  - key: < name to hold the audit command output>
    title: <title of the audit command>
    nodeType: <node type - master | worker>
    audit: <audit shell command>

General spec data

name - name of the spec (example: aks-cis)

version - version of the spec (example: 1.23.0)

title - short description of the overall spec

Specific audit data

key - parameter name to hold the audit shell command output

title - title of the audit shell command

nodeType - define the node type on which shell command should be executed (master | worker)

audit - a shell command that collect information and return the result (errors must be supressed)

Config file

The k8s-node-collector use a config file which help to obtain binaries and config files path based on different platfrom (rancher, native k8s and etc) for example:

kubelet:
    bins:
      - kubelet
      - hyperkube kubelet
    confs:
      - /etc/kubernetes/kubelet-config.yaml
      - /var/lib/kubelet/config.yaml

The node collector will obtain the kubelet binary name and config file path based on the platfrom it runs on. when writing an audit shell command the params from config files can be used to collect the appropriate data via config params example, collect the kubelet config.yaml configuration file ownership:

stat -c %U:%G $kubelet.confs

Run s k8s job

simple k8s cluster run following job

kubectl apply -f job.yaml

Check k8s pod status

kubectl get pods 

NAME                                     READY   STATUS      RESTARTS   AGE
node-collector-ng2z7                          0/1     Completed   0          6m13s

Check k8s pod audit output

kubectl logs node-collector-ng2z7