1. Introduction

Provide credential and session management capabilities for Kubernetes clusters.

For use with CyberArk PAM self-hosted or Privilege Cloud.

Tested with:

Software	Version
CyberArk PAM self-hosted	13.2
Kubernetes	1.28

2. CPM Plugins

Dependent Libraries

Name	Publisher	Version	Purpose	Link
BouncyCastle.Cryptography.dll	Legion of The Bouncy Castle Inc.	2.2.1.47552	Certificate operations	https://www.bouncycastle.org/csharp/
Newtonsoft.Json.dll	Newtonsoft	13.0.3.27908	Parse json data	https://www.newtonsoft.com/json
YamlDotNet.dll	Antoine Aubry	13.7.1.0	Parse yaml data	https://github.com/aaubry/YamlDotNet

2.1. KubeConfig

2.1.0. Preparing account for onboarding: Create user and certificate - example commands

Create user and bind to a role/clusterrole:

USERNAME=joe.tan
BINDINGNAME=joe.tan-clusterrolebinding
CLUSTERROLE=cluster-admin
kubectl create clusterrolebinding $BINDINGNAME --clusterrole=$CLUSTERROLE --user=$USERNAME

Create key-pair and CSR:

openssl ecparam -name secp384r1 -genkey -out $USERNAME.key
openssl req -new -key $USERNAME.key -subj "/CN=$USERNAME" -out $USERNAME.csr
openssl x509 -req -in $USERNAME.csr -CA /etc/kubernetes/pki/ca.crt -CAkey /etc/kubernetes/pki/ca.key -CAcreateserial -days 10958 -sha256 -out $USERNAME.pem

Submit CSR to Kubernetes cluster:

CSRNAME=joe.tan-csr
kubectl apply -f - <<EOF
apiVersion: certificates.k8s.io/v1
kind: CertificateSigningRequest
metadata:
  name: $CSRNAME
spec:
  request: $CSR
  signerName: kubernetes.io/kube-apiserver-client
  expirationSeconds: $CERTVALIDITY
  usages:
  - client auth
EOF

Approve the CSR:

kubectl certificate approve $CSRNAME

2.1.1. Account fields

1. Username
2. Kubernetes Cluster URL
3. Groups (comma separated)
4. Certificate Validity (Days)
5. Kubernetes Cluster Version

Note

The method provided by CPM for returning credentials to CPM requires the KeyID field to be populated

The Kubernetes Cluster Version field maps to the KeyID field

This parameter has no real effect on the account, but is required for the change and reconcile actions to work

2.1.2. Account password

2.1.2.1. Onboard

The plugin works with a Base-64 encoded kubeconfig file.

Example:

echo .kube/config | base64 -w0

Note

The kubeconfig file should only contain 1 user entry

If there are multiple user entries, only the first one will be managed

2.1.2.2. Change

The plugin creates a new user certificate with the following flow:

• Create a RSA-2048 private key-pair
• Create a CSR and submit to the Kubernetes cluster
• Approve the CSR

Note

The user principal in the kubeconfig file needs to have the appropriate permissions to submit and approve CSR for the change operation to succeed

Otherwise, use reconcile method with another kubeconfig account with the permissions

Warning

Kubernetes does not support certificate revocation

Minimize the chance of certificate abuse by minimizing the certificate validity and adjusting the change period to issue a new certificate for the current one expires

2.2. Service Account

2.2.0. Preparing account for onboarding: Create service account and token - example commands

NAMESPACE=kube-system
SERVICEACCOUNTNAME=joe.tan
CLUSTERROLE=cluster-admin
kubectl -n $NAMESPACE create serviceaccount $SERVICEACCOUNTNAME
kubectl -n $NAMESPACE create clusterrolebinding $CLUSTERROLE-binding --clusterrole=$CLUSTERROLE --serviceaccount=$NAMESPACE:$SERVICEACCOUNTNAME
kubectl -n $NAMESPACE create token $SERVICEACCOUNTNAME --duration=24h

2.2.1. Account fields

1. Service Account Name
2. Kubernetes Cluster URL
3. Namespace
4. Token Lifetime (Days)
5. Kubernetes Cluster Version

Note

The method provided by CPM for returning credentials to CPM requires the KeyID field to be populated

The Kubernetes Cluster Version field maps to the KeyID field

This parameter has no real effect on the account, but is required for the change and reconcile actions to work

2.2.2. Account password

2.2.2.1. Onboard

The plugin works with service account tokens

2.2.2.2. Change

The plugin works with Kubernetes time-limited API tokens

Example:

kubectl -n $NAMESPACE create token $SERVICEACCOUNTNAME --duration=24h

Note

The service account needs to have the appropriate permissions to create tokens for the change operation to succeed

Otherwise, use reconcile method with a kubeconfig account with the permissions

Warning

Kubernetes does not support revocation of time-limited API token

Minimize the chance of token abuse by minimizing the token lifetime and adjusting the change period to issue a new token for the current one expires

3. PSM Connector

The PSM connector for kubectl comprises of:

1. The PSMkubectlDispatcher

• AutoIT script to work with the PSM universal connector framework
• Dispatcher flow:
  • Retrieve the account values (username/service account name, kubeconfig/service account token, kubernetes cluster URL, kubectl directory)
  • Set the account values as environment variables for the kubectl wrapper
  • Launch the kubectl wrapper

2. The kubectlWrapper.

• Restricted shell inteface built using C# to limit user inputs to kubectl commands
• Wrapper flow:
  • Determine whether the password type is kubeconfig or service account token
  • Generate the kubeconfig file according to the password type
  • Enforce accepted/denied user inputs:
    • Accepted: inputs that starts with kubectl
    • Denied: inputs that contains: &&, |, < and >

Note

Using kubectl apply:

1. kubectl apply -f - does not work as Windows doesn't work with the << EOF + EOF redirection

2. The restricted shell interface prevents file creation/edit on the kubectl connector, to apply a Kubernetes manifest file, use the PSM drive mapping feature to upload the manifest via the drive mapping, and use kubectl apply -f Z:\<filename>

Note

Executables directory:

The kubectlWrapper.exe and kubectl.exe executables must be put into the same directory on each PSM server

The directory where the executables are in should be populated in the connection component setting: ConnectionComponent / TargetSettings / ClientSpecific / kubectlDirectory