Skip to content

yahoo/k8s-athenz-identity

master
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Code

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
k8s
 
 
pkg
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

k8s-athenz-identity

k8s-athenz-identity is a Kubernetes control plane component which aims to securely provision unique Athenz identities (X.509 certificates) for pods.

Background

Kubernetes provides a mechanism for pods to obtain bound service account JWTs issued by the API server. However, this JWT is not directly useful outside the Kubernetes cluster. In general, applications require credentials issued by an organization specific IAM provider to authenticate with external systems. This project aims to provide a way for pods to exchange service account JWTs with a credential provider for a unique identity.

Athenz

Athenz is an IAM provider which allows users to define role based access control (RBAC) for users and services. It also acts as a certificate authority (CA) by provisioning instances with unique identities through X.509 certificates.

Copper Argos

Athenz Copper Argos is a callback mechanism for identity providers to validate certificate requests from instances running in a variety of platforms such as Kubernetes pods, openstack vms, AWS EC2s, etc. This allows any orchestration layer to implement platform specific instance verification checks.

Architecture

The architecture section dives into the details of the various components built to create an Athenz Copper Argos identity provider.

Screenshot

SIA

Service Identity Agent (SIA) is a container which is bundled as a sidecar for an application pod. It is primarily responsible for creating a CSR for the application and requesting a certificate from Athenz ZTS. SIA is mounted with a bound Kubernetes service account JWT issued with an audience specific to the identity provider, "athenz-identityd" (can be the Athenz service of identity provider).

These are the steps the SIA container follows to retrieve an identity:

  1. Creates a new private key and signs a CSR with the subject common name as "athenz-domain.athenz-service", required SANs, etc.
  2. Constructs an Athenz InstanceRegisterInformation object with the CSR and the bound service account JWT as attestation data.
  3. Makes a request to ZTS to the postInstanceRegister API call.
  4. ZTS forwards the attestation data and CSR details to the identity provider for validation.
  5. Certificate is minted by Athenz ZTS and returned to the SIA container.
  6. SIA writes the certificates to a directory for the application to consume.

Identity Provider

Identity provider is an Athenz Copper Argos callback provider which validates requests for new identities. It runs as a deployment in the cluster and has an in-memory cache of all running pods.

These are the steps the Identity provider follows to validate an identity:

  1. Validate the bound service account JWT of the attestation data is valid, this involves either making a request to the Kubernetes TokenReview API or using public key validation.
  2. Validate the pod which is requesting the identity is actually running within the cluster.
  3. Validate the CSR details including IP, SANS, common name, etc.

The checks are implemented as rego policies evaluated by an OPA server.

Using Open Policy Agent (OPA) for the Identity Provider

OPA provides a policy evaluation engine and a Rego language to write the policies. The most common use case of OPA in Kubernetes is for admission control where the checks are written as rego policies and loaded to OPA through configmaps. If the admission checks require to lookup any kubernetes object, OPA provides replication (cache) setup for cluster & namespace scoped resources. This replication mechanism also allows updating policy checks dynamically.

The idea is to run the identity provider pod as an OPA instance and define the identity verification checks as Rego policies written in configmaps. OPA’s rego evaluation engine also has built-in support for JWT verification and making HTTP requests. This helps us to verify the attestation data JWT either by making Kubernetes API TokenReview requests or by locally validating the signature using the pre-loaded kubernetes API public key. Other field verifications that depend on the pod metadata can use the resource replication feature and cache the pod resources.

The identity provider pod contains two containers:

  • Opa - Evaluates the identity verification checks with information provided with from the policy configmaps and replicated pods resource information
  • Mgmt - Fetches the configmap policies from the configured system namespace and the cluster pod resources by watching the Kubernetes API and storing them on OPA through localhost REST API calls.

Getting Started

Prerequisites

There are a couple of prerequisites required to run this identity provider, they are specified below.

  • Kubernetes cluster - A running Kubernetes cluster, version 1.13 or higher, is required with admin access. More information on how to setup a cluster can be found in the official documentation here.
  • Athenz - A complete Athenz setup including ZMS, ZTS, and an optional UI server installed.

Setup

  1. Clone the repository and use kubectl to create the RBAC, and the Identity provider deployment and service

    git clone https://github.com/yahoo/k8s-athenz-identity.git
    kubectl apply -f k8s-athenz-identity/k8s
    
  2. Create an Athenz domain and a service using zms-cli or ZMS UI corresponding to the Identityd k8s service

    # Build the zms-cli utility to interact with the Athenz ZMS server
    go get github.com/yahoo/athenz/utils/zms-cli
    
    # Create an Athenz domain associated with the Kubernetes cluster
    zms-cli add-domain k8s-cluster-name
    
    # Create an Athenz service on the cluster domain to associate it with the Identityd Kubernetes service
    zms-cli -d k8s-cluster-name add-service identityd
    
    # Set the Identityd service endpoint to point to the kubernetes identityd service
    zms-cli -d k8s-cluster-name set-service-endpoint identityd identityd.default.svc.cluster.local
    

Usage

  1. The application deployments that require Athenz identity certs need the SIA container and the kubernetes bound service account JWT volume mount added to the pod template using the sample patch

    kubectl patch deploy <app.yaml> -p k8s-athenz-identity/k8s/patch/sia.yaml
    

The generated Athenz cert and key is stored under the tls-certs volume mount.

Configuration

SIA

The following table outlines the configuration parameters for the SIA. Environment variables can be used instead of the parameters

Parameter Environment Variable Description Default
mode MODE Mode, must be one of init or refresh init
endpoint ENDPOINT Athenz ZTS endpoint
provider-service PROVIDER_SERVICE Identity Provider service
dns-suffix DNS_SUFFIX DNS Suffix for the certs
refresh-interval REFRESH_INTERVAL Interval to refresh the certs 24h
out-cert CERT_FILE Path to output the certificate /var/run/athenz/service.cert.pem
out-key KEY_FILE Path to output the key /var/run/athenz/service.key.pem
out-ca-cert CA_CERT_FILE Path to output the ca certificate /var/run/athenz/ca.cert.pem
log-dir LOG_DIR Directory to store the server log files /var/log/athenz-sia
log-level LOG_LEVEL Logging level INFO
sa-token-file SA_TOKEN_FILE Bound sa token file location /var/run/secrets/kubernetes.io/bound-serviceaccount/token
server-ca-cert SERVER_CA_CERT Path to the ca cert file to verify ZTS server certs

e.g.

$ athenz-sia
    --mode=init
    --out-cert=/var/run/athenz/service.cert.pem
    --out-key=/var/run/athenz/service.key.pem
    --out-ca-cert=/var/run/athenz/ca.cert.pem
    ...

$ athenz-sia
    --mode=refresh
    --refresh-interval=720h
    --out-cert=/var/run/athenz/service.cert.pem
    --out-key=/var/run/athenz/service.key.pem
    --out-ca-cert=/var/run/athenz/ca.cert.pem
    ...

Contribute

Please refer to the contributing file for information about how to get involved. We welcome issues, questions, and pull requests.

Maintainers/Contacts

Core Team : omega-core@verizonmedia.com

License

Copyright 2019 Verizon Media Inc. Licensed under the terms of the 3-Clause BSD License.

About

Athenz Identity provider for kubernetes services

Resources

License

Code of conduct

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published