Kubernetes Applications With Azure Active Directory Identities
AAD and Kubernetes
The relationship between kubernetes and AAD is covered in three main areas:
Cluster Identity: The identity used by the cloud provider running in various kubernetes components to perform operations against Azure, typically against Azure's resource group where the cluster lives. This identity is set during the cluster bring up process. This is not included in the scope of this proposal.
User Identity: What enables user/operator to authenticate against AAD using AAD before using
kubectlcommands. This is not included in the scope of this proposal.
Application Identities: Identities that are used by applications running on kubernetes to access any resources that uses AAD as identity provider. These resources can be ARM, Applications running on the same cluster, on azure, or anywhere else. Managing, assigning these identities is the scope of this document.
This proposal does not cover how application can be configured to use AAD as identity/authentication provider.
kubernetes applications depending on other applications that use AAD as an identity provider. These applications include Azure 1st party services (such as ARM, Azure sql or Azure storage) or user applications running on Azure (including same cluster) and or on premises.
Azure 1st party services are all moving to use AAD as the primary identity provider.
Delegating authorization to user familiar tools such as AAD group memberships.
Enable identity rotation without application interruption.
Example: rotating a service principal password/cert without having to edit secrets assigned directly to applications.
Provide a framework to enable time-boxed identity assignment. Manually triggered or automated. The same framework can be used for (jit sudo style access with automation tools).
Example: a front end application can have access to centralized data store between midnight and 1 AM during business days only.
Favor little to no change to how users currently write applications against various editions of ADAL. Favor committing changes to SDKs and don't ask users to change applications that are written for Kubernetes.
Favor little to no change in the way users create kubernetes application specs (favor declarative approach). This enables users to focus their development and debugging experience in code they wrote, not code imposed on them.
Example: favor annotation and labels over side-cars (even dynamically injected).
Prep for SPIFFE work currently in progress by the wider community.
Separate identities from
identity assignmentapplications enables users to swap identities used by the applications.
AAD Identity Management and Assignment (within cluster)
Cluster operators create instances of
crd:azureIdentity. Each instance is a kubernetes object representing Azure AAD identity that can be EMSI or service principal (with password).
Cluster operators create instances of
crd:azureIdentityBinding. Each instance represents binding between
crd:azureIdentity. The binding initially will use (one of)
- Explicit assignment to applications (?)
- Weight (used in case of a single pod matched by multiple identities)
At later phases, the binding will use:
1. Expiration time (static: bind until Jan 1/1/2019. Dynamic: for 10 mins). 2. ...? > All identity bindings actions are logged as `events` to `crd:azureIdentity`. Allowing clusters operators to track how the system assigned identities to various apps.
- A Controller will run to create
Kubernetes applications (pods) will default to use MSI endpoint.
All traffic to MSI endpoint is routed via
iptablesto a daemon-set that will use
sourceIpto identify pod, then find an appropriate
crd:azureIdnetityBinding. The daemon-set mimics all the REST api offered by MSI endpoint. All tokens are presented to pods on MSI endpoint irrespective of the identity used to back this request (EMSI or service principal).
As SPIFFE becomes more mature pod identity assertion is expected to use SPIFFE pod identity.
All token issuance will be logged as events attached to
crd:azureIdnetityBindingfor audit purposes.
azureIdentityController controller is use to:
1. Create instances of `azureAssignedIdentity` as a result of `crd:azureIdentityBinding` matching. 2. Assign EMSI to nodes where pods are scheduled to run. Assignment are logged to `crd:azureIdentity` as events.
The controller can later expose an endpoint for analysis where operators can request analysis of identities, pod assignment.
A custom resource definition object that represents either
- Explicit Managed Service Identity (EMSI).
- Service Principal.
<<Object details>> type: <EMSI/Principal> secretRef: <Principal Password>
Identities are created separately by azure cli. ARM RBAC and data plane specific role assignment is done on each service, application or resource.
A custom resource definition object used to describe
azureIdentity is assigned to pod.
<<object details>> azureIdentityRef: xxx type: <Explicit/Selector> weight: ?? timeBox: <<future time box details>>
A custom resource definition object. Each instance represents linking
<<object details>> azureIdentityRef: xxx podRef: xxx
- Api, controller and daemon set (excluding time boxed assignment).
- Time box assignment
- SPIFFE integration