Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[fedora-atomic][k8s] Support default Keystone auth policy file
With the new config option `keystone_auth_default_policy`, cloud admin can set a default keystone auth policy for k8s cluster when the keystone auth is enabled. As a result, user can use their current keystone user to access k8s cluster as long as they're assigned correct roles, and they will get the pre-defined permissions set by the cloud provider. The default policy now is based on the v2 format recently introduced in k8s-keystone-auth which is getting more useful now. For example, in v1 it doesn't support a policy for user to access resources from all namespaces but kube-system, but v2 can do that. NOTE: Now we're using openstackmagnum dockerhub repo until CPO team fixing their image release issue. Task: 30069 Story: 1755770 Change-Id: I2425e957bd99edc92482b6f11ca0b1f91fe59ff6
- Loading branch information
1 parent
05c27f2
commit d8df9d0
Showing
12 changed files
with
349 additions
and
24 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,145 @@ | ||
| Currently, there are several ways to access the Kubernetes API, such as RBAC, | ||
| ABAC, Webhook, etc. Though RBAC is the best way for most of the cases, Webhook | ||
| provides a good approach for Kubernetes to query an outside REST service when | ||
| determining user privileges. In other words, we can use a Webhook to integrate | ||
| other IAM service into Kubernetes. In our case, under the OpenStack context, | ||
| we're introducing the intergration with Keystone auth for Kubernetes. | ||
|
|
||
| Since Rocky release, we introduced a new label named `keystone_auth_enabled`, | ||
| by default it's True, which means user can get this very nice feature out of | ||
| box. | ||
|
|
||
| Create roles | ||
| ------------ | ||
|
|
||
| As cloud provider, necessary Keystone roles for Kubernetes cluster operations | ||
| need to be created for different users, e.g. k8s_admin, k8s_developer, | ||
| k8s_viewer | ||
|
|
||
| - k8s_admin role can create/update/delete Kubernetes cluster, can also | ||
| associate roles to other normal users within the tenant | ||
| - k8s_developer can create/update/delete/watch Kubernetes cluster resources | ||
| - k8s_viewer can only have read access to Kubernetes cluster resources | ||
|
|
||
| NOTE: Those roles will be created automatically in devstack. Below is the | ||
| samples commands about how to create them. | ||
|
|
||
| .. code-block:: bash | ||
| source ~/openstack_admin_credentials | ||
| for role in "k8s_admin" "k8s_developer" "k8s_viewer"; do openstack role create $role; done | ||
| openstack user create demo_viewer --project demo --password password | ||
| openstack role add --user demo_viewer --project demo k8s_viewer | ||
| openstack user create demo_editor --project demo --password password | ||
| openstack role add --user demo_developer --project demo k8s_developer | ||
| openstack user create demo_admin --project demo --password password | ||
| openstack role add --user demo_admin --project demo k8s_admin | ||
| Those roles should be public and can be accessed by any project so that user | ||
| can configure their cluster's role policies with those roles. | ||
|
|
||
| Setup configmap for authorization policies | ||
| ------------------------------------------ | ||
|
|
||
| Given the k8s Keystone auth has been enable by default, user can get the | ||
| authentication support by default without doing anything. However, user can't | ||
| do anything actually before setup a default authorization policies. | ||
|
|
||
| The authorization policy can be specified using an existing configmap name in | ||
| the cluster, by doing this, the policy could be changed dynamically without | ||
| the k8s-keystone-auth service restart. | ||
|
|
||
| Or the policy can be read from a default policy file. In devstack, the policy | ||
| file will be created automatically. | ||
|
|
||
| Currently, k8s-keystone-auth service supports four types of policies: | ||
|
|
||
| - user. The Keystone user ID or name. | ||
| - roject. The Keystone project ID or name. | ||
| - role. The user role defined in Keystone. | ||
| - group. The group is not a Keystone concept actually, it’s supported for | ||
| backward compatibility, you can use group as project ID. | ||
|
|
||
| For example, in the following configmap, we only allow the users in | ||
| project demo with k8s-viewer role in OpenStack to query the pod information | ||
| from all the namespaces. So we need to update the configmap | ||
| `k8s-keystone-auth-policy` which has been created in kube-system namespace. | ||
|
|
||
| .. code-block:: bash | ||
| cat <<EOF | kubectl apply -f - | ||
| apiVersion: v1 | ||
| kind: ConfigMap | ||
| metadata: | ||
| name: k8s-keystone-auth-policy | ||
| namespace: kube-system | ||
| data: | ||
| policies: | | ||
| [ | ||
| { | ||
| "resource": { | ||
| "verbs": ["get", "list", "watch"], | ||
| "resources": ["pods"], | ||
| "version": "*", | ||
| "namespace": "default" | ||
| }, | ||
| "match": [ | ||
| { | ||
| "type": "role", | ||
| "values": ["k8s-viewer"] | ||
| }, | ||
| { | ||
| "type": "project", | ||
| "values": ["demo"] | ||
| } | ||
| ] | ||
| } | ||
| ] | ||
| EOF | ||
| Please note that the default configmap name is `k8s-keystone-auth-policy`, user | ||
| can change it, but they have to change the config of the k8s keystone auth | ||
| service configuration as well and restart the service. | ||
| Now user need to get a token from Keystone to have a kubeconfig for kubectl, | ||
| user can also get the config with Magnum python client. | ||
| Here is a sample of the kubeconfig: | ||
| .. code-block:: bash | ||
| apiVersion: v1 | ||
| clusters: | ||
| - cluster: | ||
| certificate-authority-data: CERT-DATA== | ||
| server: https://172.24.4.25:6443 | ||
| name: k8s-2 | ||
| contexts: | ||
| - context: | ||
| cluster: k8s-2 | ||
| user: openstackuser | ||
| name: openstackuser@kubernetes | ||
| current-context: openstackuser@kubernetes | ||
| kind: Config | ||
| preferences: {} | ||
| users: | ||
| - name: openstackuser | ||
| user: | ||
| exec: | ||
| command: /bin/bash | ||
| apiVersion: client.authentication.k8s.io/v1alpha1 | ||
| args: | ||
| - -c | ||
| - > | ||
| if [ -z ${OS_TOKEN} ]; then | ||
| echo 'Error: Missing OpenStack credential from environment variable $OS_TOKEN' > /dev/stderr | ||
| exit 1 | ||
| else | ||
| echo '{ "apiVersion": "client.authentication.k8s.io/v1alpha1", "kind": "ExecCredential", "status": { "token": "'"${OS_TOKEN}"'"}}' | ||
| fi | ||
| Now after export the Keystone token to OS_TOKEN, user should be able to list | ||
| pods with kubectl. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| [ | ||
| { | ||
| "users":{ | ||
| "roles":[ | ||
| "k8s_admin" | ||
| ], | ||
| "projects":[ | ||
| "$PROJECT_ID" | ||
| ] | ||
| }, | ||
| "resource_permissions":{ | ||
| "*/*":[ | ||
| "*" | ||
| ] | ||
| }, | ||
| "nonresource_permissions":{ | ||
| "/healthz":[ | ||
| "get", | ||
| "post" | ||
| ] | ||
| } | ||
| }, | ||
| { | ||
| "users":{ | ||
| "roles":[ | ||
| "k8s_developer" | ||
| ], | ||
| "projects":[ | ||
| "$PROJECT_ID" | ||
| ] | ||
| }, | ||
| "resource_permissions":{ | ||
| "!kube-system/['apiServices', 'bindings', 'componentstatuses', 'configmaps', 'cronjobs', 'customResourceDefinitions', 'deployments', 'endpoints', 'events', 'horizontalPodAutoscalers', 'ingresses', 'initializerConfigurations', 'jobs', 'limitRanges', 'localSubjectAccessReviews', 'namespaces', 'networkPolicies', 'persistentVolumeClaims', 'persistentVolumes', 'podDisruptionBudgets', 'podPresets', 'podTemplates', 'pods', 'replicaSets', 'replicationControllers', 'resourceQuotas', 'secrets', 'selfSubjectAccessReviews', 'serviceAccounts', 'services', 'statefulSets', 'storageClasses', 'subjectAccessReviews', 'tokenReviews']":[ | ||
| "*" | ||
| ], | ||
| "*/['clusterrolebindings', 'clusterroles', 'rolebindings', 'roles', 'controllerrevisions', 'nodes', 'podSecurityPolicies']":[ | ||
| "get", | ||
| "list", | ||
| "watch" | ||
| ], | ||
| "*/['certificateSigningRequests']":[ | ||
| "create", | ||
| "delete", | ||
| "get", | ||
| "list", | ||
| "watch", | ||
| "update" | ||
| ] | ||
| } | ||
| }, | ||
| { | ||
| "users":{ | ||
| "roles":[ | ||
| "k8s_viewer" | ||
| ], | ||
| "projects":[ | ||
| "$PROJECT_ID" | ||
| ] | ||
| }, | ||
| "resource_permissions":{ | ||
| "!kube-system/['tokenReviews']":[ | ||
| "*" | ||
| ], | ||
| "!kube-system/['apiServices', 'bindings', 'componentstatuses', 'configmaps', 'cronjobs', 'customResourceDefinitions', 'deployments', 'endpoints', 'events', 'horizontalPodAutoscalers', 'ingresses', 'initializerConfigurations', 'jobs', 'limitRanges', 'localSubjectAccessReviews', 'namespaces', 'networkPolicies', 'persistentVolumeClaims', 'persistentVolumes', 'podDisruptionBudgets', 'podPresets', 'podTemplates', 'pods', 'replicaSets', 'replicationControllers', 'resourceQuotas', 'secrets', 'selfSubjectAccessReviews', 'serviceAccounts', 'services', 'statefulSets', 'storageClasses', 'subjectAccessReviews']":[ | ||
| "get", | ||
| "list", | ||
| "watch" | ||
| ], | ||
| "*/['clusterrolebindings', 'clusterroles', 'rolebindings', 'roles', 'controllerrevisions', 'nodes', 'podSecurityPolicies']":[ | ||
| "get", | ||
| "list", | ||
| "watch" | ||
| ] | ||
| } | ||
| } | ||
| ] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); you may not | ||
| # use this file except in compliance with the License. You may obtain a copy | ||
| # of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| from oslo_config import cfg | ||
|
|
||
| kubernetes_group = cfg.OptGroup(name='kubernetes', | ||
| title='Options for the Kubernetes addons') | ||
|
|
||
| kubernetes_opts = [ | ||
| cfg.StrOpt('keystone_auth_default_policy', | ||
| default="/etc/magnum/keystone_auth_default_policy.json", | ||
| help='Explicitly specify the path to the file defined default ' | ||
| 'Keystone auth policy for Kubernetes cluster when ' | ||
| 'the Keystone auth is enabled. Vendors can put their ' | ||
| 'specific default policy here'), | ||
| ] | ||
|
|
||
|
|
||
| def register_opts(conf): | ||
| conf.register_group(kubernetes_group) | ||
| conf.register_opts(kubernetes_opts, group=kubernetes_group) | ||
|
|
||
|
|
||
| def list_opts(): | ||
| return { | ||
| kubernetes_group: kubernetes_opts | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.