|
| 1 | +--- |
| 2 | +title: Kubernetes |
| 3 | +--- |
| 4 | + |
| 5 | +<!-- |
| 6 | +# |
| 7 | +# Licensed to the Apache Software Foundation (ASF) under one or more |
| 8 | +# contributor license agreements. See the NOTICE file distributed with |
| 9 | +# this work for additional information regarding copyright ownership. |
| 10 | +# The ASF licenses this file to You under the Apache License, Version 2.0 |
| 11 | +# (the "License"); you may not use this file except in compliance with |
| 12 | +# the License. You may obtain a copy of the License at |
| 13 | +# |
| 14 | +# http://www.apache.org/licenses/LICENSE-2.0 |
| 15 | +# |
| 16 | +# Unless required by applicable law or agreed to in writing, software |
| 17 | +# distributed under the License is distributed on an "AS IS" BASIS, |
| 18 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| 19 | +# See the License for the specific language governing permissions and |
| 20 | +# limitations under the License. |
| 21 | +# |
| 22 | +--> |
| 23 | + |
| 24 | +## Summary |
| 25 | + |
| 26 | +The [_Kubernetes_](https://kubernetes.io/) service discovery [_List-Watch_](https://kubernetes.io/docs/reference/using-api/api-concepts/) real-time changes of [_Endpoints_](https://kubernetes.io/docs/concepts/services-networking/service/) resources, |
| 27 | +then store theirs value into ngx.shared.kubernetes \ |
| 28 | +Discovery also provides a query interface in accordance with the [_APISIX Discovery Specification_](https://github.com/apache/apisix/blob/master/docs/en/latest/discovery.md) |
| 29 | + |
| 30 | +## Configuration |
| 31 | + |
| 32 | +A detailed configuration for the kubernetes service discovery is as follows: |
| 33 | + |
| 34 | +```yaml |
| 35 | +discovery: |
| 36 | + kubernetes: |
| 37 | + service: |
| 38 | + # apiserver schema, options [http, https] |
| 39 | + schema: https #default https |
| 40 | + |
| 41 | + # apiserver host, options [ipv4, ipv6, domain, environment variable] |
| 42 | + host: ${KUBERNETES_SERVICE_HOST} #default ${KUBERNETES_SERVICE_HOST} |
| 43 | + |
| 44 | + # apiserver port, options [port number, environment variable] |
| 45 | + port: ${KUBERNETES_SERVICE_PORT} #default ${KUBERNETES_SERVICE_PORT} |
| 46 | + |
| 47 | + client: |
| 48 | + # serviceaccount token or token_file |
| 49 | + token_file: /var/run/secrets/kubernetes.io/serviceaccount/token |
| 50 | + |
| 51 | + #token: |- |
| 52 | + # eyJhbGciOiJSUzI1NiIsImtpZCI6Ikx5ME1DNWdnbmhQNkZCNlZYMXBsT3pYU3BBS2swYzBPSkN3ZnBESGpkUEEif |
| 53 | + # 6Ikx5ME1DNWdnbmhQNkZCNlZYMXBsT3pYU3BBS2swYzBPSkN3ZnBESGpkUEEifeyJhbGciOiJSUzI1NiIsImtpZCI |
| 54 | + |
| 55 | + # kubernetes discovery plugin support use namespace_selector |
| 56 | + # you can use one of [equal, not_equal, match, not_match] filter namespace |
| 57 | + namespace_selector: |
| 58 | + # only save endpoints with namespace equal default |
| 59 | + equal: default |
| 60 | + |
| 61 | + # only save endpoints with namespace not equal default |
| 62 | + #not_equal: default |
| 63 | + |
| 64 | + # only save endpoints with namespace match one of [default, ^my-[a-z]+$] |
| 65 | + #match: |
| 66 | + #- default |
| 67 | + #- ^my-[a-z]+$ |
| 68 | + |
| 69 | + # only save endpoints with namespace not match one of [default, ^my-[a-z]+$ ] |
| 70 | + #not_match: |
| 71 | + #- default |
| 72 | + #- ^my-[a-z]+$ |
| 73 | + |
| 74 | + # kubernetes discovery plugin support use label_selector |
| 75 | + # for the expression of label_selector, please refer to https://kubernetes.io/docs/concepts/overview/working-with-objects/labels |
| 76 | + label_selector: |- |
| 77 | + first="a",second="b" |
| 78 | +``` |
| 79 | +
|
| 80 | +If the kubernetes service discovery runs inside a pod, you can use minimal configuration: |
| 81 | +
|
| 82 | +```yaml |
| 83 | +discovery: |
| 84 | + kubernetes: { } |
| 85 | +``` |
| 86 | +
|
| 87 | +If the kubernetes service discovery runs outside a pod, you need to create or select a specified [_ServiceAccount_](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/), |
| 88 | +then get its token value, and use following configuration: |
| 89 | +
|
| 90 | +```yaml |
| 91 | +discovery: |
| 92 | + kubernetes: |
| 93 | + service: |
| 94 | + schema: https |
| 95 | + host: # enter apiserver host value here |
| 96 | + port: # enter apiserver port value here |
| 97 | + client: |
| 98 | + token: # enter serviceaccount token value here |
| 99 | + #token_file: # enter file path here |
| 100 | +``` |
| 101 | + |
| 102 | +## Interface |
| 103 | + |
| 104 | +the kubernetes service discovery provides a query interface in accordance with the [_APISIX Discovery Specification_](https://github.com/apache/apisix/blob/master/docs/en/latest/discovery.md) |
| 105 | + |
| 106 | +**function:** \ |
| 107 | + nodes(service_name) |
| 108 | + |
| 109 | +**description:** \ |
| 110 | + nodes() function attempts to look up the ngx.shared.kubernetes for nodes corresponding to service_name, \ |
| 111 | + service_name should match pattern: _[namespace]/[name]:[portName]_ |
| 112 | + |
| 113 | + + namespace: The namespace where the kubernetes endpoints is located |
| 114 | + |
| 115 | + + name: The name of the kubernetes endpoints |
| 116 | + |
| 117 | + + portName: The portName of the kubernetes endpoints, if there is no portName, use targetPort, port instead |
| 118 | + |
| 119 | +**return value:** \ |
| 120 | + if the kubernetes endpoints value is as follows: |
| 121 | + |
| 122 | + ```yaml |
| 123 | + apiVersion: v1 |
| 124 | + kind: Endpoints |
| 125 | + metadata: |
| 126 | + name: plat-dev |
| 127 | + namespace: default |
| 128 | + subsets: |
| 129 | + - addresses: |
| 130 | + - ip: "10.5.10.109" |
| 131 | + - ip: "10.5.10.110" |
| 132 | + ports: |
| 133 | + - port: 3306 |
| 134 | + ``` |
| 135 | +
|
| 136 | + a nodes("default/plat-dev:3306") call will get follow result: |
| 137 | +
|
| 138 | + ``` |
| 139 | + { |
| 140 | + { |
| 141 | + host="10.5.10.109", |
| 142 | + port= 3306, |
| 143 | + weight= 50, |
| 144 | + }, |
| 145 | + { |
| 146 | + host="10.5.10.110", |
| 147 | + port= 3306, |
| 148 | + weight= 50, |
| 149 | + }, |
| 150 | + } |
| 151 | + ``` |
| 152 | + |
| 153 | +## Q&A |
| 154 | + |
| 155 | +> Q: Why only support configuration token to access _Kubernetes APIServer_ \ |
| 156 | +> A: Usually, we will use three ways to complete the authentication of _Kubernetes APIServer_: |
| 157 | +> |
| 158 | +>+ mTLS |
| 159 | +>+ token |
| 160 | +>+ basic authentication |
| 161 | +> |
| 162 | +> Because lua-resty-http does not currently support mTLS, and basic authentication is not recommended,\ |
| 163 | +> So currently only the token authentication method is implemented |
| 164 | +
|
| 165 | +--- |
| 166 | + |
| 167 | +> Q: APISIX inherits Nginx's multiple process model, does it mean that each nginx worker process will [_List-Watch_](https://kubernetes.io/docs/reference/using-api/api-concepts/) kubernetes endpoints resources \ |
| 168 | +> A: The kubernetes service discovery only uses privileged processes to [_List-Watch_](https://kubernetes.io/docs/reference/using-api/api-concepts/) kubernetes endpoints resources, then store theirs value \ |
| 169 | +> into ngx.shared.kubernetes, worker processes get results by querying ngx.shared.kubernetes |
| 170 | +
|
| 171 | +--- |
| 172 | + |
| 173 | +> Q: How to get [_ServiceAccount_](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/) token value \ |
| 174 | +> A: Assume your [_ServiceAccount_](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/) located in namespace apisix and name is kubernetes-discovery, you can use the following steps to get token value |
| 175 | +> |
| 176 | +> 1. Get secret name: \ |
| 177 | +> you can execute the following command, the output of the first column is the secret name we want |
| 178 | +> |
| 179 | +> ```shell |
| 180 | +> kubectl -n apisix get secrets | grep kubernetes-discovery |
| 181 | +> ``` |
| 182 | +> |
| 183 | +> 2. Get token value: \ |
| 184 | +> assume secret resources name is kubernetes-discovery-token-c64cv, you can execute the following command, the output is the service account token value we want |
| 185 | +> |
| 186 | +> ```shell |
| 187 | +> kubectl -n apisix get secret kubernetes-discovery-token-c64cv -o jsonpath={.data.token} | base64 -d |
| 188 | +> ``` |
0 commit comments