|
| 1 | +<!-- |
| 2 | + - Licensed to the Apache Software Foundation (ASF) under one or more |
| 3 | + - contributor license agreements. See the NOTICE file distributed with |
| 4 | + - this work for additional information regarding copyright ownership. |
| 5 | + - The ASF licenses this file to You under the Apache License, Version 2.0 |
| 6 | + - (the "License"); you may not use this file except in compliance with |
| 7 | + - the License. You may obtain a copy of the License at |
| 8 | + - |
| 9 | + - http://www.apache.org/licenses/LICENSE-2.0 |
| 10 | + - |
| 11 | + - Unless required by applicable law or agreed to in writing, software |
| 12 | + - distributed under the License is distributed on an "AS IS" BASIS, |
| 13 | + - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| 14 | + - See the License for the specific language governing permissions and |
| 15 | + - limitations under the License. |
| 16 | + --> |
| 17 | + |
| 18 | +<div align=center> |
| 19 | + |
| 20 | + |
| 21 | + |
| 22 | +</div> |
| 23 | + |
| 24 | +# Deploy Kyuubi On Kubernetes |
| 25 | + |
| 26 | +## Requirements |
| 27 | + |
| 28 | +If you want to deploy Kyuubi on Kubernetes, you'd better get a sense of the following things. |
| 29 | + |
| 30 | +* Use Kyuubi official docker image or build Kyuubi docker image |
| 31 | +* An active Kubernetes cluster |
| 32 | +* Reading About [Deploy Kyuubi engines on Kubernetes](engine_on_kubernetes.md) |
| 33 | +* [Kubectl](https://kubernetes.io/docs/reference/kubectl/overview/) |
| 34 | +* KubeConfig of the target cluster |
| 35 | + |
| 36 | +## Kyuubi Official Docker Image |
| 37 | + |
| 38 | +You can find the official docker image at [Apache Kyuubi (Incubating) Docker Hub](https://registry.hub.docker.com/r/apache/kyuubi). |
| 39 | + |
| 40 | +## Build Kyuubi Docker Image |
| 41 | + |
| 42 | +You can build custom Docker images from the `${KYUUBI_HOME}/bin/docker-image-tool.sh` contained in the binary package. |
| 43 | + |
| 44 | +Examples: |
| 45 | +```shell |
| 46 | + - Build and push image with tag "v1.4.0" to docker.io/myrepo |
| 47 | + $0 -r docker.io/myrepo -t v1.4.0 build |
| 48 | + $0 -r docker.io/myrepo -t v1.4.0 push |
| 49 | + |
| 50 | + - Build and push with tag "v3.0.0" and Spark-3.1.2 as base image to docker.io/myrepo |
| 51 | + $0 -r docker.io/myrepo -t v3.0.0 -b BASE_IMAGE=repo/spark:3.1.2 build |
| 52 | + $0 -r docker.io/myrepo -t v3.0.0 push |
| 53 | + |
| 54 | + - Build and push for multiple archs to docker.io/myrepo |
| 55 | + $0 -r docker.io/myrepo -t v3.0.0 -X build |
| 56 | + |
| 57 | + - Build with Spark placed "/path/spark" |
| 58 | + $0 -s /path/spark build |
| 59 | +``` |
| 60 | + |
| 61 | +`${KYUUBI_HOME}/bin/docker-image-tool.sh` use `Kyuubi Version` as default docker tag and always build `${repo}/kyuubi:${tag}` image. |
| 62 | + |
| 63 | +The script can also help build external Spark into a Kyuubi image that acts as a client for submitting tasks by `-s ${SPAAK_HOME}`. |
| 64 | + |
| 65 | +## Deploy |
| 66 | + |
| 67 | +Multiple YAML files are provided under `${KYUUBI_HOME}/docker/` to help you deploy Kyuubi. |
| 68 | + |
| 69 | +You can deploy single-node Kyuubi through `${KYUUBI_HOME}/docker/kyuubi-pod.yaml` or `${KYUUBI_HOME}/docker/kyuubi-deployment.yaml`. |
| 70 | + |
| 71 | +Also, you can use `${KYUUBI_HOME}/docker/kyuubi-service.yaml` to deploy Kyuubi Service. |
| 72 | + |
| 73 | +## Config |
| 74 | + |
| 75 | +You can configure Kyuubi the old-fashioned way by placing kyuubi-default.conf inside the image. Kyuubi do not recommend using this way on Kubernetes. |
| 76 | + |
| 77 | +Kyuubi provide `${KYUUBI_HOME}/docker/kyuubi-configmap.yaml` to build Configmap for Kyuubi. |
| 78 | + |
| 79 | +You can find out how to use it in the comments inside the above file. |
| 80 | + |
| 81 | +If you want to know kyuubi engine on kubernetes configurations, you can refer to [Deploy Kyuubi engines on Kubernetes](engine_on_kubernetes.md) |
| 82 | + |
| 83 | +## Connect |
| 84 | + |
| 85 | +If you do not use Service or HostNetwork to get the IP address of the node where Kyuubi deployed. |
| 86 | +You should connect like: |
| 87 | +```shell |
| 88 | +kubectl exec -it kyuubi-example -- /bin/bash |
| 89 | +${SPARK_HOME}/bin/beeline -u 'jdbc:hive2://localhost:10009' |
| 90 | +``` |
| 91 | + |
| 92 | +Or you can submit tasks directly through local beeline: |
| 93 | +```shell |
| 94 | +${SPARK_HOME}/bin/beeline -u 'jdbc:hive2://${hostname}:${port}' |
| 95 | +``` |
| 96 | +As using service nodePort, port means nodePort and hostname means any hostname of kubernetes node. |
| 97 | + |
| 98 | +As using HostNetwork, port means kyuubi containerPort and hostname means hostname of node where Kyuubi deployed. |
| 99 | + |
| 100 | +## TODO |
| 101 | +Kyuubi will provide other connection methods in the future, like `Ingress`, `Load Balance`. |
0 commit comments