Skip to content

Commit 3968832

Browse files
zwangshengulysses-you
authored andcommitted
[KYUUBI #1454] Add Doc About Deploy Kyuubi On Kubernetes
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> The Kyuubi documentation is missing about deploying Kyuubi on Kubernetes. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1457 from zwangsheng/doc/kyuubi_on_k8s. Closes #1454 f916866 [zwangsheng] modify .html to .md 707368c [zwangsheng] fix url & add todo 1a50978 [zwangsheng] Fix spelling a624a60 [zwangsheng] fix 4ca7416 [zwangsheng] rename & add Authored-by: zwangsheng <2213335496@qq.com> Signed-off-by: ulysses-you <ulyssesyou@apache.org>
1 parent 795b7e1 commit 3968832

File tree

4 files changed

+104
-2
lines changed

4 files changed

+104
-2
lines changed
File renamed without changes.

docs/deployment/index.rst

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,8 +29,9 @@ Basics
2929
:numbered: 3
3030
:glob:
3131

32-
on_yarn
33-
on_kubernetes
32+
engine_on_yarn
33+
engine_on_kubernetes
34+
kyuubi_on_kubernetes
3435
hive_metastore
3536
high_availability_guide
3637
metrics
Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
<!--
2+
- Licensed to the Apache Software Foundation (ASF) under one or more
3+
- contributor license agreements. See the NOTICE file distributed with
4+
- this work for additional information regarding copyright ownership.
5+
- The ASF licenses this file to You under the Apache License, Version 2.0
6+
- (the "License"); you may not use this file except in compliance with
7+
- the License. You may obtain a copy of the License at
8+
-
9+
- http://www.apache.org/licenses/LICENSE-2.0
10+
-
11+
- Unless required by applicable law or agreed to in writing, software
12+
- distributed under the License is distributed on an "AS IS" BASIS,
13+
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
- See the License for the specific language governing permissions and
15+
- limitations under the License.
16+
-->
17+
18+
<div align=center>
19+
20+
![](../imgs/kyuubi_logo.png)
21+
22+
</div>
23+
24+
# Deploy Kyuubi On Kubernetes
25+
26+
## Requirements
27+
28+
If you want to deploy Kyuubi on Kubernetes, you'd better get a sense of the following things.
29+
30+
* Use Kyuubi official docker image or build Kyuubi docker image
31+
* An active Kubernetes cluster
32+
* Reading About [Deploy Kyuubi engines on Kubernetes](engine_on_kubernetes.md)
33+
* [Kubectl](https://kubernetes.io/docs/reference/kubectl/overview/)
34+
* KubeConfig of the target cluster
35+
36+
## Kyuubi Official Docker Image
37+
38+
You can find the official docker image at [Apache Kyuubi (Incubating) Docker Hub](https://registry.hub.docker.com/r/apache/kyuubi).
39+
40+
## Build Kyuubi Docker Image
41+
42+
You can build custom Docker images from the `${KYUUBI_HOME}/bin/docker-image-tool.sh` contained in the binary package.
43+
44+
Examples:
45+
```shell
46+
- Build and push image with tag "v1.4.0" to docker.io/myrepo
47+
$0 -r docker.io/myrepo -t v1.4.0 build
48+
$0 -r docker.io/myrepo -t v1.4.0 push
49+
50+
- Build and push with tag "v3.0.0" and Spark-3.1.2 as base image to docker.io/myrepo
51+
$0 -r docker.io/myrepo -t v3.0.0 -b BASE_IMAGE=repo/spark:3.1.2 build
52+
$0 -r docker.io/myrepo -t v3.0.0 push
53+
54+
- Build and push for multiple archs to docker.io/myrepo
55+
$0 -r docker.io/myrepo -t v3.0.0 -X build
56+
57+
- Build with Spark placed "/path/spark"
58+
$0 -s /path/spark build
59+
```
60+
61+
`${KYUUBI_HOME}/bin/docker-image-tool.sh` use `Kyuubi Version` as default docker tag and always build `${repo}/kyuubi:${tag}` image.
62+
63+
The script can also help build external Spark into a Kyuubi image that acts as a client for submitting tasks by `-s ${SPAAK_HOME}`.
64+
65+
## Deploy
66+
67+
Multiple YAML files are provided under `${KYUUBI_HOME}/docker/` to help you deploy Kyuubi.
68+
69+
You can deploy single-node Kyuubi through `${KYUUBI_HOME}/docker/kyuubi-pod.yaml` or `${KYUUBI_HOME}/docker/kyuubi-deployment.yaml`.
70+
71+
Also, you can use `${KYUUBI_HOME}/docker/kyuubi-service.yaml` to deploy Kyuubi Service.
72+
73+
## Config
74+
75+
You can configure Kyuubi the old-fashioned way by placing kyuubi-default.conf inside the image. Kyuubi do not recommend using this way on Kubernetes.
76+
77+
Kyuubi provide `${KYUUBI_HOME}/docker/kyuubi-configmap.yaml` to build Configmap for Kyuubi.
78+
79+
You can find out how to use it in the comments inside the above file.
80+
81+
If you want to know kyuubi engine on kubernetes configurations, you can refer to [Deploy Kyuubi engines on Kubernetes](engine_on_kubernetes.md)
82+
83+
## Connect
84+
85+
If you do not use Service or HostNetwork to get the IP address of the node where Kyuubi deployed.
86+
You should connect like:
87+
```shell
88+
kubectl exec -it kyuubi-example -- /bin/bash
89+
${SPARK_HOME}/bin/beeline -u 'jdbc:hive2://localhost:10009'
90+
```
91+
92+
Or you can submit tasks directly through local beeline:
93+
```shell
94+
${SPARK_HOME}/bin/beeline -u 'jdbc:hive2://${hostname}:${port}'
95+
```
96+
As using service nodePort, port means nodePort and hostname means any hostname of kubernetes node.
97+
98+
As using HostNetwork, port means kyuubi containerPort and hostname means hostname of node where Kyuubi deployed.
99+
100+
## TODO
101+
Kyuubi will provide other connection methods in the future, like `Ingress`, `Load Balance`.

0 commit comments

Comments
 (0)