You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[KYUUBI #2243][DOCS] Add quick start for trino engine
### _Why are the changes needed?_
Add quick start for trino engine
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2265 from hddong/trino-quick-start.
Closes#2243fc4c274 [hongdongdong] [KYUUBI #2243] Add quick start for trino engine
Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com>
Signed-off-by: Kent Yao <yao@apache.org>
Copy file name to clipboardExpand all lines: docs/quick_start/quick_start.md
+73-5Lines changed: 73 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -44,12 +44,13 @@ Components| Role | Optional | Version | Remarks
44
44
Java | Java<br>Runtime<br>Environment | Required | Java 8/11 | Kyuubi is pre-built with Java 8
45
45
Spark | Distributed<br>SQL<br>Engine | Optional | 3.0.0 and above | By default Kyuubi binary release is delivered without<br> a Spark tarball.
46
46
Flink | Distributed<br>SQL<br>Engine | Optional | 1.14.0 and above | By default Kyuubi binary release is delivered without<br> a Flink tarball.
47
+
Trino | Distributed<br>SQL<br>Engine | Optional | 363 and above | By default Kyuubi binary release is delivered without<br> a Trino tarball.
47
48
HDFS | Distributed<br>File<br>System | Optional | referenced<br>by<br>Spark | Hadoop Distributed File System is a <br>part of Hadoop framework, used to<br> store and process the datasets.<br> You can interact with any<br> Spark-compatible versions of HDFS.
48
49
Hive | Metastore | Optional | referenced<br>by<br>Spark | Hive Metastore for Spark SQL to connect
49
50
Zookeeper | Service<br>Discovery | Optional | Any<br>zookeeper<br>ensemble<br>compatible<br>with<br>curator(2.12.0) | By default, Kyuubi provides a<br> embedded Zookeeper server inside for<br> non-production use.
50
51
51
-
Additionally, if you want to work with other Spark/Flink compatible systems or plugins, you only need to take care of them as using them with regular Spark/Flink applications.
52
-
For example, you can run Spark/Flink SQL engines created by the Kyuubi on any cluster manager, including YARN, Kubernetes, Mesos, e.t.c...
52
+
Additionally, if you want to work with other Spark/Flink/Trino compatible systems or plugins, you only need to take care of them as using them with regular Spark/Flink/Trino applications.
53
+
For example, you can run Spark/Flink/Trino SQL engines created by the Kyuubi on any cluster manager, including YARN, Kubernetes, Mesos, e.t.c...
53
54
Or, you can manipulate data from different data sources with the Spark Datasource/Flink Table API, e.g. Delta Lake, Apache Hudi, Apache Iceberg, Apache Kudu and e.t.c...
54
55
55
56
## Installation
@@ -99,7 +100,7 @@ From top to bottom are:
99
100
- bin: the entry of the Kyuubi server with `kyuubi` as the startup script.
100
101
- conf: all the defaults used by Kyuubi Server itself or creating a session with Spark applications.
101
102
- externals
102
-
- engines: contains all kinds of SQL engines that we support, e.g. Apache Spark, Apache Flink, Trino(coming soon).
103
+
- engines: contains all kinds of SQL engines that we support, e.g. Apache Spark, Apache Flink, Trino.
103
104
- licenses: a bunch of licenses included.
104
105
- jars: packages needed by the Kyuubi server.
105
106
- logs: where the logs of the Kyuubi server locates.
@@ -113,6 +114,7 @@ As mentioned above, for a quick start deployment, then only you need to be sure
113
114
- Java runtime environment
114
115
-`SPARK_HOME` for the Spark engine
115
116
-`FLINK_HOME` and `kyuubi.engine.type` in `$KYUUBI_HOME/conf/kyuubi-defaults.conf` for the Flink engine.
117
+
-`kyuubi.engine.type``session.engine.trino.connection.url` and `session.engine.trino.connection.catalog` in `$KYUUBI_HOME/conf/kyuubi-defaults.conf` for the Trino engine
116
118
117
119
### Setup JAVA
118
120
@@ -170,6 +172,21 @@ To enable the Flink SQL engine, the `kyuubi.engine.type` in `$KYUUBI_HOME/conf/k
170
172
kyuubi.engine.type FLINK_SQL
171
173
```
172
174
175
+
### Trino Engine
176
+
177
+
#### Setup Trino
178
+
Different from Spark/Flink, you must have a Trino cluster first. Trino client stored in `$KYUUBI_HOME/externals/engines/trino`.
179
+
180
+
#### Setup Kyuubi Trino Configration
181
+
182
+
To enable the Trino engine, the `kyuubi.engine.type` need to be set as `TRINO`. And `session.engine.trino.connection.url` and `session.engine.trino.connection.catalog` are also necessary. You can set all those configs in `$KYUUBI_HOME/conf/kyuubi-defaults.conf`, or set them in your connection parameters.
183
+
184
+
```bash
185
+
kyuubi.engine.type TRINO
186
+
session.engine.trino.connection.url http://localhost:8080 # Your trino cluster server url
187
+
session.engine.trino.connection.catalog hive # The default catalog connect to.
188
+
```
189
+
173
190
### Starting Kyuubi
174
191
175
192
```bash
@@ -241,7 +258,7 @@ Beeline version 2.3.7 by Apache Hive
241
258
242
259
In this case, the session will create for the user named 'anonymous'.
243
260
244
-
Kyuubi will create a Spark/Flink SQL engine application using `kyuubi-<engine>-sql-engine_2.12-<version>.jar`.
261
+
Kyuubi will create a Spark/Flink/Trino SQL engine application using `kyuubi-<engine>-sql-engine_2.12-<version>.jar`.
245
262
It will cost awhile for the application to be ready before fully establishing the session.
246
263
Otherwise, an existing application will be reused, and the time cost here is negligible.
The formerly created Spark application for user 'anonymous' will not be reused in this case, while a brand new application will be submitted for user 'kentyao' instead.
255
272
256
-
Then, you can see two processes running in your local environment, including one `KyuubiServer` instance, one `SparkSubmit` or `FlinkSQLEngine` instances as the SQL engines.
273
+
Then, you can see two processes running in your local environment, including one `KyuubiServer` instance, one `SparkSubmit``FlinkSQLEngine`or `TrinoSqlEngine` instances as the SQL engines.
257
274
258
275
- Spark
259
276
@@ -271,6 +288,14 @@ Then, you can see two processes running in your local environment, including one
271
288
43260 FlinkSQLEngine
272
289
```
273
290
291
+
- Trino
292
+
293
+
```
294
+
63483 Jps
295
+
63693 KyuubiServer
296
+
63266 TrinoSqlEngine
297
+
```
298
+
274
299
### Execute Statements
275
300
276
301
#### Execute Spark SQL Statements
@@ -425,6 +450,49 @@ For example, you can get the Flink web UI from the log for debugging or tuning.
425
450
426
451
427
452
453
+
#### Execute Trino Statements
454
+
455
+
If the beeline session is successfully connected, then you can run any query supported by Trino now. For example,
| 749 | Customer#000000749 | U1Dvu0r793a | 24 | 34-158-697-9591 | 7491.42 | MACHINERY | accounts was. final, final requests wake. theodolites was slyly. blithely even foxes wake carefully ac |
480
+
| 750 | Customer#000000750 | 5OyNRajjgjjbaXtI rkxvB2lX4c6u | 8 | 18-235-587-1274 | 269.9 | BUILDING | s. regular, regular deposits sleep carefully blithely bol |
481
+
| 751 | Customer#000000751 | e OSrreG6sx7l1t3wAg8u11DWk D 9 | 0 | 10-658-550-2257 | 2130.98 | FURNITURE | ges sleep furiously bold deposits. furiously regular requests cajole slyly. unusual accounts nag unusual ide |
482
+
| 752 | Customer#000000752 | KtdEacPUecPdPLt99kwZrnH9oIxUxpw | 8 | 18-924-993-6038 | 8363.66 | MACHINERY | mong the ironic, final waters. regular deposits above the fluffily ironic instructions |
483
+
| 753 | Customer#000000753 | 9k2PLlDRbMq4oSvW5Hh7Ak5iRDH | 17 | 27-817-126-3646 | 8114.44 | HOUSEHOLD | cies. deposits snooze. final, regular excuses wake furiously about the furiously final foxes. dependencies |
484
+
| 754 | Customer#000000754 | 8r5wwhhlL9MkAxOhRK | 0 | 10-646-595-5871 | -566.86 | BUILDING | er regular accounts against the furiously unusual somas sleep carefull |
0 commit comments