Skip to content

Commit 1ea245d

Browse files
mahmoudbahaaSteNicholas
authored and
pan3793
committed
[KYUUBI #886] Add HTTP transport mode support to KYUUBI - no Kerberos support
Same as #2815 but with Kerberos auth removed as was not complete nor tested. so thought to make a separated PR with no Kerberos support then we can add Kerberos support later on ### _Why are the changes needed?_ Add http transport protocol support to Kyuubi main code was taken from: 1. SteNicholas@25ac073 2. https://github.com/apache/hive/blob/branch-3.1/service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpCLIService.java 2. https://github.com/apache/hive/blob/branch-3.1/service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpServlet.java ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #2833 from mahmoudbahaa/transport-mode-http-no-kerberos. Closes #886 3a24abe [Mahmoud Bahaa] fix test cases ce982af [Mahmoud Bahaa] [KYUUBI #886] Add HTTP transport mode support to KYUUBI - no kerberos support b637185 [SteNicholas] [KYUUBI #886] Add HTTP transport mode support to KYUUBI 36fb076 [Mahmoud Bahaa] update .gitignore to include conf non template files Lead-authored-by: Mahmoud Bahaa <mahmoud.bahaa@incorta.com> Co-authored-by: SteNicholas <programgeek@163.com> Signed-off-by: Cheng Pan <chengpan@apache.org>
1 parent 27330dd commit 1ea245d

File tree

17 files changed

+1231
-8
lines changed

17 files changed

+1231
-8
lines changed

‎.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,3 +74,6 @@ embedded_zookeeper/
7474
**/operation_logs/
7575
**/server_operation_logs/
7676
**/engine_operation_logs/
77+
conf/log4j2.properties
78+
conf/kyuubi-defaults.conf
79+
conf/kyuubi-env.sh

‎docs/deployment/settings.md

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -269,12 +269,30 @@ kyuubi.frontend.mysql.max.worker.threads|999|Maximum number of threads in the co
269269
kyuubi.frontend.mysql.min.worker.threads|9|Minimum number of threads in the command execution thread pool for the MySQL frontend service|int|1.4.0
270270
kyuubi.frontend.mysql.netty.worker.threads|&lt;undefined&gt;|Number of thread in the netty worker event loop of MySQL frontend service. Use min(cpu_cores, 8) in default.|int|1.4.0
271271
kyuubi.frontend.mysql.worker.keepalive.time|PT1M|Time(ms) that an idle async thread of the command execution thread pool will wait for a new task to arrive before terminating in MySQL frontend service|duration|1.4.0
272-
kyuubi.frontend.protocols|THRIFT_BINARY|A comma separated list for all frontend protocols <ul> <li>THRIFT_BINARY - HiveServer2 compatible thrift binary protocol.</li> <li>REST - Kyuubi defined REST API(experimental).</li> <li>MYSQL - MySQL compatible text protocol(experimental).</li> </ul>|seq|1.4.0
272+
kyuubi.frontend.protocols|THRIFT_BINARY|A comma separated list for all frontend protocols <ul> <li>THRIFT_BINARY - HiveServer2 compatible thrift binary protocol.</li> <li>THRIFT_HTTP - HiveServer2 compatible thrift http protocol.</li> <li>REST - Kyuubi defined REST API(experimental).</li> <li>MYSQL - MySQL compatible text protocol(experimental).</li> </ul>|seq|1.4.0
273273
kyuubi.frontend.rest.bind.host|&lt;undefined&gt;|Hostname or IP of the machine on which to run the REST frontend service.|string|1.4.0
274274
kyuubi.frontend.rest.bind.port|10099|Port of the machine on which to run the REST frontend service.|int|1.4.0
275275
kyuubi.frontend.thrift.backoff.slot.length|PT0.1S|Time to back off during login to the thrift frontend service.|duration|1.4.0
276276
kyuubi.frontend.thrift.binary.bind.host|&lt;undefined&gt;|Hostname or IP of the machine on which to run the thrift frontend service via binary protocol.|string|1.4.0
277277
kyuubi.frontend.thrift.binary.bind.port|10009|Port of the machine on which to run the thrift frontend service via binary protocol.|int|1.4.0
278+
kyuubi.frontend.thrift.http.allow.user.substitution|true|Allow alternate user to be specified as part of open connection request when using HTTP transport mode.|boolean|1.6.0
279+
kyuubi.frontend.thrift.http.bind.host|&lt;undefined&gt;|Hostname or IP of the machine on which to run the thrift frontend service via http protocol.|string|1.6.0
280+
kyuubi.frontend.thrift.http.bind.port|10010|Port of the machine on which to run the thrift frontend service via http protocol.|int|1.6.0
281+
kyuubi.frontend.thrift.http.compression.enabled|true|Enable thrift http compression via Jetty compression support|boolean|1.6.0
282+
kyuubi.frontend.thrift.http.cookie.auth.enabled|true|When true, Kyuubi in HTTP transport mode, will use cookie based authentication mechanism|boolean|1.6.0
283+
kyuubi.frontend.thrift.http.cookie.domain|&lt;undefined&gt;|Domain for the Kyuubi generated cookies|string|1.6.0
284+
kyuubi.frontend.thrift.http.cookie.is.httponly|true|HttpOnly attribute of the Kyuubi generated cookie.|boolean|1.6.0
285+
kyuubi.frontend.thrift.http.cookie.max.age|86400|Maximum age in seconds for server side cookie used by Kyuubi in HTTP mode.|int|1.6.0
286+
kyuubi.frontend.thrift.http.cookie.path|&lt;undefined&gt;|Path for the Kyuubi generated cookies|string|1.6.0
287+
kyuubi.frontend.thrift.http.max.idle.time|PT30M|Maximum idle time for a connection on the server when in HTTP mode.|duration|1.6.0
288+
kyuubi.frontend.thrift.http.path|cliservice|Path component of URL endpoint when in HTTP mode.|string|1.6.0
289+
kyuubi.frontend.thrift.http.request.header.size|6144|Request header size in bytes, when using HTTP transport mode. Jetty defaults used.|int|1.6.0
290+
kyuubi.frontend.thrift.http.response.header.size|6144|Response header size in bytes, when using HTTP transport mode. Jetty defaults used.|int|1.6.0
291+
kyuubi.frontend.thrift.http.ssl.keystore.password|&lt;undefined&gt;|SSL certificate keystore password.|string|1.6.0
292+
kyuubi.frontend.thrift.http.ssl.keystore.path|&lt;undefined&gt;|SSL certificate keystore location.|string|1.6.0
293+
kyuubi.frontend.thrift.http.ssl.protocol.blacklist|SSLv2,SSLv3|SSL Versions to disable when using HTTP transport mode.|string|1.6.0
294+
kyuubi.frontend.thrift.http.use.SSL|false|Set this to true for using SSL encryption in http mode.|boolean|1.6.0
295+
kyuubi.frontend.thrift.http.xsrf.filter.enabled|false|If enabled, Kyuubi will block any requests made to it over http if an X-XSRF-HEADER header is not present|boolean|1.6.0
278296
kyuubi.frontend.thrift.login.timeout|PT20S|Timeout for Thrift clients during login to the thrift frontend service.|duration|1.4.0
279297
kyuubi.frontend.thrift.max.message.size|104857600|Maximum message size in bytes a Kyuubi server will accept.|int|1.4.0
280298
kyuubi.frontend.thrift.max.worker.threads|999|Maximum number of threads in the of frontend worker thread pool for the thrift frontend service|int|1.4.0

‎kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala

Lines changed: 134 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -135,6 +135,8 @@ case class KyuubiConf(loadSysDefault: Boolean = true) extends Logging {
135135
FRONTEND_BIND_PORT,
136136
FRONTEND_THRIFT_BINARY_BIND_HOST,
137137
FRONTEND_THRIFT_BINARY_BIND_PORT,
138+
FRONTEND_THRIFT_HTTP_BIND_HOST,
139+
FRONTEND_THRIFT_HTTP_BIND_PORT,
138140
FRONTEND_REST_BIND_HOST,
139141
FRONTEND_REST_BIND_PORT,
140142
FRONTEND_MYSQL_BIND_HOST,
@@ -300,14 +302,15 @@ object KyuubiConf {
300302

301303
object FrontendProtocols extends Enumeration {
302304
type FrontendProtocol = Value
303-
val THRIFT_BINARY, REST, MYSQL = Value
305+
val THRIFT_BINARY, THRIFT_HTTP, REST, MYSQL = Value
304306
}
305307

306308
val FRONTEND_PROTOCOLS: ConfigEntry[Seq[String]] =
307309
buildConf("kyuubi.frontend.protocols")
308310
.doc("A comma separated list for all frontend protocols " +
309311
"<ul>" +
310312
" <li>THRIFT_BINARY - HiveServer2 compatible thrift binary protocol.</li>" +
313+
" <li>THRIFT_HTTP - HiveServer2 compatible thrift http protocol.</li>" +
311314
" <li>REST - Kyuubi defined REST API(experimental).</li> " +
312315
" <li>MYSQL - MySQL compatible text protocol(experimental).</li> " +
313316
"</ul>")
@@ -349,6 +352,21 @@ object KyuubiConf {
349352
.version("1.4.0")
350353
.fallbackConf(FRONTEND_BIND_PORT)
351354

355+
val FRONTEND_THRIFT_HTTP_BIND_HOST: ConfigEntry[Option[String]] =
356+
buildConf("kyuubi.frontend.thrift.http.bind.host")
357+
.doc("Hostname or IP of the machine on which to run the thrift frontend service " +
358+
"via http protocol.")
359+
.version("1.6.0")
360+
.fallbackConf(FRONTEND_BIND_HOST)
361+
362+
val FRONTEND_THRIFT_HTTP_BIND_PORT: ConfigEntry[Int] =
363+
buildConf("kyuubi.frontend.thrift.http.bind.port")
364+
.doc("Port of the machine on which to run the thrift frontend service via http protocol.")
365+
.version("1.6.0")
366+
.intConf
367+
.checkValue(p => p == 0 || (p > 1024 && p < 65535), "Invalid Port number")
368+
.createWithDefault(10010)
369+
352370
val FRONTEND_MIN_WORKER_THREADS: ConfigEntry[Int] =
353371
buildConf("kyuubi.frontend.min.worker.threads")
354372
.doc("(deprecated) Minimum number of threads in the of frontend worker thread pool for " +
@@ -434,6 +452,121 @@ object KyuubiConf {
434452
.version("1.4.0")
435453
.fallbackConf(FRONTEND_LOGIN_BACKOFF_SLOT_LENGTH)
436454

455+
val FRONTEND_THRIFT_HTTP_REQUEST_HEADER_SIZE: ConfigEntry[Int] =
456+
buildConf("kyuubi.frontend.thrift.http.request.header.size")
457+
.doc("Request header size in bytes, when using HTTP transport mode. Jetty defaults used.")
458+
.version("1.6.0")
459+
.intConf
460+
.createWithDefault(6 * 1024)
461+
462+
val FRONTEND_THRIFT_HTTP_RESPONSE_HEADER_SIZE: ConfigEntry[Int] =
463+
buildConf("kyuubi.frontend.thrift.http.response.header.size")
464+
.doc("Response header size in bytes, when using HTTP transport mode. Jetty defaults used.")
465+
.version("1.6.0")
466+
.intConf
467+
.createWithDefault(6 * 1024)
468+
469+
val FRONTEND_THRIFT_HTTP_MAX_IDLE_TIME: ConfigEntry[Long] =
470+
buildConf("kyuubi.frontend.thrift.http.max.idle.time")
471+
.doc("Maximum idle time for a connection on the server when in HTTP mode.")
472+
.version("1.6.0")
473+
.timeConf
474+
.createWithDefault(Duration.ofSeconds(1800).toMillis)
475+
476+
val FRONTEND_THRIFT_HTTP_PATH: ConfigEntry[String] =
477+
buildConf("kyuubi.frontend.thrift.http.path")
478+
.doc("Path component of URL endpoint when in HTTP mode.")
479+
.version("1.6.0")
480+
.stringConf
481+
.createWithDefault("cliservice")
482+
483+
val FRONTEND_THRIFT_HTTP_COMPRESSION_ENABLED: ConfigEntry[Boolean] =
484+
buildConf("kyuubi.frontend.thrift.http.compression.enabled")
485+
.doc("Enable thrift http compression via Jetty compression support")
486+
.version("1.6.0")
487+
.booleanConf
488+
.createWithDefault(true)
489+
490+
val FRONTEND_THRIFT_HTTP_COOKIE_AUTH_ENABLED: ConfigEntry[Boolean] =
491+
buildConf("kyuubi.frontend.thrift.http.cookie.auth.enabled")
492+
.doc("When true, Kyuubi in HTTP transport mode, " +
493+
"will use cookie based authentication mechanism")
494+
.version("1.6.0")
495+
.booleanConf
496+
.createWithDefault(true)
497+
498+
val FRONTEND_THRIFT_HTTP_COOKIE_MAX_AGE: ConfigEntry[Int] =
499+
buildConf("kyuubi.frontend.thrift.http.cookie.max.age")
500+
.doc("Maximum age in seconds for server side cookie used by Kyuubi in HTTP mode.")
501+
.version("1.6.0")
502+
.intConf
503+
.createWithDefault(86400)
504+
505+
val FRONTEND_THRIFT_HTTP_COOKIE_DOMAIN: OptionalConfigEntry[String] =
506+
buildConf("kyuubi.frontend.thrift.http.cookie.domain")
507+
.doc("Domain for the Kyuubi generated cookies")
508+
.version("1.6.0")
509+
.stringConf
510+
.createOptional
511+
512+
val FRONTEND_THRIFT_HTTP_COOKIE_PATH: OptionalConfigEntry[String] =
513+
buildConf("kyuubi.frontend.thrift.http.cookie.path")
514+
.doc("Path for the Kyuubi generated cookies")
515+
.version("1.6.0")
516+
.stringConf
517+
.createOptional
518+
519+
val FRONTEND_THRIFT_HTTP_COOKIE_IS_HTTPONLY: ConfigEntry[Boolean] =
520+
buildConf("kyuubi.frontend.thrift.http.cookie.is.httponly")
521+
.doc("HttpOnly attribute of the Kyuubi generated cookie.")
522+
.version("1.6.0")
523+
.booleanConf
524+
.createWithDefault(true)
525+
526+
val FRONTEND_THRIFT_HTTP_XSRF_FILTER_ENABLED: ConfigEntry[Boolean] =
527+
buildConf("kyuubi.frontend.thrift.http.xsrf.filter.enabled")
528+
.doc("If enabled, Kyuubi will block any requests made to it over http " +
529+
"if an X-XSRF-HEADER header is not present")
530+
.version("1.6.0")
531+
.booleanConf
532+
.createWithDefault(false)
533+
534+
val FRONTEND_THRIFT_HTTP_USE_SSL: ConfigEntry[Boolean] =
535+
buildConf("kyuubi.frontend.thrift.http.use.SSL")
536+
.doc("Set this to true for using SSL encryption in http mode.")
537+
.version("1.6.0")
538+
.booleanConf
539+
.createWithDefault(false)
540+
541+
val FRONTEND_THRIFT_HTTP_SSL_KEYSTORE_PATH: OptionalConfigEntry[String] =
542+
buildConf("kyuubi.frontend.thrift.http.ssl.keystore.path")
543+
.doc("SSL certificate keystore location.")
544+
.version("1.6.0")
545+
.stringConf
546+
.createOptional
547+
548+
val FRONTEND_THRIFT_HTTP_SSL_KEYSTORE_PASSWORD: OptionalConfigEntry[String] =
549+
buildConf("kyuubi.frontend.thrift.http.ssl.keystore.password")
550+
.doc("SSL certificate keystore password.")
551+
.version("1.6.0")
552+
.stringConf
553+
.createOptional
554+
555+
val FRONTEND_THRIFT_HTTP_SSL_PROTOCOL_BLACKLIST: ConfigEntry[String] =
556+
buildConf("kyuubi.frontend.thrift.http.ssl.protocol.blacklist")
557+
.doc("SSL Versions to disable when using HTTP transport mode.")
558+
.version("1.6.0")
559+
.stringConf
560+
.createWithDefault("SSLv2,SSLv3")
561+
562+
val FRONTEND_THRIFT_HTTP_ALLOW_USER_SUBSTITUTION: ConfigEntry[Boolean] =
563+
buildConf("kyuubi.frontend.thrift.http.allow.user.substitution")
564+
.doc("Allow alternate user to be specified as part of open connection" +
565+
" request when using HTTP transport mode.")
566+
.version("1.6.0")
567+
.booleanConf
568+
.createWithDefault(true)
569+
437570
val AUTHENTICATION_METHOD: ConfigEntry[Seq[String]] = buildConf("kyuubi.authentication")
438571
.doc("A comma separated list of client authentication types.<ul>" +
439572
" <li>NOSASL: raw transport.</li>" +

‎kyuubi-common/src/main/scala/org/apache/kyuubi/service/TBinaryFrontendService.scala

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,8 @@ abstract class TBinaryFrontendService(name: String)
4242
* @note this is final because we don't want new implementations for engine to override this.
4343
* and we shall simply set it to zero for randomly picking an available port
4444
*/
45+
final override protected lazy val serverHost: Option[String] =
46+
conf.get(FRONTEND_THRIFT_BINARY_BIND_HOST)
4547
final override protected lazy val portNum: Int = conf.get(FRONTEND_THRIFT_BINARY_BIND_PORT)
4648

4749
private var server: Option[TServer] = None

‎kyuubi-common/src/main/scala/org/apache/kyuubi/service/TFrontendService.scala

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ import org.apache.thrift.transport.TTransport
3131

3232
import org.apache.kyuubi.{KyuubiSQLException, Logging, Utils}
3333
import org.apache.kyuubi.Utils.stringifyException
34-
import org.apache.kyuubi.config.KyuubiConf.{FRONTEND_CONNECTION_URL_USE_HOSTNAME, FRONTEND_THRIFT_BINARY_BIND_HOST}
34+
import org.apache.kyuubi.config.KyuubiConf.FRONTEND_CONNECTION_URL_USE_HOSTNAME
3535
import org.apache.kyuubi.operation.{FetchOrientation, OperationHandle}
3636
import org.apache.kyuubi.service.authentication.KyuubiAuthenticationFactory
3737
import org.apache.kyuubi.session.SessionHandle
@@ -48,12 +48,13 @@ abstract class TFrontendService(name: String)
4848
private val started = new AtomicBoolean(false)
4949
private lazy val _hadoopConf: Configuration = KyuubiHadoopUtils.newHadoopConf(conf)
5050
private lazy val serverThread = new NamedThreadFactory(getName, false).newThread(this)
51-
private lazy val serverHost = conf.get(FRONTEND_THRIFT_BINARY_BIND_HOST)
5251

52+
protected def serverHost: Option[String]
5353
protected def portNum: Int
5454
protected lazy val serverAddr: InetAddress =
5555
serverHost.map(InetAddress.getByName).getOrElse(Utils.findLocalInetAddress)
5656
protected lazy val serverSocket = new ServerSocket(portNum, -1, serverAddr)
57+
protected lazy val actualPort: Int = serverSocket.getLocalPort
5758
protected lazy val authFactory: KyuubiAuthenticationFactory =
5859
new KyuubiAuthenticationFactory(conf, isServer())
5960

@@ -117,11 +118,11 @@ abstract class TFrontendService(name: String)
117118
case None =>
118119
serverAddr.getHostAddress
119120
}
120-
val actualPort = serverSocket.getLocalPort
121+
121122
host + ":" + actualPort
122123
}
123124

124-
private def getProxyUser(
125+
protected def getProxyUser(
125126
sessionConf: java.util.Map[String, String],
126127
ipAddress: String,
127128
realUser: String): String = {
@@ -134,7 +135,7 @@ abstract class TFrontendService(name: String)
134135
}
135136
}
136137

137-
private def getUserName(req: TOpenSessionReq): String = {
138+
protected def getUserName(req: TOpenSessionReq): String = {
138139
val realUser: String =
139140
ServiceUtils.getShortName(authFactory.getRemoteUser.getOrElse(req.getUsername))
140141
if (req.getConfiguration == null) {
@@ -143,6 +144,9 @@ abstract class TFrontendService(name: String)
143144
getProxyUser(req.getConfiguration, authFactory.getIpAddress.orNull, realUser)
144145
}
145146
}
147+
protected def getIpAddress: String = {
148+
authFactory.getIpAddress.orNull
149+
}
146150

147151
private def getMinVersion(versions: TProtocolVersion*): TProtocolVersion = {
148152
versions.minBy(_.getValue)
@@ -153,7 +157,7 @@ abstract class TFrontendService(name: String)
153157
val protocol = getMinVersion(SERVER_VERSION, req.getClient_protocol)
154158
res.setServerProtocolVersion(protocol)
155159
val userName = getUserName(req)
156-
val ipAddress = authFactory.getIpAddress.orNull
160+
val ipAddress = getIpAddress
157161
val configuration =
158162
Option(req.getConfiguration).map(_.asScala.toMap).getOrElse(Map.empty[String, String])
159163
val sessionHandle = be.openSession(

‎kyuubi-common/src/main/scala/org/apache/kyuubi/service/authentication/KyuubiAuthenticationFactory.scala

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,11 +32,13 @@ import org.apache.thrift.transport.{TSaslServerTransport, TTransportException, T
3232
import org.apache.kyuubi.{KyuubiSQLException, Logging}
3333
import org.apache.kyuubi.config.KyuubiConf
3434
import org.apache.kyuubi.config.KyuubiConf._
35+
import org.apache.kyuubi.service.authentication.AuthMethods.AuthMethod
3536
import org.apache.kyuubi.service.authentication.AuthTypes._
3637

3738
class KyuubiAuthenticationFactory(conf: KyuubiConf, isServer: Boolean = true) extends Logging {
3839

3940
private val authTypes = conf.get(AUTHENTICATION_METHOD).map(AuthTypes.withName)
41+
private val none = authTypes.contains(NONE)
4042
private val noSasl = authTypes == Seq(NOSASL)
4143
private val kerberosEnabled = authTypes.contains(KERBEROS)
4244
private val plainAuthTypeOpt = authTypes.filterNot(_.equals(KERBEROS))
@@ -117,6 +119,30 @@ class KyuubiAuthenticationFactory(conf: KyuubiConf, isServer: Boolean = true) ex
117119
hadoopAuthServer.map(_.getRemoteAddress).map(_.getHostAddress)
118120
.orElse(Option(TSetIpAddressProcessor.getUserIpAddress))
119121
}
122+
123+
def isNoSaslEnabled: Boolean = {
124+
noSasl
125+
}
126+
127+
def isKerberosEnabled: Boolean = {
128+
kerberosEnabled
129+
}
130+
131+
def isPlainAuthEnabled: Boolean = {
132+
plainAuthTypeOpt.isDefined
133+
}
134+
135+
def isNoneEnabled: Boolean = {
136+
none
137+
}
138+
139+
def getValidPasswordAuthMethod: AuthMethod = {
140+
debug(authTypes)
141+
if (none) AuthMethods.NONE
142+
else if (authTypes.contains(LDAP)) AuthMethods.LDAP
143+
else if (authTypes.contains(CUSTOM)) AuthMethods.CUSTOM
144+
else throw new IllegalArgumentException("No valid Password Auth detected")
145+
}
120146
}
121147
object KyuubiAuthenticationFactory {
122148
val HS2_PROXY_USER = "hive.server2.proxy.user"

‎kyuubi-common/src/test/scala/org/apache/kyuubi/config/KyuubiConfSuite.scala

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,9 @@ class KyuubiConfSuite extends KyuubiFunSuite {
5858
conf.set(FRONTEND_THRIFT_BINARY_BIND_HOST.key, "kentyao.org")
5959
assert(conf.get(FRONTEND_THRIFT_BINARY_BIND_HOST).get === "kentyao.org")
6060

61+
conf.set(FRONTEND_THRIFT_HTTP_BIND_HOST.key, "kentyao.org")
62+
assert(conf.get(FRONTEND_THRIFT_HTTP_BIND_HOST).get === "kentyao.org")
63+
6164
conf.setIfMissing(OPERATION_IDLE_TIMEOUT, 60L)
6265
assert(conf.get(OPERATION_IDLE_TIMEOUT) === 5)
6366

@@ -72,6 +75,7 @@ class KyuubiConfSuite extends KyuubiFunSuite {
7275

7376
val map = conf.getAllWithPrefix("kyuubi", "")
7477
assert(map(FRONTEND_THRIFT_BINARY_BIND_HOST.key.substring(7)) === "kentyao.org")
78+
assert(map(FRONTEND_THRIFT_HTTP_BIND_HOST.key.substring(7)) === "kentyao.org")
7579
val map1 = conf.getAllWithPrefix("kyuubi", "operation")
7680
assert(map1(OPERATION_IDLE_TIMEOUT.key.substring(7)) === "PT0.005S")
7781
assert(map1.size === 1)

‎kyuubi-common/src/test/scala/org/apache/kyuubi/operation/SparkQueryTests.scala

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,8 @@ trait SparkQueryTests extends HiveJDBCTestHelper {
3232

3333
protected lazy val SPARK_ENGINE_MAJOR_MINOR_VERSION: (Int, Int) = sparkEngineMajorMinorVersion
3434

35+
protected lazy val httpMode = false;
36+
3537
test("execute statement - select null") {
3638
withJdbcStatement() { statement =>
3739
val resultSet = statement.executeQuery("SELECT NULL AS col")
@@ -332,6 +334,8 @@ trait SparkQueryTests extends HiveJDBCTestHelper {
332334
}
333335

334336
test("execute statement - select with variable substitution") {
337+
assume(!httpMode)
338+
335339
withThriftClient { client =>
336340
val req = new TOpenSessionReq()
337341
req.setUsername("chengpan")

‎kyuubi-metrics/src/main/scala/org/apache/kyuubi/metrics/MetricsConstants.scala

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,11 +31,16 @@ object MetricsConstants {
3131
final val EXEC_POOL_ACTIVE: String = KYUUBI + "exec.pool.threads.active"
3232

3333
final private val CONN = KYUUBI + "connection."
34+
final private val THRIFT_HTTP_CONN = KYUUBI + "thrift.http.connection."
3435

3536
final val CONN_OPEN: String = CONN + "opened"
3637
final val CONN_FAIL: String = CONN + "failed"
3738
final val CONN_TOTAL: String = CONN + "total"
3839

40+
final val THRIFT_HTTP_CONN_OPEN: String = THRIFT_HTTP_CONN + "opened"
41+
final val THRIFT_HTTP_CONN_FAIL: String = THRIFT_HTTP_CONN + "failed"
42+
final val THRIFT_HTTP_CONN_TOTAL: String = THRIFT_HTTP_CONN + "total"
43+
3944
final private val ENGINE = KYUUBI + "engine."
4045
final val ENGINE_FAIL: String = ENGINE + "failed"
4146
final val ENGINE_TIMEOUT: String = ENGINE + "timeout"

‎kyuubi-server/src/main/scala/org/apache/kyuubi/server/KyuubiServer.scala

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -117,6 +117,7 @@ class KyuubiServer(name: String) extends Serverable(name) {
117117
override lazy val frontendServices: Seq[AbstractFrontendService] =
118118
conf.get(FRONTEND_PROTOCOLS).map(FrontendProtocols.withName).map {
119119
case THRIFT_BINARY => new KyuubiTBinaryFrontendService(this)
120+
case THRIFT_HTTP => new KyuubiTHttpFrontendService(this)
120121
case REST =>
121122
warn("REST frontend protocol is experimental, API may change in the future.")
122123
new KyuubiRestFrontendService(this)

0 commit comments

Comments
 (0)