[KYUUBI #2719] [SUB-TASK][KPIP-4] Support internal rest request authentication to enable http request redirection across kyuubi instances

turboFei · turboFei · commit f1cf95fe960f · 2022-05-23T15:44:48.000+08:00
### _Why are the changes needed?_ Support internal rest request authentication to enable redirect http request across kyuubi instances ### _How was this patch tested?_ - [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #2719 from turboFei/internal_access. Closes #2719 b9e150e [Fei Wang] revert config key change aa37872 [Fei Wang] make it internal ce96d92 [Fei Wang] comments 6fe8523 [Fei Wang] Support internal rest request authentication to enable redirect http request across kyuubi instances Authored-by: Fei Wang <fwang12@ebay.com> Signed-off-by: Fei Wang <fwang12@ebay.com>
diff --git a/docs/deployment/settings.md b/docs/deployment/settings.md
@@ -220,13 +220,6 @@ Key | Default | Meaning | Type | Since
 <code>kyuubi.engine.pool.name</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>engine-pool</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>The name of engine pool.</div>|<div style='width: 30pt'>string</div>|<div style='width: 20pt'>1.5.0</div>
 <code>kyuubi.engine.pool.size</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>-1</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>The size of engine pool. Note that, if the size is less than 1, the engine pool will not be enabled; otherwise, the size of the engine pool will be min(this, kyuubi.engine.pool.size.threshold).</div>|<div style='width: 30pt'>int</div>|<div style='width: 20pt'>1.4.0</div>
 <code>kyuubi.engine.pool.size.threshold</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>9</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>This parameter is introduced as a server-side parameter, and controls the upper limit of the engine pool.</div>|<div style='width: 30pt'>int</div>|<div style='width: 20pt'>1.4.0</div>
-<code>kyuubi.engine.security.crypto.cipher</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>AES/CBC/PKCS5PADDING</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>The cipher transformation to use for encrypting engine access token.</div>|<div style='width: 30pt'>string</div>|<div style='width: 20pt'>1.5.0</div>
-<code>kyuubi.engine.security.crypto.ivLength</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>16</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>Initial vector length, in bytes.</div>|<div style='width: 30pt'>int</div>|<div style='width: 20pt'>1.5.0</div>
-<code>kyuubi.engine.security.crypto.keyAlgorithm</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>AES</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>The algorithm for generated secret keys.</div>|<div style='width: 30pt'>string</div>|<div style='width: 20pt'>1.5.0</div>
-<code>kyuubi.engine.security.crypto.keyLength</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>128</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>The length in bits of the encryption key to generate. Valid values are 128, 192 and 256</div>|<div style='width: 30pt'>int</div>|<div style='width: 20pt'>1.5.0</div>
-<code>kyuubi.engine.security.enabled</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>false</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>Whether to enable the internal secure access between Kyuubi server and engine.</div>|<div style='width: 30pt'>boolean</div>|<div style='width: 20pt'>1.5.0</div>
-<code>kyuubi.engine.security.secret.provider</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>org.apache.kyuubi.service.authentication.ZooKeeperEngineSecuritySecretProviderImpl</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>The class used to manage the engine security secret. This class must be a subclass of EngineSecuritySecretProvider.</div>|<div style='width: 30pt'>string</div>|<div style='width: 20pt'>1.5.0</div>
-<code>kyuubi.engine.security.token.max.lifetime</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>PT10M</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>The max lifetime of the token used for secure access between Kyuubi server and engine.</div>|<div style='width: 30pt'>duration</div>|<div style='width: 20pt'>1.5.0</div>
 <code>kyuubi.engine.session.initialize.sql</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'></div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>SemiColon-separated list of SQL statements to be initialized in the newly created engine session before queries. This configuration can not be used in JDBC url due to the limitation of Beeline/JDBC driver.</div>|<div style='width: 30pt'>seq</div>|<div style='width: 20pt'>1.3.0</div>
 <code>kyuubi.engine.share.level</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>USER</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>Engines will be shared in different levels, available configs are: <ul> <li>CONNECTION: engine will not be shared but only used by the current client connection</li> <li>USER: engine will be shared by all sessions created by a unique username, see also kyuubi.engine.share.level.subdomain</li> <li>GROUP: engine will be shared by all sessions created by all users belong to the same primary group name. The engine will be launched by the group name as the effective username, so here the group name is kind of special user who is able to visit the compute resources/data of a team. It follows the [Hadoop GroupsMapping](https://reurl.cc/xE61Y5) to map user to a primary group. If the primary group is not found, it fallback to the USER level. <li>SERVER: the App will be shared by Kyuubi servers</li></ul></div>|<div style='width: 30pt'>string</div>|<div style='width: 20pt'>1.2.0</div>
 <code>kyuubi.engine.share.level.sub.domain</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>&lt;undefined&gt;</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>(deprecated) - Using kyuubi.engine.share.level.subdomain instead</div>|<div style='width: 30pt'>string</div>|<div style='width: 20pt'>1.2.0</div>
@@ -292,7 +285,6 @@ Key | Default | Meaning | Type | Since
 <code>kyuubi.ha.zookeeper.connection.retry.policy</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>EXPONENTIAL_BACKOFF</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>The retry policy for connecting to the zookeeper ensemble, all candidates are: <ul><li>ONE_TIME</li><li> N_TIME</li><li> EXPONENTIAL_BACKOFF</li><li> BOUNDED_EXPONENTIAL_BACKOFF</li><li> UNTIL_ELAPSED</li></ul></div>|<div style='width: 30pt'>string</div>|<div style='width: 20pt'>1.0.0</div>
 <code>kyuubi.ha.zookeeper.connection.timeout</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>15000</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>The timeout(ms) of creating the connection to the zookeeper ensemble</div>|<div style='width: 30pt'>int</div>|<div style='width: 20pt'>1.0.0</div>
 <code>kyuubi.ha.zookeeper.engine.auth.type</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>NONE</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>The type of zookeeper authentication for engine, all candidates are <ul><li>NONE</li><li> KERBEROS</li><li> DIGEST</li></ul></div>|<div style='width: 30pt'>string</div>|<div style='width: 20pt'>1.3.2</div>
-<code>kyuubi.ha.zookeeper.engine.secure.secret.node</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>&lt;undefined&gt;</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>The zk node contains the secret that used for internal secure between Kyuubi server and Kyuubi engine, please make sure that it is only visible for Kyuubi.</div>|<div style='width: 30pt'>string</div>|<div style='width: 20pt'>1.5.0</div>
 <code>kyuubi.ha.zookeeper.namespace</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>kyuubi</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>The root directory for the service to deploy its instance uri</div>|<div style='width: 30pt'>string</div>|<div style='width: 20pt'>1.0.0</div>
 <code>kyuubi.ha.zookeeper.node.creation.timeout</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>PT2M</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>Timeout for creating zookeeper node</div>|<div style='width: 30pt'>duration</div>|<div style='width: 20pt'>1.2.0</div>
 <code>kyuubi.ha.zookeeper.publish.configs</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>false</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>When set to true, publish Kerberos configs to Zookeeper.Note that the Hive driver needs to be greater than 1.3 or 2.0 or apply HIVE-11581 patch.</div>|<div style='width: 30pt'>boolean</div>|<div style='width: 20pt'>1.4.0</div>
diff --git a/kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala b/kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
@@ -1250,21 +1250,26 @@ object KyuubiConf {
 
   val ENGINE_SECURITY_ENABLED: ConfigEntry[Boolean] =
     buildConf("kyuubi.engine.security.enabled")
-      .doc("Whether to enable the internal secure access between Kyuubi server and engine.")
+      .internal
+      .doc("Whether to enable the internal secure access. Before 1.6.0, it is used for the secure" +
+        " access between kyuubi server and kyuubi engine. Since 1.6.0, kyuubi supports internal" +
+        " secure across kyuubi server instances.")
       .version("1.5.0")
       .booleanConf
       .createWithDefault(false)
 
   val ENGINE_SECURITY_TOKEN_MAX_LIFETIME: ConfigEntry[Long] =
     buildConf("kyuubi.engine.security.token.max.lifetime")
-      .doc("The max lifetime of the token used for secure access between Kyuubi server and engine.")
+      .internal
+      .doc("The max lifetime of the token used for internal secure access.")
       .version("1.5.0")
       .timeConf
       .createWithDefault(Duration.ofMinutes(10).toMillis)
 
   val ENGINE_SECURITY_SECRET_PROVIDER: ConfigEntry[String] =
     buildConf("kyuubi.engine.security.secret.provider")
-      .doc("The class used to manage the engine security secret. This class must be a " +
+      .internal
+      .doc("The class used to manage the internal security secret. This class must be a " +
         "subclass of EngineSecuritySecretProvider.")
       .version("1.5.0")
       .stringConf
@@ -1273,6 +1278,7 @@ object KyuubiConf {
 
   val ENGINE_SECURITY_CRYPTO_KEY_LENGTH: ConfigEntry[Int] =
     buildConf("kyuubi.engine.security.crypto.keyLength")
+      .internal
       .doc("The length in bits of the encryption key to generate. " +
         "Valid values are 128, 192 and 256")
       .version("1.5.0")
@@ -1282,21 +1288,24 @@ object KyuubiConf {
 
   val ENGINE_SECURITY_CRYPTO_IV_LENGTH: ConfigEntry[Int] =
     buildConf("kyuubi.engine.security.crypto.ivLength")
+      .internal
       .doc("Initial vector length, in bytes.")
       .version("1.5.0")
       .intConf
       .createWithDefault(16)
 
   val ENGINE_SECURITY_CRYPTO_KEY_ALGORITHM: ConfigEntry[String] =
     buildConf("kyuubi.engine.security.crypto.keyAlgorithm")
+      .internal
       .doc("The algorithm for generated secret keys.")
       .version("1.5.0")
       .stringConf
       .createWithDefault("AES")
 
   val ENGINE_SECURITY_CRYPTO_CIPHER_TRANSFORMATION: ConfigEntry[String] =
     buildConf("kyuubi.engine.security.crypto.cipher")
-      .doc("The cipher transformation to use for encrypting engine access token.")
+      .internal
+      .doc("The cipher transformation to use for encrypting internal access token.")
       .version("1.5.0")
       .stringConf
       .createWithDefault("AES/CBC/PKCS5PADDING")
diff --git a/kyuubi-common/src/main/scala/org/apache/kyuubi/service/authentication/EngineSecureAuthenticationProviderImpl.scala b/kyuubi-common/src/main/scala/org/apache/kyuubi/service/authentication/EngineSecureAuthenticationProviderImpl.scala
@@ -19,6 +19,6 @@ package org.apache.kyuubi.service.authentication
 
 class EngineSecureAuthenticationProviderImpl extends PasswdAuthenticationProvider {
   override def authenticate(user: String, password: String): Unit = {
-    EngineSecurityAccessor.get().authToken(password)
+    InternalSecurityAccessor.get().authToken(password)
   }
 }
diff --git a/kyuubi-common/src/main/scala/org/apache/kyuubi/service/authentication/InternalSecurityAccessor.scala b/kyuubi-common/src/main/scala/org/apache/kyuubi/service/authentication/InternalSecurityAccessor.scala
@@ -24,7 +24,7 @@ import org.apache.kyuubi.{KyuubiSQLException, Logging}
 import org.apache.kyuubi.config.KyuubiConf
 import org.apache.kyuubi.config.KyuubiConf._
 
-class EngineSecurityAccessor(conf: KyuubiConf, val isServer: Boolean) {
+class InternalSecurityAccessor(conf: KyuubiConf, val isServer: Boolean) {
   val cryptoKeyLengthBytes = conf.get(ENGINE_SECURITY_CRYPTO_KEY_LENGTH) / java.lang.Byte.SIZE
   val cryptoIvLength = conf.get(ENGINE_SECURITY_CRYPTO_IV_LENGTH)
   val cryptoKeyAlgorithm = conf.get(ENGINE_SECURITY_CRYPTO_KEY_ALGORITHM)
@@ -109,16 +109,16 @@ class EngineSecurityAccessor(conf: KyuubiConf, val isServer: Boolean) {
   }
 }
 
-object EngineSecurityAccessor extends Logging {
-  @volatile private var _engineSecurityAccessor: EngineSecurityAccessor = _
+object InternalSecurityAccessor extends Logging {
+  @volatile private var _engineSecurityAccessor: InternalSecurityAccessor = _
 
   def initialize(conf: KyuubiConf, isServer: Boolean): Unit = {
     if (_engineSecurityAccessor == null) {
-      _engineSecurityAccessor = new EngineSecurityAccessor(conf, isServer)
+      _engineSecurityAccessor = new InternalSecurityAccessor(conf, isServer)
     }
   }
 
-  def get(): EngineSecurityAccessor = {
+  def get(): InternalSecurityAccessor = {
     _engineSecurityAccessor
   }
 }
diff --git a/kyuubi-common/src/main/scala/org/apache/kyuubi/service/authentication/KyuubiAuthenticationFactory.scala b/kyuubi-common/src/main/scala/org/apache/kyuubi/service/authentication/KyuubiAuthenticationFactory.scala
@@ -57,7 +57,7 @@ class KyuubiAuthenticationFactory(conf: KyuubiConf, isServer: Boolean = true) ex
   }
 
   if (conf.get(ENGINE_SECURITY_ENABLED)) {
-    EngineSecurityAccessor.initialize(conf, isServer)
+    InternalSecurityAccessor.initialize(conf, isServer)
   }
 
   private def getSaslProperties: java.util.Map[String, String] = {
diff --git a/kyuubi-common/src/test/scala/org/apache/kyuubi/service/authentication/InternalSecurityAccessorSuite.scala b/kyuubi-common/src/test/scala/org/apache/kyuubi/service/authentication/InternalSecurityAccessorSuite.scala
@@ -20,7 +20,7 @@ package org.apache.kyuubi.service.authentication
 import org.apache.kyuubi.{KyuubiFunSuite, KyuubiSQLException}
 import org.apache.kyuubi.config.KyuubiConf
 
-class EngineSecurityAccessorSuite extends KyuubiFunSuite {
+class InternalSecurityAccessorSuite extends KyuubiFunSuite {
   private val conf = KyuubiConf()
   conf.set(
     KyuubiConf.ENGINE_SECURITY_SECRET_PROVIDER,
@@ -31,7 +31,7 @@ class EngineSecurityAccessorSuite extends KyuubiFunSuite {
       val newConf = conf.clone
       newConf.set(KyuubiConf.ENGINE_SECURITY_CRYPTO_CIPHER_TRANSFORMATION, cipher)
 
-      val secureAccessor = new EngineSecurityAccessor(newConf, true)
+      val secureAccessor = new InternalSecurityAccessor(newConf, true)
       val value = "tokenToEncrypt"
       val encryptedValue = secureAccessor.encrypt(value)
       assert(secureAccessor.decrypt(encryptedValue) === value)
@@ -40,7 +40,7 @@ class EngineSecurityAccessorSuite extends KyuubiFunSuite {
       secureAccessor.authToken(token)
       intercept[KyuubiSQLException](secureAccessor.authToken("invalidToken"))
 
-      val engineSecureAccessor = new EngineSecurityAccessor(newConf, false)
+      val engineSecureAccessor = new InternalSecurityAccessor(newConf, false)
       engineSecureAccessor.authToken(token)
     }
   }
diff --git a/kyuubi-ha/src/main/scala/org/apache/kyuubi/ha/HighAvailabilityConf.scala b/kyuubi-ha/src/main/scala/org/apache/kyuubi/ha/HighAvailabilityConf.scala
@@ -153,8 +153,9 @@ object HighAvailabilityConf {
 
   val HA_ZK_ENGINE_SECURE_SECRET_NODE: OptionalConfigEntry[String] =
     buildConf("kyuubi.ha.zookeeper.engine.secure.secret.node")
-      .doc("The zk node contains the secret that used for internal secure between Kyuubi server " +
-        "and Kyuubi engine, please make sure that it is only visible for Kyuubi.")
+      .internal
+      .doc("The zk node contains the secret that used for internal secure, please make sure " +
+        "that it is only visible for Kyuubi.")
       .version("1.5.0")
       .stringConf
       .createOptional
diff --git a/kyuubi-server/src/main/scala/org/apache/kyuubi/server/http/authentication/AuthSchemes.scala b/kyuubi-server/src/main/scala/org/apache/kyuubi/server/http/authentication/AuthSchemes.scala
@@ -20,5 +20,5 @@ package org.apache.kyuubi.server.http.authentication
 object AuthSchemes extends Enumeration {
   type AuthScheme = Value
 
-  val BASIC, NEGOTIATE = Value
+  val BASIC, NEGOTIATE, KYUUBI_INTERNAL = Value
 }
diff --git a/kyuubi-server/src/main/scala/org/apache/kyuubi/server/http/authentication/AuthenticationFilter.scala b/kyuubi-server/src/main/scala/org/apache/kyuubi/server/http/authentication/AuthenticationFilter.scala
@@ -28,7 +28,7 @@ import org.apache.hadoop.security.authentication.client.AuthenticationException
 import org.apache.kyuubi.Logging
 import org.apache.kyuubi.config.KyuubiConf
 import org.apache.kyuubi.config.KyuubiConf.AUTHENTICATION_METHOD
-import org.apache.kyuubi.service.authentication.AuthTypes
+import org.apache.kyuubi.service.authentication.{AuthTypes, InternalSecurityAccessor}
 import org.apache.kyuubi.service.authentication.AuthTypes.{KERBEROS, NOSASL}
 
 class AuthenticationFilter(conf: KyuubiConf) extends Filter with Logging {
@@ -72,6 +72,10 @@ class AuthenticationFilter(conf: KyuubiConf) extends Filter with Logging {
       val basicHandler = new BasicAuthenticationHandler(basicAuthType)
       addAuthHandler(basicHandler)
     }
+    if (InternalSecurityAccessor.get() != null) {
+      val internalHandler = new KyuubiInternalAuthenticationHandler
+      addAuthHandler(internalHandler)
+    }
     super.init(filterConfig)
   }
 
diff --git a/kyuubi-server/src/main/scala/org/apache/kyuubi/server/http/authentication/KyuubiHttpAuthenticationFactory.scala b/kyuubi-server/src/main/scala/org/apache/kyuubi/server/http/authentication/KyuubiHttpAuthenticationFactory.scala
@@ -26,7 +26,7 @@ import org.eclipse.jetty.server.handler.HandlerWrapper
 
 import org.apache.kyuubi.config.KyuubiConf
 import org.apache.kyuubi.config.KyuubiConf.{AUTHENTICATION_METHOD, ENGINE_SECURITY_ENABLED}
-import org.apache.kyuubi.service.authentication.{AuthTypes, EngineSecurityAccessor}
+import org.apache.kyuubi.service.authentication.{AuthTypes, InternalSecurityAccessor}
 import org.apache.kyuubi.service.authentication.AuthTypes.KERBEROS
 
 class KyuubiHttpAuthenticationFactory(conf: KyuubiConf) {
@@ -35,7 +35,7 @@ class KyuubiHttpAuthenticationFactory(conf: KyuubiConf) {
   private val ugi = UserGroupInformation.getCurrentUser
 
   if (conf.get(ENGINE_SECURITY_ENABLED)) {
-    EngineSecurityAccessor.initialize(conf, true)
+    InternalSecurityAccessor.initialize(conf, true)
   }
 
   private[kyuubi] val httpHandlerWrapperFactory =
diff --git a/kyuubi-server/src/main/scala/org/apache/kyuubi/server/http/authentication/KyuubiInternalAuthenticationHandler.scala b/kyuubi-server/src/main/scala/org/apache/kyuubi/server/http/authentication/KyuubiInternalAuthenticationHandler.scala
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kyuubi.server.http.authentication
+
+import java.nio.charset.Charset
+import java.util.Base64
+import javax.servlet.http.{HttpServletRequest, HttpServletResponse}
+
+import org.apache.kyuubi.Logging
+import org.apache.kyuubi.config.KyuubiConf
+import org.apache.kyuubi.server.http.authentication.AuthSchemes.AuthScheme
+import org.apache.kyuubi.service.authentication.InternalSecurityAccessor
+
+class KyuubiInternalAuthenticationHandler extends AuthenticationHandler with Logging {
+  import AuthenticationHandler._
+
+  private var conf: KyuubiConf = _
+  override val authScheme: AuthScheme = AuthSchemes.KYUUBI_INTERNAL
+  private val internalSecurityAccessor = InternalSecurityAccessor.get()
+
+  override def init(conf: KyuubiConf): Unit = {
+    this.conf = conf
+  }
+
+  override def authenticationSupported: Boolean = {
+    internalSecurityAccessor != null
+  }
+
+  override def authenticate(
+      request: HttpServletRequest,
+      response: HttpServletResponse): String = {
+    var authUser: String = null
+    val authorization = getAuthorization(request)
+    val inputToken = Option(authorization).map(a => Base64.getDecoder.decode(a.getBytes()))
+      .getOrElse(Array.empty[Byte])
+    val creds = new String(inputToken, Charset.forName("UTF-8")).split(":")
+
+    if (creds.size < 2 || creds(0).trim.isEmpty || creds(1).trim.isEmpty) {
+      response.setHeader(WWW_AUTHENTICATE, authScheme.toString)
+      response.setStatus(HttpServletResponse.SC_UNAUTHORIZED)
+    } else {
+      val Seq(user, password) = creds.toSeq.take(2)
+      internalSecurityAccessor.authToken(password)
+      response.setStatus(HttpServletResponse.SC_OK)
+      authUser = user
+    }
+    authUser
+  }
+
+  override def destroy(): Unit = {}
+}
diff --git a/kyuubi-server/src/main/scala/org/apache/kyuubi/session/KyuubiSessionImpl.scala b/kyuubi-server/src/main/scala/org/apache/kyuubi/session/KyuubiSessionImpl.scala
diff --git a/kyuubi-server/src/test/scala/org/apache/kyuubi/operation/KyuubiOperationWithEngineSecurity.scala b/kyuubi-server/src/test/scala/org/apache/kyuubi/operation/KyuubiOperationWithEngineSecurity.scala
diff --git a/kyuubi-server/src/test/scala/org/apache/kyuubi/operation/KyuubiRestAuthenticationSuite.scala b/kyuubi-server/src/test/scala/org/apache/kyuubi/operation/KyuubiRestAuthenticationSuite.scala

Original file line number	Diff line number	Diff line change
`@@ -19,6 +19,6 @@ package org.apache.kyuubi.service.authentication`
`19`	`19`
`20`	`20`	`class EngineSecureAuthenticationProviderImpl extends PasswdAuthenticationProvider {`
`21`	`21`	`override def authenticate(user: String, password: String): Unit = {`
`22`		`- EngineSecurityAccessor.get().authToken(password)`
	`22`	`+ InternalSecurityAccessor.get().authToken(password)`
`23`	`23`	`}`
`24`	`24`	`}`
Original file line number	Diff line number	Diff line change
`@@ -24,7 +24,7 @@ import org.apache.kyuubi.{KyuubiSQLException, Logging}`
`24`	`24`	`import org.apache.kyuubi.config.KyuubiConf`
`25`	`25`	`import org.apache.kyuubi.config.KyuubiConf._`
`26`	`26`
`27`		`-class EngineSecurityAccessor(conf: KyuubiConf, val isServer: Boolean) {`
	`27`	`+class InternalSecurityAccessor(conf: KyuubiConf, val isServer: Boolean) {`
`28`	`28`	`val cryptoKeyLengthBytes = conf.get(ENGINE_SECURITY_CRYPTO_KEY_LENGTH) / java.lang.Byte.SIZE`
`29`	`29`	`val cryptoIvLength = conf.get(ENGINE_SECURITY_CRYPTO_IV_LENGTH)`
`30`	`30`	`val cryptoKeyAlgorithm = conf.get(ENGINE_SECURITY_CRYPTO_KEY_ALGORITHM)`
`@@ -109,16 +109,16 @@ class EngineSecurityAccessor(conf: KyuubiConf, val isServer: Boolean) {`
`109`	`109`	`}`
`110`	`110`	`}`
`111`	`111`
`112`		`-object EngineSecurityAccessor extends Logging {`
`113`		`- @volatile private var _engineSecurityAccessor: EngineSecurityAccessor = _`
	`112`	`+object InternalSecurityAccessor extends Logging {`
	`113`	`+ @volatile private var _engineSecurityAccessor: InternalSecurityAccessor = _`
`114`	`114`
`115`	`115`	`def initialize(conf: KyuubiConf, isServer: Boolean): Unit = {`
`116`	`116`	`if (_engineSecurityAccessor == null) {`
`117`		`- _engineSecurityAccessor = new EngineSecurityAccessor(conf, isServer)`
	`117`	`+ _engineSecurityAccessor = new InternalSecurityAccessor(conf, isServer)`
`118`	`118`	`}`
`119`	`119`	`}`
`120`	`120`
`121`		`- def get(): EngineSecurityAccessor = {`
	`121`	`+ def get(): InternalSecurityAccessor = {`
`122`	`122`	`_engineSecurityAccessor`
`123`	`123`	`}`
`124`	`124`	`}`
Original file line number	Diff line number	Diff line change
`@@ -57,7 +57,7 @@ class KyuubiAuthenticationFactory(conf: KyuubiConf, isServer: Boolean = true) ex`
`57`	`57`	`}`
`58`	`58`
`59`	`59`	`if (conf.get(ENGINE_SECURITY_ENABLED)) {`
`60`		`- EngineSecurityAccessor.initialize(conf, isServer)`
	`60`	`+ InternalSecurityAccessor.initialize(conf, isServer)`
`61`	`61`	`}`
`62`	`62`
`63`	`63`	`private def getSaslProperties: java.util.Map[String, String] = {`
Original file line number	Diff line number	Diff line change
`@@ -20,5 +20,5 @@ package org.apache.kyuubi.server.http.authentication`
`20`	`20`	`object AuthSchemes extends Enumeration {`
`21`	`21`	`type AuthScheme = Value`
`22`	`22`
`23`		`- val BASIC, NEGOTIATE = Value`
	`23`	`+ val BASIC, NEGOTIATE, KYUUBI_INTERNAL = Value`
`24`	`24`	`}`