Skip to content

Commit a63587b

Browse files
bowenliang123pan3793
authored andcommitted
[KYUUBI #3222] JDBC Authentication Provider for server
### _Why are the changes needed?_ Add JDBC authentication provider as implementation of PasswdAuthenticationProvider out of box. Compared to currently support authentication methods like Kerberos and LDAP, JDBC source is a much easy and stable source for practical deployment. The solution should provide: easy to use and config jdbc connection details handy to customize query statements for authentication Adding JDBC to AuthMethods enum and JDBCPasswdAuthenticationProvider into package org.apache.kyuubi.service.authentication with listed features: - specify JDBC driver name and load the driver class - configs of JDBC url, username and password - select query with placeholders for checking user and password , like `SELECT 1 from user_pass_hash where username = ${user} and password = MD5(${password})` ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #3235 from bowenliang123/feature-jdbc-auth-provider. Closes #3222 17403b3 [liangbowen] cleanup docs d5f43e0 [liangbowen] remove unuseful logs for unrecognized placeholder error e9af096 [liangbowen] use clone instead of repeatly generating configs 6fc42bf [liangbowen] code styling a9404fa [liangbowen] use {} for intercept 77f5f86 [liangbowen] remove unuseful comment 6765aff [liangbowen] changed to use in-memory derby db for test 543c66c [liangbowen] prefer scala style string usage a4fe582 [liangbowen] refactor connection creation on using HikariDataSource in HikariCP. add HikariCP dependencies to kyuubi-common 3a4d5fe [liangbowen] update checkConfigs() signature 5a0ac49 [liangbowen] output password length only in checkConfigs 1c956df [liangbowen] update KyuubiAuthenticationFactorySuite 4ebe12e [liangbowen] add JDBC value to AuthTypes enum 9885f81 [liangbowen] add JDBC condition for getValidPasswordAuthMethod b9ffac3 [liangbowen] Merge branch 'master' into feature-jdbc-auth-provider aeb19ce [liangbowen] update doc 653bc12 [liangbowen] add more checks for query sql cdec206 [liangbowen] more test cases 3672919 [liangbowen] fix ddl statement and remove truncate statement in test 30974d1 [liangbowen] update format 575301c [liangbowen] update options usage 1dc4187 [liangbowen] update settings.md config doc cd2c7c2 [liangbowen] update settings.md config doc 15176b2 [liangbowen] fix import orders 46cc1dd [liangbowen] add config docs in docs/deployment/settings.md 7025330 [liangbowen] fix derby startup error in test df4be56 [liangbowen] update code style 49c18c2 [liangbowen] update 996f796 [liangbowen] add unit test in JdbcAuthenticationProviderImplSuite 0e7f0ad [liangbowen] refactor config and init process.remove unused import. 0dc75fe [liangbowen] implement JDBC Authentication Method Authored-by: liangbowen <liangbowen@gf.com.cn> Signed-off-by: Cheng Pan <chengpan@apache.org>
1 parent 8c2e774 commit a63587b

File tree

10 files changed

+419
-5
lines changed

10 files changed

+419
-5
lines changed

docs/deployment/settings.md

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -136,8 +136,13 @@ You can configure the Kyuubi properties in `$KYUUBI_HOME/conf/kyuubi-defaults.co
136136

137137
Key | Default | Meaning | Type | Since
138138
--- | --- | --- | --- | ---
139-
kyuubi.authentication|NONE|A comma separated list of client authentication types.<ul> <li>NOSASL: raw transport.</li> <li>NONE: no authentication check.</li> <li>KERBEROS: Kerberos/GSSAPI authentication.</li> <li>CUSTOM: User-defined authentication.</li> <li>LDAP: Lightweight Directory Access Protocol authentication.</li></ul> Note that: For KERBEROS, it is SASL/GSSAPI mechanism, and for NONE, CUSTOM and LDAP, they are all SASL/PLAIN mechanism. If only NOSASL is specified, the authentication will be NOSASL. For SASL authentication, KERBEROS and PLAIN auth type are supported at the same time, and only the first specified PLAIN auth type is valid.|seq|1.0.0
139+
kyuubi.authentication|NONE|A comma separated list of client authentication types.<ul> <li>NOSASL: raw transport.</li> <li>NONE: no authentication check.</li> <li>KERBEROS: Kerberos/GSSAPI authentication.</li> <li>CUSTOM: User-defined authentication.</li> <li>JDBC: JDBC query authentication.</li> <li>LDAP: Lightweight Directory Access Protocol authentication.</li></ul> Note that: For KERBEROS, it is SASL/GSSAPI mechanism, and for NONE, CUSTOM and LDAP, they are all SASL/PLAIN mechanism. If only NOSASL is specified, the authentication will be NOSASL. For SASL authentication, KERBEROS and PLAIN auth type are supported at the same time, and only the first specified PLAIN auth type is valid.|seq|1.0.0
140140
kyuubi.authentication.custom.class|&lt;undefined&gt;|User-defined authentication implementation of org.apache.kyuubi.service.authentication.PasswdAuthenticationProvider|string|1.3.0
141+
kyuubi.authentication.jdbc.driver.class|&lt;undefined&gt;|Driver class name for JDBC Authentication Provider.|string|1.6.0
142+
kyuubi.authentication.jdbc.password|&lt;undefined&gt;|Database password for JDBC Authentication Provider.|string|1.6.0
143+
kyuubi.authentication.jdbc.query|&lt;undefined&gt;|Query SQL template with placeholders for JDBC Authentication Provider to execute. Authentication passes if at least one row fetched in the result set.Available placeholders are: <ul><li>`${username}`</li><li>`${password}`</li></ul>eg.: query sql `SELECT 1 FROM auth_table WHERE user=${username} AND passwd=MD5(CONCAT(salt,${password}));` will be prepared as: `SELECT 1 FROM auth_table WHERE user=? AND passwd=MD5(CONCAT(salt,?));` with value replacement of `username` and `password` in string type.|string|1.6.0
144+
kyuubi.authentication.jdbc.url|&lt;undefined&gt;|JDBC URL for JDBC Authentication Provider.|string|1.6.0
145+
kyuubi.authentication.jdbc.username|&lt;undefined&gt;|Database username for JDBC Authentication Provider.|string|1.6.0
141146
kyuubi.authentication.ldap.base.dn|&lt;undefined&gt;|LDAP base DN.|string|1.0.0
142147
kyuubi.authentication.ldap.domain|&lt;undefined&gt;|LDAP domain.|string|1.0.0
143148
kyuubi.authentication.ldap.guidKey|uid|LDAP attribute name whose values are unique in this LDAP server.For example:uid or cn.|string|1.2.0

kyuubi-common/pom.xml

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -114,6 +114,11 @@
114114
<artifactId>jackson-databind</artifactId>
115115
</dependency>
116116

117+
<dependency>
118+
<groupId>com.zaxxer</groupId>
119+
<artifactId>HikariCP</artifactId>
120+
</dependency>
121+
117122
<dependency>
118123
<groupId>org.apache.hadoop</groupId>
119124
<artifactId>hadoop-minikdc</artifactId>
@@ -137,6 +142,12 @@
137142
<artifactId>failureaccess</artifactId>
138143
<scope>test</scope>
139144
</dependency>
145+
146+
<dependency>
147+
<groupId>org.apache.derby</groupId>
148+
<artifactId>derby</artifactId>
149+
<scope>test</scope>
150+
</dependency>
140151
</dependencies>
141152

142153
<build>

kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala

Lines changed: 48 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -593,7 +593,9 @@ object KyuubiConf {
593593
" <li>NONE: no authentication check.</li>" +
594594
" <li>KERBEROS: Kerberos/GSSAPI authentication.</li>" +
595595
" <li>CUSTOM: User-defined authentication.</li>" +
596-
" <li>LDAP: Lightweight Directory Access Protocol authentication.</li></ul>" +
596+
" <li>JDBC: JDBC query authentication.</li>" +
597+
" <li>LDAP: Lightweight Directory Access Protocol authentication.</li>" +
598+
"</ul>" +
597599
" Note that: For KERBEROS, it is SASL/GSSAPI mechanism," +
598600
" and for NONE, CUSTOM and LDAP, they are all SASL/PLAIN mechanism." +
599601
" If only NOSASL is specified, the authentication will be NOSASL." +
@@ -645,6 +647,51 @@ object KyuubiConf {
645647
.stringConf
646648
.createWithDefault("uid")
647649

650+
val AUTHENTICATION_JDBC_DRIVER: OptionalConfigEntry[String] =
651+
buildConf("kyuubi.authentication.jdbc.driver.class")
652+
.doc("Driver class name for JDBC Authentication Provider.")
653+
.version("1.6.0")
654+
.stringConf
655+
.createOptional
656+
657+
val AUTHENTICATION_JDBC_URL: OptionalConfigEntry[String] =
658+
buildConf("kyuubi.authentication.jdbc.url")
659+
.doc("JDBC URL for JDBC Authentication Provider.")
660+
.version("1.6.0")
661+
.stringConf
662+
.createOptional
663+
664+
val AUTHENTICATION_JDBC_USERNAME: OptionalConfigEntry[String] =
665+
buildConf("kyuubi.authentication.jdbc.username")
666+
.doc("Database username for JDBC Authentication Provider.")
667+
.version("1.6.0")
668+
.stringConf
669+
.createOptional
670+
671+
val AUTHENTICATION_JDBC_PASSWORD: OptionalConfigEntry[String] =
672+
buildConf("kyuubi.authentication.jdbc.password")
673+
.doc("Database password for JDBC Authentication Provider.")
674+
.version("1.6.0")
675+
.stringConf
676+
.createOptional
677+
678+
val AUTHENTICATION_JDBC_QUERY: OptionalConfigEntry[String] =
679+
buildConf("kyuubi.authentication.jdbc.query")
680+
.doc("Query SQL template with placeholders " +
681+
"for JDBC Authentication Provider to execute. " +
682+
"Authentication passes if at least one row fetched in the result set." +
683+
"Available placeholders are: <ul>" +
684+
"<li>`${username}`</li>" +
685+
"<li>`${password}`</li></ul>" +
686+
"eg.: query sql `SELECT 1 FROM auth_table WHERE user=${username} AND " +
687+
"passwd=MD5(CONCAT(salt,${password}));` " +
688+
"will be prepared as: `SELECT 1 FROM auth_table " +
689+
"WHERE user=? AND passwd=MD5(CONCAT(salt,?));`" +
690+
" with value replacement of `username` and `password` in string type.")
691+
.version("1.6.0")
692+
.stringConf
693+
.createOptional
694+
648695
val DELEGATION_KEY_UPDATE_INTERVAL: ConfigEntry[Long] =
649696
buildConf("kyuubi.delegation.key.update.interval")
650697
.doc("unused yet")

kyuubi-common/src/main/scala/org/apache/kyuubi/service/authentication/AuthMethods.scala

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,5 +19,5 @@ package org.apache.kyuubi.service.authentication
1919

2020
object AuthMethods extends Enumeration {
2121
type AuthMethod = Value
22-
val NONE, LDAP, CUSTOM = Value
22+
val NONE, LDAP, JDBC, CUSTOM = Value
2323
}

kyuubi-common/src/main/scala/org/apache/kyuubi/service/authentication/AuthTypes.scala

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,5 +20,5 @@ package org.apache.kyuubi.service.authentication
2020
object AuthTypes extends Enumeration {
2121
type AuthType = Value
2222

23-
val NOSASL, NONE, LDAP, KERBEROS, CUSTOM = Value
23+
val NOSASL, NONE, LDAP, JDBC, KERBEROS, CUSTOM = Value
2424
}

kyuubi-common/src/main/scala/org/apache/kyuubi/service/authentication/AuthenticationProviderFactory.scala

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,7 @@ object AuthenticationProviderFactory {
4444
conf: KyuubiConf): PasswdAuthenticationProvider = method match {
4545
case AuthMethods.NONE => new AnonymousAuthenticationProviderImpl
4646
case AuthMethods.LDAP => new LdapAuthenticationProviderImpl(conf)
47+
case AuthMethods.JDBC => new JdbcAuthenticationProviderImpl(conf)
4748
case AuthMethods.CUSTOM =>
4849
val className = conf.get(KyuubiConf.AUTHENTICATION_CUSTOM_CLASS)
4950
if (className.isEmpty) {
Lines changed: 204 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,204 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one or more
3+
* contributor license agreements. See the NOTICE file distributed with
4+
* this work for additional information regarding copyright ownership.
5+
* The ASF licenses this file to You under the Apache License, Version 2.0
6+
* (the "License"); you may not use this file except in compliance with
7+
* the License. You may obtain a copy of the License at
8+
*
9+
* http://www.apache.org/licenses/LICENSE-2.0
10+
*
11+
* Unless required by applicable law or agreed to in writing, software
12+
* distributed under the License is distributed on an "AS IS" BASIS,
13+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
* See the License for the specific language governing permissions and
15+
* limitations under the License.
16+
*/
17+
18+
package org.apache.kyuubi.service.authentication
19+
20+
import java.sql.{Connection, PreparedStatement, Statement}
21+
import java.util.Properties
22+
import javax.security.sasl.AuthenticationException
23+
24+
import com.zaxxer.hikari.{HikariConfig, HikariDataSource}
25+
import org.apache.commons.lang3.StringUtils
26+
27+
import org.apache.kyuubi.Logging
28+
import org.apache.kyuubi.config.KyuubiConf
29+
import org.apache.kyuubi.config.KyuubiConf._
30+
31+
class JdbcAuthenticationProviderImpl(conf: KyuubiConf) extends PasswdAuthenticationProvider
32+
with Logging {
33+
34+
private val driverClass = conf.get(AUTHENTICATION_JDBC_DRIVER)
35+
private val jdbcUrl = conf.get(AUTHENTICATION_JDBC_URL)
36+
private val jdbcUsername = conf.get(AUTHENTICATION_JDBC_USERNAME)
37+
private val jdbcUserPassword = conf.get(AUTHENTICATION_JDBC_PASSWORD)
38+
private val authQuerySql = conf.get(AUTHENTICATION_JDBC_QUERY)
39+
40+
private val SQL_PLACEHOLDER_REGEX = """\$\{.+?}""".r
41+
private val USERNAME_SQL_PLACEHOLDER = "${username}"
42+
private val PASSWORD_SQL_PLACEHOLDER = "${password}"
43+
44+
checkJdbcConfigs()
45+
46+
private[kyuubi] val hikariDataSource = getHikariDataSource
47+
48+
/**
49+
* The authenticate method is called by the Kyuubi Server authentication layer
50+
* to authenticate users for their requests.
51+
* If a user is to be granted, return nothing/throw nothing.
52+
* When a user is to be disallowed, throw an appropriate [[AuthenticationException]].
53+
*
54+
* @param user The username received over the connection request
55+
* @param password The password received over the connection request
56+
* @throws AuthenticationException When a user is found to be invalid by the implementation
57+
*/
58+
@throws[AuthenticationException]
59+
override def authenticate(user: String, password: String): Unit = {
60+
if (StringUtils.isBlank(user)) {
61+
throw new AuthenticationException(s"Error validating, user is null" +
62+
s" or contains blank space")
63+
}
64+
65+
if (StringUtils.isBlank(password)) {
66+
throw new AuthenticationException(s"Error validating, password is null" +
67+
s" or contains blank space")
68+
}
69+
70+
var connection: Connection = null
71+
var queryStatement: PreparedStatement = null
72+
73+
try {
74+
connection = hikariDataSource.getConnection
75+
76+
queryStatement = getAndPrepareQueryStatement(connection, user, password)
77+
78+
val resultSet = queryStatement.executeQuery()
79+
80+
if (resultSet == null || !resultSet.next()) {
81+
// auth failed
82+
throw new AuthenticationException(s"Password does not match or no such user. user:" +
83+
s" $user , password length: ${password.length}")
84+
}
85+
86+
// auth passed
87+
88+
} catch {
89+
case e: AuthenticationException =>
90+
throw e
91+
case e: Exception =>
92+
error("Cannot get user info", e);
93+
throw e
94+
} finally {
95+
closeDbConnection(connection, queryStatement)
96+
}
97+
}
98+
99+
private def checkJdbcConfigs(): Unit = {
100+
def configLog(config: String, value: String): String = s"JDBCAuthConfig: $config = '$value'"
101+
102+
debug(configLog("Driver Class", driverClass.orNull))
103+
debug(configLog("JDBC URL", jdbcUrl.orNull))
104+
debug(configLog("Database username", jdbcUsername.orNull))
105+
debug(configLog("Database password length", jdbcUserPassword.getOrElse("").length.toString))
106+
debug(configLog("Query SQL", authQuerySql.orNull))
107+
108+
// Check if JDBC parameters valid
109+
if (driverClass.isEmpty) {
110+
throw new IllegalArgumentException("JDBC driver class is not configured.")
111+
}
112+
113+
if (jdbcUrl.isEmpty) {
114+
throw new IllegalArgumentException("JDBC url is not configured")
115+
}
116+
117+
if (jdbcUsername.isEmpty || jdbcUserPassword.isEmpty) {
118+
throw new IllegalArgumentException("JDBC username or password is not configured")
119+
}
120+
121+
// Check Query SQL
122+
if (authQuerySql.isEmpty) {
123+
throw new IllegalArgumentException("Query SQL is not configured")
124+
}
125+
val querySqlInLowerCase = authQuerySql.get.trim.toLowerCase
126+
if (!querySqlInLowerCase.startsWith("select")) { // allow select query sql only
127+
throw new IllegalArgumentException("Query SQL must start with \"SELECT\"");
128+
}
129+
if (!querySqlInLowerCase.contains("where")) {
130+
warn("Query SQL does not contains \"WHERE\" keyword");
131+
}
132+
if (!querySqlInLowerCase.contains("${username}")) {
133+
warn("Query SQL does not contains \"${username}\" placeholder");
134+
}
135+
}
136+
137+
private def getPlaceholderList(sql: String): List[String] = {
138+
SQL_PLACEHOLDER_REGEX.findAllMatchIn(sql)
139+
.map(m => m.matched)
140+
.toList
141+
}
142+
143+
private def getAndPrepareQueryStatement(
144+
connection: Connection,
145+
user: String,
146+
password: String): PreparedStatement = {
147+
148+
val preparedSql: String = {
149+
SQL_PLACEHOLDER_REGEX.replaceAllIn(authQuerySql.get, "?")
150+
}
151+
debug(s"prepared auth query sql: $preparedSql")
152+
153+
val stmt = connection.prepareStatement(preparedSql)
154+
stmt.setMaxRows(1) // minimum result size required for authentication
155+
156+
// Extract placeholder list and fill parameters to placeholders
157+
val placeholderList: List[String] = getPlaceholderList(authQuerySql.get)
158+
for (i <- placeholderList.indices) {
159+
val param = placeholderList(i) match {
160+
case USERNAME_SQL_PLACEHOLDER => user
161+
case PASSWORD_SQL_PLACEHOLDER => password
162+
case otherPlaceholder =>
163+
throw new IllegalArgumentException(
164+
s"Unrecognized Placeholder In Query SQL: $otherPlaceholder")
165+
}
166+
167+
stmt.setString(i + 1, param)
168+
}
169+
170+
stmt
171+
}
172+
173+
private def closeDbConnection(connection: Connection, statement: Statement): Unit = {
174+
if (statement != null && !statement.isClosed) {
175+
try {
176+
statement.close()
177+
} catch {
178+
case e: Exception =>
179+
error("Cannot close PreparedStatement to auth database ", e)
180+
}
181+
}
182+
183+
if (connection != null && !connection.isClosed) {
184+
try {
185+
connection.close()
186+
} catch {
187+
case e: Exception =>
188+
error("Cannot close connection to auth database ", e)
189+
}
190+
}
191+
}
192+
193+
private def getHikariDataSource: HikariDataSource = {
194+
val datasourceProperties = new Properties()
195+
val hikariConfig = new HikariConfig(datasourceProperties)
196+
hikariConfig.setDriverClassName(driverClass.orNull)
197+
hikariConfig.setJdbcUrl(jdbcUrl.orNull)
198+
hikariConfig.setUsername(jdbcUsername.orNull)
199+
hikariConfig.setPassword(jdbcUserPassword.orNull)
200+
hikariConfig.setPoolName("jdbc-auth-pool")
201+
202+
new HikariDataSource(hikariConfig)
203+
}
204+
}

kyuubi-common/src/main/scala/org/apache/kyuubi/service/authentication/KyuubiAuthenticationFactory.scala

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -140,6 +140,7 @@ class KyuubiAuthenticationFactory(conf: KyuubiConf, isServer: Boolean = true) ex
140140
debug(authTypes)
141141
if (none) AuthMethods.NONE
142142
else if (authTypes.contains(LDAP)) AuthMethods.LDAP
143+
else if (authTypes.contains(JDBC)) AuthMethods.JDBC
143144
else if (authTypes.contains(CUSTOM)) AuthMethods.CUSTOM
144145
else throw new IllegalArgumentException("No valid Password Auth detected")
145146
}

0 commit comments

Comments
 (0)