Druid Extension to enable Authentication using Kerberos. #3853
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,70 @@ | ||
| --- | ||
| layout: doc_page | ||
| --- | ||
|
|
||
| # Druid-Kerberos | ||
|
|
||
| Druid Extension to enable Authentication for Druid Nodes using Kerberos. | ||
| This extension adds AuthenticationFilter which is used to protect HTTP Endpoints using the simple and protected GSSAPI negotiation mechanism [SPNEGO](https://en.wikipedia.org/wiki/SPNEGO). | ||
| Make sure to [include](../../operations/including-extensions.html) `druid-kerberos` as an extension. | ||
|
|
||
|
|
||
| ## Configuration | ||
|
|
||
| |Property|Possible Values|Description|Default|required| | ||
| |--------|---------------|-----------|-------|--------| | ||
| |`druid.hadoop.security.kerberos.principal`|`druid@EXAMPLE.COM`| Principal user name, used for internal node communication|empty|Yes| | ||
| |`druid.hadoop.security.kerberos.keytab`|`/etc/security/keytabs/druid.headlessUser.keytab`|Path to keytab file used for internal node communication|empty|Yes| | ||
| |`druid.hadoop.security.spnego.principal`|`HTTP/_HOST@EXAMPLE.COM`| SPNego service principal used by druid nodes|empty|Yes| | ||
| |`druid.hadoop.security.spnego.keytab`|`/etc/security/keytabs/spnego.service.keytab`|SPNego service keytab used by druid nodes|empty|Yes| | ||
| |`druid.hadoop.security.spnego.authToLocal`|`RULE:[1:$1@$0](druid@EXAMPLE.COM)s/.*/druid DEFAULT`|It allows you to set a general rule for mapping principal names to local user names. It will be used if there is not an explicit mapping for the principal name that is being translated.|DEFAULT|No| | ||
| |`druid.hadoop.security.spnego.excludedPaths`|`['/status','/health']`| Array of HTTP paths which which does NOT need to be authenticated.|None|No| | ||
| |`druid.hadoop.security.spnego.cookieSignatureSecret`|`secretString`| Secret used to sign authentication cookies|<Random value>|No| | ||
|
|
||
| As a note, it is required that the SPNego principal in use by the druid nodes must start with HTTP (This specified by [RFC-4559](https://tools.ietf.org/html/rfc4559)) and must be of the form "HTTP/_HOST@REALM". | ||
| The special string _HOST will be replaced automatically with the value of config `druid.host` | ||
|
|
||
| ### Auth to Local Syntax | ||
|
|
||
| `druid.hadoop.security.spnego.authToLocal` allows you to set a general rules for mapping principal names to local user names. | ||
| The syntax for mapping rules is `RULE:\[n:string](regexp)s/pattern/replacement/g`. The integer n indicates how many components the target principal should have. If this matches, then a string will be formed from string, substituting the realm of the principal for $0 and the n‘th component of the principal for $n. e.g. if the principal was druid/admin then `\[2:$2$1suffix]` would result in the string `admindruidsuffix`. | ||
| If this string matches regexp, then the s//\[g] substitution command will be run over the string. The optional g will cause the substitution to be global over the string, instead of replacing only the first match in the string. | ||
| If required, multiple rules can be be joined by newline character and specified as a String. | ||
|
|
||
| ## Accessing Druid HTTP end points when kerberos security is enabled | ||
| 1. To access druid HTTP endpoints via curl user will need to first login using `kinit` command as follows - | ||
|
|
||
| ``` | ||
| kinit -k -t <path_to_keytab_file> user@REALM.COM | ||
| ``` | ||
|
|
||
| 2. Once the login is successful verify that login is successful using `klist` command | ||
| 3. Now you can access druid HTTP endpoints using curl command as follows - | ||
|
|
||
| ``` | ||
| curl --negotiate -u:anyUser -b ~/cookies.txt -c ~/cookies.txt -X POST -H'Content-Type: application/json' <HTTP_END_POINT> | ||
|
There was a problem hiding this comment. can we mention that negotiate is needed only once ? There was a problem hiding this comment. IMO, user should always specify negotiate. Fwiw, adding this will not mean that authentication handshake will be always done.
Will add comment about the use of cookies. |
||
| ``` | ||
|
|
||
| e.g to send a query from file `query.json` to druid broker use this command - | ||
|
|
||
| ``` | ||
| curl --negotiate -u:anyUser -b ~/cookies.txt -c ~/cookies.txt -X POST -H'Content-Type: application/json' http://broker-host:port/druid/v2/?pretty -d @query.json | ||
| ``` | ||
|
|
||
|
|
||
| ## Accessing coordinator or overlord console from web browser | ||
| To access Coordinator/Overlord console from browser you will need to configure your browser for SPNego authentication as follows - | ||
|
|
||
| 1. Safari - No configurations required. | ||
| 2. Firefox - Open firefox and follow these steps - | ||
| 1. Go to `about:config` and search for `network.negotiate-auth.trusted-uris`. | ||
| 2. Double-click and add the following values: `"http://druid-coordinator-hostname:ui-port"` and `"http://druid-overlord-hostname:port"` | ||
| 3. Google Chrome - From the command line run following commands - | ||
| 1. `google-chrome --auth-server-whitelist="druid-coordinator-hostname" --auth-negotiate-delegate-whitelist="druid-coordinator-hostname"` | ||
| 2. `google-chrome --auth-server-whitelist="druid-overlord-hostname" --auth-negotiate-delegate-whitelist="druid-overlord-hostname"` | ||
| 4. Internet Explorer - | ||
| 1. Configure trusted websites to include `"druid-coordinator-hostname"` and `"druid-overlord-hostname"` | ||
| 2. Allow negotiation for the UI website. | ||
|
|
||
| ## Sending Queries programmatically | ||
| Many HTTP client libraries, such as Apache Commons [HttpComponents](https://hc.apache.org/), already have support for performing SPNEGO authentication. You can use any of the available HTTP client library to communicate with druid cluster. | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,182 @@ | ||
| <?xml version="1.0" encoding="UTF-8"?> | ||
| <!-- | ||
| ~ Licensed to Metamarkets Group Inc. (Metamarkets) under one | ||
| ~ or more contributor license agreements. See the NOTICE file | ||
| ~ distributed with this work for additional information | ||
| ~ regarding copyright ownership. Metamarkets licenses this file | ||
| ~ to you under the Apache License, Version 2.0 (the | ||
| ~ "License"); you may not use this file except in compliance | ||
| ~ with the License. You may obtain a copy of the License at | ||
| ~ | ||
| ~ http://www.apache.org/licenses/LICENSE-2.0 | ||
| ~ | ||
| ~ Unless required by applicable law or agreed to in writing, | ||
| ~ software distributed under the License is distributed on an | ||
| ~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| ~ KIND, either express or implied. See the License for the | ||
| ~ specific language governing permissions and limitations | ||
| ~ under the License. | ||
| --> | ||
|
|
||
| <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" | ||
| xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> | ||
| <modelVersion>4.0.0</modelVersion> | ||
|
|
||
| <groupId>io.druid.extensions</groupId> | ||
| <artifactId>druid-kerberos</artifactId> | ||
| <name>druid-kerberos</name> | ||
| <description>druid-kerberos</description> | ||
|
|
||
| <parent> | ||
| <groupId>io.druid</groupId> | ||
| <artifactId>druid</artifactId> | ||
| <version>0.9.3-SNAPSHOT</version> | ||
| <relativePath>../../pom.xml</relativePath> | ||
| </parent> | ||
|
|
||
| <dependencies> | ||
| <dependency> | ||
| <groupId>io.druid</groupId> | ||
| <artifactId>druid-processing</artifactId> | ||
| <version>${project.parent.version}</version> | ||
| <scope>provided</scope> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>io.druid</groupId> | ||
| <artifactId>druid-server</artifactId> | ||
| <version>${project.parent.version}</version> | ||
| <scope>provided</scope> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.eclipse.jetty</groupId> | ||
| <artifactId>jetty-server</artifactId> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.eclipse.jetty</groupId> | ||
| <artifactId>jetty-util</artifactId> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.eclipse.jetty</groupId> | ||
| <artifactId>jetty-proxy</artifactId> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.eclipse.jetty</groupId> | ||
| <artifactId>jetty-servlet</artifactId> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.eclipse.jetty</groupId> | ||
| <artifactId>jetty-servlets</artifactId> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.apache.hadoop</groupId> | ||
| <artifactId>hadoop-common</artifactId> | ||
| <version>${hadoop.compile.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>commons-cli</groupId> | ||
| <artifactId>commons-cli</artifactId> | ||
| </exclusion> | ||
| <exclusion> | ||
| <groupId>commons-httpclient</groupId> | ||
| <artifactId>commons-httpclient</artifactId> | ||
| </exclusion> | ||
| <exclusion> | ||
| <groupId>log4j</groupId> | ||
| <artifactId>log4j</artifactId> | ||
| </exclusion> | ||
| <exclusion> | ||
| <groupId>commons-codec</groupId> | ||
| <artifactId>commons-codec</artifactId> | ||
| </exclusion> | ||
| <exclusion> | ||
| <groupId>commons-logging</groupId> | ||
| <artifactId>commons-logging</artifactId> | ||
| </exclusion> | ||
| <exclusion> | ||
| <groupId>commons-io</groupId> | ||
| <artifactId>commons-io</artifactId> | ||
| </exclusion> | ||
| <exclusion> | ||
| <groupId>commons-lang</groupId> | ||
| <artifactId>commons-lang</artifactId> | ||
| </exclusion> | ||
| <exclusion> | ||
| <groupId>org.apache.httpcomponents</groupId> | ||
| <artifactId>httpclient</artifactId> | ||
| </exclusion> | ||
| <exclusion> | ||
| <groupId>org.apache.httpcomponents</groupId> | ||
| <artifactId>httpcore</artifactId> | ||
| </exclusion> | ||
| <exclusion> | ||
| <groupId>org.codehaus.jackson</groupId> | ||
| <artifactId>jackson-core-asl</artifactId> | ||
| </exclusion> | ||
| <exclusion> | ||
| <groupId>org.codehaus.jackson</groupId> | ||
| <artifactId>jackson-mapper-asl</artifactId> | ||
| </exclusion> | ||
| <exclusion> | ||
| <groupId>org.apache.zookeeper</groupId> | ||
| <artifactId>zookeeper</artifactId> | ||
| </exclusion> | ||
| <exclusion> | ||
| <groupId>org.slf4j</groupId> | ||
| <artifactId>slf4j-api</artifactId> | ||
| </exclusion> | ||
| <exclusion> | ||
| <groupId>org.slf4j</groupId> | ||
| <artifactId>slf4j-log4j12</artifactId> | ||
| </exclusion> | ||
| <exclusion> | ||
| <groupId>javax.ws.rs</groupId> | ||
| <artifactId>jsr311-api</artifactId> | ||
| </exclusion> | ||
| <exclusion> | ||
| <groupId>com.google.code.findbugs</groupId> | ||
| <artifactId>jsr305</artifactId> | ||
| </exclusion> | ||
| <exclusion> | ||
| <groupId>org.mortbay.jetty</groupId> | ||
| <artifactId>jetty-util</artifactId> | ||
| </exclusion> | ||
| <exclusion> | ||
| <groupId>org.apache.hadoop</groupId> | ||
| <artifactId>hadoop-annotations</artifactId> | ||
| </exclusion> | ||
| <exclusion> | ||
| <groupId>javax.activation</groupId> | ||
| <artifactId>activation</artifactId> | ||
| </exclusion> | ||
| <exclusion> | ||
| <groupId>com.google.protobuf</groupId> | ||
| <artifactId>protobuf-java</artifactId> | ||
| </exclusion> | ||
| <exclusion> | ||
| <groupId>com.sun.jersey</groupId> | ||
| <artifactId>jersey-core</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
|
|
||
| <!-- Tests --> | ||
| <dependency> | ||
| <groupId>io.druid</groupId> | ||
| <artifactId>druid-processing</artifactId> | ||
| <version>${project.parent.version}</version> | ||
| <scope>test</scope> | ||
| <type>test-jar</type> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>junit</groupId> | ||
| <artifactId>junit</artifactId> | ||
| <scope>test</scope> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.easymock</groupId> | ||
| <artifactId>easymock</artifactId> | ||
| <scope>test</scope> | ||
| </dependency> | ||
| </dependencies> | ||
|
|
||
| </project> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,78 @@ | ||
| /* | ||
| * Licensed to Metamarkets Group Inc. (Metamarkets) under one | ||
| * or more contributor license agreements. See the NOTICE file | ||
| * distributed with this work for additional information | ||
| * regarding copyright ownership. Metamarkets licenses this file | ||
| * to you under the Apache License, Version 2.0 (the | ||
| * "License"); you may not use this file except in compliance | ||
| * with the License. You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, | ||
| * software distributed under the License is distributed on an | ||
| * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| * KIND, either express or implied. See the License for the | ||
| * specific language governing permissions and limitations | ||
| * under the License. | ||
| */ | ||
|
|
||
| package io.druid.security.kerberos; | ||
|
|
||
|
|
||
| import com.fasterxml.jackson.annotation.JsonCreator; | ||
| import com.fasterxml.jackson.annotation.JsonProperty; | ||
|
|
||
| public class DruidKerberosConfig | ||
| { | ||
| @JsonProperty | ||
| private final String principal; | ||
| @JsonProperty | ||
| private final String keytab; | ||
|
|
||
| @JsonCreator | ||
| public DruidKerberosConfig(@JsonProperty("principal") String principal, @JsonProperty("keytab") String keytab) | ||
| { | ||
| this.principal = principal; | ||
| this.keytab = keytab; | ||
| } | ||
|
|
||
| @JsonProperty | ||
| public String getPrincipal() | ||
| { | ||
| return principal; | ||
| } | ||
|
|
||
| @JsonProperty | ||
| public String getKeytab() | ||
| { | ||
| return keytab; | ||
| } | ||
|
|
||
| @Override | ||
| public boolean equals(Object o) | ||
| { | ||
| if (this == o) { | ||
| return true; | ||
| } | ||
| if (!(o instanceof DruidKerberosConfig)) { | ||
| return false; | ||
| } | ||
|
|
||
| DruidKerberosConfig that = (DruidKerberosConfig) o; | ||
|
|
||
| if (getPrincipal() != null ? !getPrincipal().equals(that.getPrincipal()) : that.getPrincipal() != null) { | ||
| return false; | ||
| } | ||
| return getKeytab() != null ? getKeytab().equals(that.getKeytab()) : that.getKeytab() == null; | ||
|
|
||
| } | ||
|
|
||
| @Override | ||
| public int hashCode() | ||
| { | ||
| int result = getPrincipal() != null ? getPrincipal().hashCode() : 0; | ||
| result = 31 * result + (getKeytab() != null ? getKeytab().hashCode() : 0); | ||
| return result; | ||
| } | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
wondering : if we can get away with a random String why we need this config ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
main reason is that the cookie specs does not guarantee isolation against port numbers.
e.g This is a sample cookie content from my host which does not have any port info-
so if you have multiple druid nodes running on same host using same paths and different signature (generated randomly instead of being explictly set ), user will keep doing token negotiation as cookie generated by one node will not be accepted by other node. setting the cookie signature ensures that this does not occur.
Have added more clarification in docs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@nishantmonu51 This is a very important information to add to the docs thought looks like it is mandetory to have it random if you have multiple process in the same machine.