Skip to content

Commit

Permalink
[KYUUBI #2156][FOLLOWUP] Fix configuration format in document
Browse files Browse the repository at this point in the history 
### _Why are the changes needed?_

fix #2157 cofiguration format error

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #2165 from jiaoqingbo/2156-followup.

Closes #2156

9862ddb [jiaoqingbo] [KYUUBI #2156][FOLLOWUP] Fix configuration format in document
47157c5 [jiaoqingbo] Merge branch 'master' into 2156-followup
9c89c08 [jiaoqingbo] [KYUUBI #2156][FOLLOWUP]fix cofiguration format
4608432 [jiaoqingbo] Merge branch 'master' into 2156
96a22e5 [jiaoqingbo] add link to doc
dc11a21 [jiaoqingbo] [KYUUBI #2156] Change log to reflect exactly why getting token failed

Authored-by: jiaoqingbo <1178404354@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
  • Loading branch information
@jiaoqingbo @pan3793
jiaoqingbo authored and pan3793 committed Mar 18, 2022
1 parent 21ca754 commit 5225b54
Show file tree
Hide file tree
Showing 3 changed files with 24 additions and 24 deletions.
20 changes: 10 additions & 10 deletions docs/security/authentication.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -36,16 +36,16 @@ In `$KYUUBI_HOME/conf/kyuubi-defaults.conf`, specify `kyuubi.authentication` to

Key | Default | Meaning | Since
--- | --- | --- | ---
kyuubi\.authentication|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>NONE</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Client authentication types.<ul> <li>NOSASL: raw transport.</li> <li>NONE: no authentication check.</li> <li>KERBEROS: Kerberos/GSSAPI authentication.</li> <li>LDAP: Lightweight Directory Access Protocol authentication.</li></ul></div>|<div style='width: 20pt'>1.0.0</div>
<code>kyuubi.authentication</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>NONE</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Client authentication types.<ul> <li>NOSASL: raw transport.</li> <li>NONE: no authentication check.</li> <li>KERBEROS: Kerberos/GSSAPI authentication.</li> <li>LDAP: Lightweight Directory Access Protocol authentication.</li></ul></div>|<div style='width: 20pt'>1.0.0</div>


Key | Default | Meaning | Since
--- | --- | --- | ---
kyuubi\.authentication|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>NONE</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Client authentication types.<ul> <li>NOSASL: raw transport.</li> <li>NONE: no authentication check.</li> <li>KERBEROS: Kerberos/GSSAPI authentication.</li> <li>LDAP: Lightweight Directory Access Protocol authentication.</li></ul></div>|<div style='width: 20pt'>1.0.0</div>
kyuubi\.authentication<br>\.ldap\.base\.dn|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>&lt;undefined&gt;</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>LDAP base DN.</div>|<div style='width: 20pt'>1.0.0</div>
kyuubi\.authentication<br>\.ldap\.domain|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>&lt;undefined&gt;</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>LDAP base DN.</div>|<div style='width: 20pt'>1.0.0</div>
kyuubi\.authentication<br>\.ldap\.url|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>&lt;undefined&gt;</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>SPACE character separated LDAP connection URL(s).</div>|<div style='width: 20pt'>1.0.0</div>
kyuubi\.authentication<br>\.sasl\.qop|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>auth</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Sasl QOP enable higher levels of protection for Kyuubi communication with clients.<ul> <li>auth - authentication only (default)</li> <li>auth-int - authentication plus integrity protection</li> <li>auth-conf - authentication plus integrity and confidentiality protection. This is applicable only if Kyuubi is configured to use Kerberos authentication.</li> </ul></div>|<div style='width: 20pt'>1.0.0</div>
<code>kyuubi.authentication</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>NONE</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Client authentication types.<ul> <li>NOSASL: raw transport.</li> <li>NONE: no authentication check.</li> <li>KERBEROS: Kerberos/GSSAPI authentication.</li> <li>LDAP: Lightweight Directory Access Protocol authentication.</li></ul></div>|<div style='width: 20pt'>1.0.0</div>
<code>kyuubi.authentication.ldap.base.dn</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>&lt;undefined&gt;</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>LDAP base DN.</div>|<div style='width: 20pt'>1.0.0</div>
<code>kyuubi.authentication.ldap.domain</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>&lt;undefined&gt;</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>LDAP base DN.</div>|<div style='width: 20pt'>1.0.0</div>
<code>kyuubi.authentication.ldap.url</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>&lt;undefined&gt;</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>SPACE character separated LDAP connection URL(s).</div>|<div style='width: 20pt'>1.0.0</div>
<code>kyuubi.authentication.sasl.qop</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>auth</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Sasl QOP enable higher levels of protection for Kyuubi communication with clients.<ul> <li>auth - authentication only (default)</li> <li>auth-int - authentication plus integrity protection</li> <li>auth-conf - authentication plus integrity and confidentiality protection. This is applicable only if Kyuubi is configured to use Kerberos authentication.</li> </ul></div>|<div style='width: 20pt'>1.0.0</div>


#### Using KERBEROS
Expand All @@ -64,10 +64,10 @@ Following configurations also need to be set to enable KERBEROS authentication:

Key | Default | Meaning | Since
--- | --- | --- | ---
kyuubi\.kinit<br>\.principal|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>&lt;undefined&gt;</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Name of the Kerberos principal.</div>|<div style='width: 20pt'>1.0.0</div>
kyuubi\.kinit\.keytab|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>&lt;undefined&gt;</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Location of Kyuubi server's keytab.</div>|<div style='width: 20pt'>1.0.0</div>
kyuubi\.kinit\.interval|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>PT1H</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>How often will Kyuubi server run `kinit -kt [keytab] [principal]` to renew the local Kerberos credentials cache</div>|<div style='width: 20pt'>1.0.0</div>
kyuubi\.kinit\.max<br>\.attempts|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>10</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>How many times will `kinit` process retry</div>|<div style='width: 20pt'>1.0.0</div>
<code>kyuubi.kinit.principal</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>&lt;undefined&gt;</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Name of the Kerberos principal.</div>|<div style='width: 20pt'>1.0.0</div>
<code>kyuubi.kinit.keytab</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>&lt;undefined&gt;</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Location of Kyuubi server's keytab.</div>|<div style='width: 20pt'>1.0.0</div>
<code>kyuubi.kinit.interval</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>PT1H</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>How often will Kyuubi server run `kinit -kt [keytab] [principal]` to renew the local Kerberos credentials cache</div>|<div style='width: 20pt'>1.0.0</div>
<code>kyuubi.kinit.max.attempts</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>10</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>How many times will `kinit` process retry</div>|<div style='width: 20pt'>1.0.0</div>


Please refer to [Kinit Auxiliary Service](kinit.html) to get configuration steps.
20 changes: 10 additions & 10 deletions docs/security/hadoop_credentials_manager.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -70,11 +70,11 @@ Hadoop Credentials Manager is initialized when Kyuubi server starts.

Key | Default | Meaning | Type | Since
--- | --- | --- | --- | ---
kyuubi\.credentials<br>\.hadoopfs\.enabled|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>true</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>Whether to renew Hadoop filesystem delegation tokens</div>|<div style='width: 30pt'>boolean</div>|<div style='width: 20pt'>1.4.0</div>
kyuubi\.credentials<br>\.hadoopfs\.uris|<div style='width: 65pt;word-wrap: break-word;white-space: normal'></div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>Extra Hadoop filesystem URIs for which to request delegation tokens. The filesystem that hosts fs.defaultFS does not need to be listed here.</div>|<div style='width: 30pt'>seq</div>|<div style='width: 20pt'>1.4.0</div>
kyuubi\.credentials<br>\.hive\.enabled|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>true</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>Whether to renew Hive metastore delegation token</div>|<div style='width: 30pt'>boolean</div>|<div style='width: 20pt'>1.4.0</div>
kyuubi\.credentials<br>\.renewal\.interval|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>PT1H</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>How often Kyuubi renews one user's delegation tokens</div>|<div style='width: 30pt'>duration</div>|<div style='width: 20pt'>1.4.0</div>
kyuubi\.credentials<br>\.renewal\.retry\.wait|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>PT1M</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>How long to wait before retrying to fetch new credentials after a failure.</div>|<div style='width: 30pt'>duration</div>|<div style='width: 20pt'>1.4.0</div>
<code>kyuubi.credentials.hadoopfs.enabled</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>true</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>Whether to renew Hadoop filesystem delegation tokens</div>|<div style='width: 30pt'>boolean</div>|<div style='width: 20pt'>1.4.0</div>
<code>kyuubi.credentials.hadoopfs.uris</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'></div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>Extra Hadoop filesystem URIs for which to request delegation tokens. The filesystem that hosts fs.defaultFS does not need to be listed here.</div>|<div style='width: 30pt'>seq</div>|<div style='width: 20pt'>1.4.0</div>
<code>kyuubi.credentials.hive.enabled</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>true</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>Whether to renew Hive metastore delegation token</div>|<div style='width: 30pt'>boolean</div>|<div style='width: 20pt'>1.4.0</div>
<code>kyuubi.credentials.renewal.interval</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>PT1H</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>How often Kyuubi renews one user's delegation tokens</div>|<div style='width: 30pt'>duration</div>|<div style='width: 20pt'>1.4.0</div>
<code>kyuubi.credentials.renewal.retry.wait</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>PT1M</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>How long to wait before retrying to fetch new credentials after a failure.</div>|<div style='width: 30pt'>duration</div>|<div style='width: 20pt'>1.4.0</div>


### Required Security Configs
Expand All @@ -83,8 +83,8 @@ The necessary configurations for hdfs and hive to obtain delegation token are as

Key | Meaning | value
--- | --- | ---
<div style='width: 40pt;word-wrap: break-word;white-space: normal'>hadoop.security.authentication</div>|<div style='width: 40pt;word-wrap: break-word;white-space: normal'>Set the authentication for the cluster</div>|<div style='width: 40pt;word-wrap: break-word;white-space: normal'>kerberos</div>
<div style='width: 40pt;word-wrap: break-word;white-space: normal'>hive.metastore.uris</div>|<div style='width: 40pt;word-wrap: break-word;white-space: normal'>URI for client to contact metastore server</div>|<div style='width: 40pt;word-wrap: break-word;white-space: normal'>thrift://{metastoreHost}:{metastorePort}}</div>
<div style='width: 40pt;word-wrap: break-word;white-space: normal'>hive.metastore.sasl.enabled</div>|<div style='width: 40pt;word-wrap: break-word;white-space: normal'>If true, the metastore thrift interface will be secured with SASL.Clients must authenticate with Kerberos.</div>|<div style='width: 40pt;word-wrap: break-word;white-space: normal'>true</div>
<div style='width: 40pt;word-wrap: break-word;white-space: normal'>hive.metastore.kerberos.principal</div>|<div style='width: 40pt;word-wrap: break-word;white-space: normal'>The service principal for the metastore thrift server. The special string _HOST will be replaced automatically with the correct host name.</div>|<div style='width: 40pt;word-wrap: break-word;white-space: normal'>for example hive/_HOST@${realm}</div>
<div style='width: 40pt;word-wrap: break-word;white-space: normal'>hive.metastore.kerberos.keytab.file</div>|<div style='width: 40pt;word-wrap: break-word;white-space: normal'>The path to the Kerberos Keytab file containing the metastore thrift server's service principal.</div>|<div style='width: 40pt;word-wrap: break-word;white-space: normal'>for example /etc/security/keytabs/hive.service.keytab</div>
<code>hadoop.security.authentication</code>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>Set the authentication for the cluster</div>|<div style='width: 120pt;word-wrap: break-word;white-space: normal'>kerberos</div>
<code>hive.metastore.uris</code>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>URI for client to contact metastore server</div>|<div style='width: 120pt;word-wrap: break-word;white-space: normal'>thrift://{metastoreHost}:{metastorePort}}</div>
<code>hive.metastore.sasl.enabled</code>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>If true, the metastore thrift interface will be secured with SASL.Clients must authenticate with Kerberos.</div>|<div style='width: 120pt;word-wrap: break-word;white-space: normal'>true</div>
<code>hive.metastore.kerberos.principal</code>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>The service principal for the metastore thrift server. The special string _HOST will be replaced automatically with the correct host name.</div>|<div style='width: 120pt;word-wrap: break-word;white-space: normal'>for example hive/_HOST@${realm}</div>
<code>hive.metastore.kerberos.keytab.file</code>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>The path to the Kerberos Keytab file containing the metastore thrift server's service principal.</div>|<div style='width: 120pt;word-wrap: break-word;white-space: normal'>for example /etc/security/keytabs/hive.service.keytab</div>
8 changes: 4 additions & 4 deletions docs/security/kinit.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -77,10 +77,10 @@ They are valid for relatively short period. So, we always need to refresh it for
Key | Default | Meaning | Since
--- | --- | --- | ---
kyuubi\.kinit<br>\.principal|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>&lt;undefined&gt;</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Name of the Kerberos principal.</div>|<div style='width: 20pt'>1.0.0</div>
kyuubi\.kinit\.keytab|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>&lt;undefined&gt;</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Location of Kyuubi server's keytab.</div>|<div style='width: 20pt'>1.0.0</div>
kyuubi\.kinit\.interval|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>PT1H</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>How often will Kyuubi server run `kinit -kt [keytab] [principal]` to renew the local Kerberos credentials cache</div>|<div style='width: 20pt'>1.0.0</div>
kyuubi\.kinit\.max<br>\.attempts|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>10</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>How many times will `kinit` process retry</div>|<div style='width: 20pt'>1.0.0</div>
<code>kyuubi.kinit.principal</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>&lt;undefined&gt;</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Name of the Kerberos principal.</div>|<div style='width: 20pt'>1.0.0</div>
<code>kyuubi.kinit.keytab</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>&lt;undefined&gt;</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Location of Kyuubi server's keytab.</div>|<div style='width: 20pt'>1.0.0</div>
<code>kyuubi.kinit.interval</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>PT1H</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>How often will Kyuubi server run `kinit -kt [keytab] [principal]` to renew the local Kerberos credentials cache</div>|<div style='width: 20pt'>1.0.0</div>
<code>kyuubi.kinit.max.attempts</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>10</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>How many times will `kinit` process retry</div>|<div style='width: 20pt'>1.0.0</div>
When working with a Kerberos-enabled Hadoop cluster, we should ensure that `hadoop.security.authentication`
is set to `KERBEROS` in `$HADOOP_CONF_DIR/core-site.xml` or `$KYUUBI_HOME/conf/kyuubi-defaults.conf`.
Expand Down

0 comments on commit 5225b54

Please sign in to comment.