Permalink
Browse files

Documentation fixes for cdh3u6

  • Loading branch information...
1 parent 4c4f794 commit f42a6f927a2a71d690070cb4e9f9ffde163701db Alejandro Abdelnur committed Mar 11, 2013
Showing with 19 additions and 19 deletions.
  1. +7 −7 ServerSetup.html
  2. +6 −6 UsingHttpTools.html
  3. +6 −6 index.html
View
@@ -1,15 +1,15 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-<!-- Generated by Apache Maven Doxia at Jul 25, 2012 -->
+<!-- Generated by Apache Maven Doxia at Mar 11, 2013 -->
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <title>HttpFS - Hadoop HDFS over HTTP 0.20.2-cdh3u4 - Server Setup</title>
+ <title>HttpFS - Hadoop HDFS over HTTP 0.20.2-cdh3u6 - Server Setup</title>
<style type="text/css" media="all">
@import url("./css/maven-base.css");
@import url("./css/maven-theme.css");
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css" media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20120725" />
+ <meta name="Date-Revision-yyyymmdd" content="20130311" />
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
</head>
<body class="composite">
@@ -24,7 +24,7 @@
<div id="breadcrumbs">
<div class="xleft">
- Last Published: 2012-07-25
+ Last Published: 2013-03-11
</div>
<div class="xright"> <a href="http://hadoop.apache.org/" class="externalLink">Apache Hadoop</a>
@@ -44,7 +44,7 @@
</div>
<div id="bodyColumn">
<div id="contentBox">
- <!-- Licensed under the Apache License, Version 2.0 (the "License"); --><!-- you may not use this file except in compliance with the License. --><!-- You may obtain a copy of the License at --><!-- --><!-- http://www.apache.org/licenses/LICENSE-2.0 --><!-- --><!-- Unless required by applicable law or agreed to in writing, software --><!-- distributed under the License is distributed on an "AS IS" BASIS, --><!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. --><!-- See the License for the specific language governing permissions and --><!-- limitations under the License. --><div class="section"><h2>Hadoop HDFS over HTTP 0.20.2-cdh3u4 - Server Setup<a name="Hadoop_HDFS_over_HTTP_0.20.2-cdh3u4_-_Server_Setup"></a></h2><p>[ <a href="./index.html">Go Back</a> ]</p><p>This page explains how to quickly setup HttpFS with Pseudo authentication against a Hadoop cluster with Pseudo authentication.</p><div class="section"><h3>Requirements<a name="Requirements"></a></h3><ul><li>Java 6+</li><li>Maven 3+</li></ul></div><div class="section"><h3>Install HttpFS<a name="Install_HttpFS"></a></h3><div class="source"><pre>~ $ tar xzf httpfs-0.20.2-cdh3u4.tar.gz</pre></div></div><div class="section"><h3>Configure HttpFS<a name="Configure_HttpFS"></a></h3><p>By default, HttpFS assumes that Hadoop configuration files (<tt>core-site.xml &amp; hdfs-site.xml</tt>) are in the HttpFS configuration directory.</p><p>If this is not the case, add to the <tt>httpfs-site.xml</tt> file the <tt>httpfs.hadoop.config.dir</tt> property set to the location of the Hadoop configuration directory.</p></div><div class="section"><h3>Configure Hadoop<a name="Configure_Hadoop"></a></h3><p>Edit Hadoop <tt>core-site.xml</tt> and defined the Unix user that will run the HttpFS server as a proxyuser. For example:</p><div class="source"><pre> ...
+ <!-- Licensed under the Apache License, Version 2.0 (the "License"); --><!-- you may not use this file except in compliance with the License. --><!-- You may obtain a copy of the License at --><!-- --><!-- http://www.apache.org/licenses/LICENSE-2.0 --><!-- --><!-- Unless required by applicable law or agreed to in writing, software --><!-- distributed under the License is distributed on an "AS IS" BASIS, --><!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. --><!-- See the License for the specific language governing permissions and --><!-- limitations under the License. --><div class="section"><h2>Hadoop HDFS over HTTP 0.20.2-cdh3u6 - Server Setup<a name="Hadoop_HDFS_over_HTTP_0.20.2-cdh3u6_-_Server_Setup"></a></h2><p>[ <a href="./index.html">Go Back</a> ]</p><p>This page explains how to quickly setup HttpFS with Pseudo authentication against a Hadoop cluster with Pseudo authentication.</p><div class="section"><h3>Requirements<a name="Requirements"></a></h3><ul><li>Java 6+</li><li>Maven 3+</li></ul></div><div class="section"><h3>Install HttpFS<a name="Install_HttpFS"></a></h3><div class="source"><pre>~ $ tar xzf httpfs-0.20.2-cdh3u6.tar.gz</pre></div></div><div class="section"><h3>Configure HttpFS<a name="Configure_HttpFS"></a></h3><p>By default, HttpFS assumes that Hadoop configuration files (<tt>core-site.xml &amp; hdfs-site.xml</tt>) are in the HttpFS configuration directory.</p><p>If this is not the case, add to the <tt>httpfs-site.xml</tt> file the <tt>httpfs.hadoop.config.dir</tt> property and set the value to the location of the Hadoop configuration directory.</p></div><div class="section"><h3>Configure Hadoop<a name="Configure_Hadoop"></a></h3><p>Edit Hadoop <tt>core-site.xml</tt> and define the Unix user that will run the HttpFS server as a proxyuser. For example:</p><div class="source"><pre> ...
&lt;property&gt;
&lt;name&gt;hadoop.proxyuser.#HTTPFSUSER#.hosts&lt;/name&gt;
&lt;value&gt;httpfs-host.foo.com&lt;/value&gt;
@@ -53,7 +53,7 @@
&lt;name&gt;hadoop.proxyuser.#HTTPFSUSER#.groups&lt;/name&gt;
&lt;value&gt;*&lt;/value&gt;
&lt;/property&gt;
- ...</pre></div><p>IMPORTANT: Replace <tt>#HTTPFSUSER#</tt> with the Unix user that will start the HttpFS server.</p></div><div class="section"><h3>Restart Hadoop<a name="Restart_Hadoop"></a></h3><p>You need to restart Hadoop for the proxyuser configuration ot become active.</p></div><div class="section"><h3>Start/Stop HttpFS<a name="StartStop_HttpFS"></a></h3><p>To start/stop HttpFS use HttpFS's bin/httpfs.sh script. For example:</p><div class="source"><pre>httpfs-0.20.2-cdh3u4 $ bin/httpfs.sh start</pre></div><p>NOTE: Invoking the script without any parameters list all possible parameters (start, stop, run, etc.). The <tt>httpfs.sh</tt> script is a wrapper for Tomcat's <tt>catalina.sh</tt> script that sets the environment variables and Java System properties required to run HttpFS server.</p></div><div class="section"><h3>Test HttpFS is working<a name="Test_HttpFS_is_working"></a></h3><div class="source"><pre>~ $ curl -i &quot;http://&lt;HTTPFSHOSTNAME&gt;:14000?user.name=babu&amp;op=homedir&quot;
+ ...</pre></div><p>IMPORTANT: Replace <tt>#HTTPFSUSER#</tt> with the Unix user that will start the HttpFS server.</p></div><div class="section"><h3>Restart Hadoop<a name="Restart_Hadoop"></a></h3><p>You need to restart Hadoop for the proxyuser configuration to become active.</p></div><div class="section"><h3>Start/Stop HttpFS<a name="StartStop_HttpFS"></a></h3><p>To start/stop HttpFS use HttpFS's bin/httpfs.sh script. For example:</p><div class="source"><pre>httpfs-0.20.2-cdh3u6 $ bin/httpfs.sh start</pre></div><p>NOTE: Invoking the script without any parameters lists all possible parameters (start, stop, run, etc.). The <tt>httpfs.sh</tt> script is a wrapper for Tomcat's <tt>catalina.sh</tt> script that sets the environment variables and Java System properties required to run HttpFS server.</p></div><div class="section"><h3>Test HttpFS is working<a name="Test_HttpFS_is_working"></a></h3><div class="source"><pre>~ $ curl -i &quot;http://&lt;HTTPFSHOSTNAME&gt;:14000?user.name=babu&amp;op=homedir&quot;
HTTP/1.1 200 OK
Content-Type: application/json
Transfer-Encoding: chunked
@@ -65,7 +65,7 @@
<hr/>
</div>
<div id="footer">
- <div class="xright">&#169; 2012
+ <div class="xright">&#169; 2013
Cloudera
- <a href="http://maven.apache.org/privacy-policy.html">Privacy Policy</a></div>
View
@@ -1,15 +1,15 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-<!-- Generated by Apache Maven Doxia at Jul 25, 2012 -->
+<!-- Generated by Apache Maven Doxia at Mar 11, 2013 -->
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <title>HttpFS - Hadoop HDFS over HTTP 0.20.2-cdh3u4 - Using HTTP Tools</title>
+ <title>HttpFS - Hadoop HDFS over HTTP 0.20.2-cdh3u6 - Using HTTP Tools</title>
<style type="text/css" media="all">
@import url("./css/maven-base.css");
@import url("./css/maven-theme.css");
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css" media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20120725" />
+ <meta name="Date-Revision-yyyymmdd" content="20130311" />
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
</head>
<body class="composite">
@@ -24,7 +24,7 @@
<div id="breadcrumbs">
<div class="xleft">
- Last Published: 2012-07-25
+ Last Published: 2013-03-11
</div>
<div class="xright"> <a href="http://hadoop.apache.org/" class="externalLink">Apache Hadoop</a>
@@ -44,7 +44,7 @@
</div>
<div id="bodyColumn">
<div id="contentBox">
- <!-- Licensed under the Apache License, Version 2.0 (the "License"); --><!-- you may not use this file except in compliance with the License. --><!-- You may obtain a copy of the License at --><!-- --><!-- http://www.apache.org/licenses/LICENSE-2.0 --><!-- --><!-- Unless required by applicable law or agreed to in writing, software --><!-- distributed under the License is distributed on an "AS IS" BASIS, --><!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. --><!-- See the License for the specific language governing permissions and --><!-- limitations under the License. --><div class="section"><h2>Hadoop HDFS over HTTP 0.20.2-cdh3u4 - Using HTTP Tools<a name="Hadoop_HDFS_over_HTTP_0.20.2-cdh3u4_-_Using_HTTP_Tools"></a></h2><p>[ <a href="./index.html">Go Back</a> ]</p><div class="section"><h3>Security<a name="Security"></a></h3><p>Out of the box HttpFS supports both pseudo authentication and Kerberos HTTP SPNEGO authentication.</p><div class="section"><h4>Pseudo Authentication<a name="Pseudo_Authentication"></a></h4><p>With pseudo authentication the user name must be specified in the <tt>user.name=&lt;USERNAME&gt;</tt> query string parameter of a HttpFS URL. For example:</p><div class="source"><pre>$ curl &quot;http://&lt;HTTFS_HOST&gt;:14000/webhdfs/v1?op=homedir&amp;user.name=babu&quot;</pre></div></div><div class="section"><h4>Kerberos HTTP SPNEGO Authentication<a name="Kerberos_HTTP_SPNEGO_Authentication"></a></h4><p>Kerberos HTTP SPNEGO authentication requires a tool or library supporting Kerberos HTTP SPNEGO protocol.</p><p>IMPORTANT: If using <tt>curl</tt>, the <tt>curl</tt> version being used must support GSS (<tt>curl -V</tt> prints out 'GSS' if it supports it).</p><p>For example:</p><div class="source"><pre>$ kinit
+ <!-- Licensed under the Apache License, Version 2.0 (the "License"); --><!-- you may not use this file except in compliance with the License. --><!-- You may obtain a copy of the License at --><!-- --><!-- http://www.apache.org/licenses/LICENSE-2.0 --><!-- --><!-- Unless required by applicable law or agreed to in writing, software --><!-- distributed under the License is distributed on an "AS IS" BASIS, --><!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. --><!-- See the License for the specific language governing permissions and --><!-- limitations under the License. --><div class="section"><h2>Hadoop HDFS over HTTP 0.20.2-cdh3u6 - Using HTTP Tools<a name="Hadoop_HDFS_over_HTTP_0.20.2-cdh3u6_-_Using_HTTP_Tools"></a></h2><p>[ <a href="./index.html">Go Back</a> ]</p><div class="section"><h3>Security<a name="Security"></a></h3><p>Out of the box HttpFS supports both pseudo authentication and Kerberos HTTP SPNEGO authentication.</p><div class="section"><h4>Pseudo Authentication<a name="Pseudo_Authentication"></a></h4><p>With pseudo authentication the user name must be specified in the <tt>user.name=&lt;USERNAME&gt;</tt> query string parameter of a HttpFS URL. For example:</p><div class="source"><pre>$ curl &quot;http://&lt;HTTFS_HOST&gt;:14000/webhdfs/v1?op=homedir&amp;user.name=babu&quot;</pre></div></div><div class="section"><h4>Kerberos HTTP SPNEGO Authentication<a name="Kerberos_HTTP_SPNEGO_Authentication"></a></h4><p>Kerberos HTTP SPNEGO authentication requires a tool or library supporting Kerberos HTTP SPNEGO protocol.</p><p>IMPORTANT: If using <tt>curl</tt>, the <tt>curl</tt> version being used must support GSS (<tt>curl -V</tt> prints out 'GSS' if it supports it).</p><p>For example:</p><div class="source"><pre>$ kinit
Please enter the password for tucu@LOCALHOST:
$ curl --negotiate -u foo &quot;http://&lt;HTTPFS_HOST&gt;:14000/webhdfs/v1?op=homedir&quot;
Enter host password for user 'foo':</pre></div><p>NOTE: the <tt>-u USER</tt> option is required by the <tt>--negotiate</tt> but it is not used. Use any value as <tt>USER</tt> and when asked for the password press [ENTER] as the password value is ignored.</p></div><div class="section"><h4><a name="Remembering_Who_I_Am">Remembering Who I Am</a> (Establishing an Authenticated Session)<a name="Remembering_Who_I_Am_Establishing_an_Authenticated_Session"></a></h4><p>As most authentication mechanisms, Hadoop HTTP authentication authenticates users once and issues a short-lived authentication token to be presented in subsequent requests. This authentication token is a signed HTTP Cookie.</p><p>When using tools like <tt>curl</tt>, the authentication token must be stored on the first request doing authentication, and submitted in subsequent requests. To do this with curl the <tt>-b</tt> and <tt>-c</tt> options to save and send HTTP Cookies must be used.</p><p>For example, the first request doing authentication should save the received HTTP Cookies.</p><p>Using Pseudo Authentication:</p><div class="source"><pre>$ curl -c ~/.httpfsauth &quot;http://&lt;HTTPFS_HOST&gt;:14000/webhdfs/v1?op=homedir&amp;user.name=babu&quot;</pre></div><p>Using Kerberos HTTP SPNEGO authentication:</p><div class="source"><pre>$ curl --negotiate -u foo -c ~/.httpfsauth &quot;http://&lt;HTTPFS_HOST&gt;:14000/webhdfs/v1?op=homedir&quot;</pre></div><p>Then, subsequent requests forward the previously received HTTP Cookie:</p><div class="source"><pre>$ curl -b ~/.httpfsauth &quot;http://&lt;HTTPFS_HOST&gt;:14000/webhdfs/v1?op=liststatus&quot;</pre></div><p>[ <a href="./index.html">Go Back</a> ]</p></div></div></div>
@@ -54,7 +54,7 @@
<hr/>
</div>
<div id="footer">
- <div class="xright">&#169; 2012
+ <div class="xright">&#169; 2013
Cloudera
- <a href="http://maven.apache.org/privacy-policy.html">Privacy Policy</a></div>
Oops, something went wrong.

0 comments on commit f42a6f9

Please sign in to comment.