Unable to connect to a Hadoop cluster with Kerberos security configured. #42
Comments
|
Hi, I'm not an expert in security but I think that kerberos uses credentials of the user that running the process. So if you run h-rider with the user that have access to your cluster it should be ok. After all h-rider uses regular hbase client so if you could do it on your own the h-rider should work as well. Unfortunately I don't have environment to check it. Best Regards, |
|
Thanks for the reply. Yes, I configure the Kerberos ticket on my macine In HRider's case, which hbase-site.xml it is using? The one in its own lib I will try that and let you know. Regards, On Mon, Jun 3, 2013 at 10:05 AM, Igor Cher notifications@github.com wrote:
|
|
So I latest downloaded the code and built it and now when I try to run it on Linux, I get the following exception: 2013-06-03 15:41:00,832 INFO [AWT-EventQueue-0] JavaPackage : Loading classes from jar '/Users/syunus/Downloads/hrider-master/Users/syunus/Downloads/hrider-master/target/h-rider-1.0.7.0.jar' I get the similar exception when I download and run the packaged the 1.0.7 version from the site. Thanks, |
|
Just adding the exception that I get when I directly try to run the 1.0.7 version (the pre-packaged one downloaded from the main site.) 2013-06-03 16:16:45,256 INFO [AWT-EventQueue-0] RunnableAction : Action 'compare-versions' started. The following path does not exist. Why is it trying to go there? The older version that I am using 1.0.3, works fine with non-Kerberos clusters, as mentioned before. Thanks, |
|
Hi, The h-rider doesn't use hbase-site.xml or any other XML. It simply configures Configuration object directly before connecting to hbase. Can you tell me how do you configure kerberos in hbase-site.xml? In regards to the errors you get, are you sure you have the latest code? The jar still contains the bug (although it should not appear when the h-rider is executed from the jar) but the code is fixed. So running in debug or recompiling the code should work. |
|
First, just to clarify one thing, HBase first loads the hbase-default.xml and core-default.xml which comes bundled in the hadoop and hbase jars as dependencies and then loads these custom ones (*-site.xml) on top of that. Point being, the client does need some hbase/core config files on its classpath. In most of the cases it is the one packaged in the dependent libs which suffices with user only have to enter zookeeper.quorum. This is what is happening, I believe, with the H-rider too where you only get the zookeeper.quorom from the user. Issue#1: 2013-06-04 09:32:37,809 INFO [AWT-EventQueue-0] RunnableAction : Action 'compare-versions' started. I am running it from Users/syunus/Downloads/h-rider-1.0.7.0-hbase-0.94.1, so I don't get why it is appending the extra path. I then downloaded the code, built it myself (using maven profile 0.94.1) and when I run it, I see the same error. Of course, the directory paths are different but still incorrect. As mentioned, the 1.0.3 version is working fine. Issue# 2: I have set those for thee xmls, in my regular Java client and it works. I have tried adding these configuration files on the class path of h-rider with both versions (1.0.3 and 1.0.7) and H-rider does not work. If I run h-rider, without any of these custom configuration files then I get 'Connection reset by peer' exceptions when I try to connect to a Kerberos configured cluster. With these files, I get NoClassFoundError (KerberosName) which is kind of strange as this class does exist in the included hadoop library. I am still looking into it in case I am adding thoes configuration files incorrectly in the class path. But in any case, as per my understanding (and experience with the regular Java client) we do need to have Kerberos-configured hbase/core-site.xml files on the class path which will be read by the HBaseConfiguration.create() which reads all files on the path with hbase-*.xml. I added some debug statements in the code in the Connection.java and ConnectionDetails and that is how I got the messages above. Let me know if any question. |
|
Hi, In regards to the first issue, it fails because by some reason h-rider cannot read the jar version from its manifest file. I experienced this only when I ran it from the Intellij in debug mode. In regular execution it should work. Regarding the path this is a known issue I will check the security configuration required for kerberos. |
|
Hi, Try to add the following code to the ConnectionDetails.createConfig() method: config.set("hbase.security.authentication", "kerberos"); Let me know if it helped you. Best Regards, |
|
Nope, that doesn't help. I have to copy my hbase/core-site.xml files in the On Tue, Jun 4, 2013 at 10:15 AM, Igor Cher notifications@github.com wrote:
|
|
Ok, I made it work. So what I did: 1- Included those hbase-site and core-site XML files (in the src/main/resources) with correct information about our cluster's specific Kerberos configuration. These are exactly same what we use in other HBase clients. 2- Set the following System properties before loading the configuration (creating the Configuration object) for HBase (I did it in ConnectionDetails.java): Of course you need a kerberos ticket on your local machine but that is something independent of H-Rider and a general step required to make Kerberos work on your machine. 3- Rebuilt it with the cdh4 profile (not 0.94.1). It now works. This is for 1.0.3. I am unable to run 1.0.7 in general, kerberos or not. The thing to note is that this is of course hard-coded solution and not an ideal deployment/design. I mean that I have to include the hbase/core-site XMLs in the code and rebuild. So in future if any of the Kerberos settings change I have update the config files and build it again. Plus more importantly, this specific jar that I have generated will now ONLY work for this cluster and no where else. But at least we know that how we can use Kerberos with H-Rider :) Regards, |
|
Good to know you handled it. You probably cannot run 1.0.7 because it is compiled with 0.94.1 hbase. At some point I added to h-rider a feature which is supported by hbase only starting from 0.94 version so two other profiles are not supported any more. Best Regards, |
|
Hi, Instead of hardcoding System properties, you can also set it at launchtime |
|
Thanks.
Feel free to prepare a merge request, and we will add it!
On Nov 10, 2017 15:22, "Damien Claveau" <notifications@github.com> wrote:
Hi,
Instead of hardcoding System properties, you can also set it at launchtime
java -Djava.security.krb5.conf=/etc/krb5.conf -jar
target/h-rider-1.0.9-SNAPSHOT.jar
and add core-site.xml,hdfs-site.xml and hbase-site.xml in the classpath.
Works great for me.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#42 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ACSLZDeYgtfI6Nnvq-YdN-eQ40hnWyaeks5s1E4AgaJpZM4AtAqY>
.
|
Hi,
I am using the HBase UI tool 'H-Rider'. I was able to use it with a cluster which does not have Kerberos security configured but was getting 'failed to connect' errors when trying to connect to a cluster with Kerberos authentication. Is it possible to connect H-Rider with such a cluster? I couldn't find anything where I could set this settings or properties file. I am able to connect to this secured cluster through a Java HBase client so Kerberos on my machine is setup properly, I believe.
Thanks & Regards,
Shahab
The text was updated successfully, but these errors were encountered: