Beeline can not connect to Hive2

[tumblewind]

Connection failed on host chs-vmb-469-mn00xxxxx.cloud:10001 (Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/SPARK2/package/scripts/alerts/alert_spark2_thrift_port.py", line 147, in execute
Execute(cmd, user=sparkuser, path=[beeline_cmd], timeout=CHECK_COMMAND_TIMEOUT_DEFAULT)
File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line 166, in init
self.env.run()
File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 160, in run
self.run_action(resource, action)
File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 124, in run_action
provider_action()
File "/usr/lib/ambari-agent/lib/resource_management/core/providers/system.py", line 263, in action_run
returns=self.resource.returns)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 72, in inner
result = function(command, **kwargs)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 102, in checked_call
tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy, returns=returns)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 150, in _call_wrapper
result = _call(command, **kwargs_copy)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 314, in _call
raise ExecutionFailed(err_msg, code, out, err)
ExecutionFailed: Execution of '! /usr/hdp/current/spark2-client/bin/beeline -u 'jdbc:hive2://chs-vmb-469-mn002.us-south.ae.appdomain.cloud:10001/default;transportMode=http' -e '' 2>&1| awk '{print}'|grep -i -e 'Connection refused' -e 'Invalid URL' -e 'Error: Could not open'' returned 1. Error: Could not open client transport with JDBC Uri: jdbc:hive2://chs-vmb-469-mn002.us-south.ae.appdomain.cloud:10001/default;transportMode=http: Could not create http connection to jdbc:hive2://chs-vmb-469-mn002.xxxxappdomain.cloud:10001/default;transportMode=http. HTTP Response code: 405 (state=08S01,code=0)
Error: Could not open client transport with JDBC Uri: jdbc:hive2://chs-vmb-469-mn002.xxxxappdomain.cloud:10001/default;transportMode=http: Could not create http connection to jdbc:hive2://chs-vmb-469-mn002.us-south.ae.appdomain.cloud:10001/default;transportMode=http. HTTP Response code: 405 (state=08S01,code=0)
)

[gabydreckschmidt]

@tumblewind
I spoke with the development team and it seems that you haven't followed the instructions in the documentation completely. You are connecting to 10001 port instead of 8443. Also the JDBC URL is not complete.

• Please see https://cloud.ibm.com/docs/services/AnalyticsEngine?topic=AnalyticsEngine-working-with-hive#connecting-to-the-hive-server for the port numbers.
• See https://cloud.ibm.com/docs/services/AnalyticsEngine?topic=AnalyticsEngine-retrieve-endpoints to ensure you have the correct JDBC URL.

[tumblewind]

This failure is from the Analytics engine Hive2 status check. Your service is not doing it right.

[tumblewind]

Again we followed the documentation to use the Hive JDBC. Due to unstable Thrift server it does not simply work. Not sure how this service qualified to be publicly available ?

[tumblewind]

After many hours of frustration I wrote a hive jdbc client and used the following url

jdbc:hive2://chs-vxxxxxx.us-south.ae.appdomain.cloud:8443/;ssl=true;;sslTrustStore=/Users/smkhan/ANALYTICS-ENGINE/Hive/us-southaeappdomaincloud.jks;trustStorePassword=changeit;transportMode=http;httpPath=gateway/default/hive", "clsadmin", "xxxxxxxxx");

Here is the error

ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. Set system property 'log4j2.debug' to show Log4j2 internal initialization logging.
05:08:33.053 [main] ERROR org.apache.hive.jdbc.HiveConnection - Error opening session
org.apache.hive.org.apache.thrift.transport.TTransportException: HTTP Response code: 404
at org.apache.hive.org.apache.thrift.transport.THttpClient.flushUsingHttpClient(THttpClient.java:262) ~[hive-jdbc-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78]
at org.apache.hive.org.apache.thrift.transport.THttpClient.flush(THttpClient.java:313) ~[hive-jdbc-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78]
at org.apache.hive.org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:73) ~[hive-jdbc-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78]
at org.apache.hive.org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62) ~[hive-jdbc-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78]
at org.apache.hive.service.rpc.thrift.TCLIService$Client.send_OpenSession(TCLIService.java:170) ~[hive-jdbc-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78]
at org.apache.hive.service.rpc.thrift.TCLIService$Client.OpenSession(TCLIService.java:162) ~[hive-jdbc-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78]
at org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:796) [hive-jdbc-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78]
at org.apache.hive.jdbc.HiveConnection.(HiveConnection.java:305) [hive-jdbc-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78]
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107) [hive-jdbc-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78]
at java.sql.DriverManager.getConnection(DriverManager.java:664) [?:1.8.0_151]
at java.sql.DriverManager.getConnection(DriverManager.java:247) [?:1.8.0_151]
at com.ibm.whc.appfmwk.hive.TestHive.main(TestHive.java:31) [classes/:?]
java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://chs-vxxxxxx.us-south.ae.appdomain.cloud:8443/;ssl=true;;sslTrustStore=/Users/smkhan/ANALYTICS-ENGINE/Hive/us-southaeappdomaincloud.jks;trustStorePassword=changeit;transportMode=http;httpPath=gateway/default/hive: Could not establish connection to jdbc:hive2://chs-vmb-469-mn003.us-south.ae.appdomain.cloud:8443/;ssl=true;;sslTrustStore=/Users/smkhan/ANALYTICS-ENGINE/Hive/us-southaeappdomaincloud.jks;trustStorePassword=changeit;transportMode=http;httpPath=gateway/default/hive: HTTP Response code: 404
at org.apache.hive.jdbc.HiveConnection.(HiveConnection.java:333)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at com.ibm.whc.appfmwk.hive.TestHive.main(TestHive.java:31)
Caused by: java.sql.SQLException: Could not establish connection to jdbc:hive2://chs-vmb-xxxxx.us-south.ae.appdomain.cloud:8443/;ssl=true;;sslTrustStore=/Users/smkhan/ANALYTICS-ENGINE/Hive/us-southaeappdomaincloud.jks;trustStorePassword=changeit;transportMode=http;httpPath=gateway/default/hive: HTTP Response code: 404
at org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:815)
at org.apache.hive.jdbc.HiveConnection.(HiveConnection.java:305)
... 4 more
Caused by: org.apache.hive.org.apache.thrift.transport.TTransportException: HTTP Response code: 404
at org.apache.hive.org.apache.thrift.transport.THttpClient.flushUsingHttpClient(THttpClient.java:262)
at org.apache.hive.org.apache.thrift.transport.THttpClient.flush(THttpClient.java:313)
at org.apache.hive.org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:73)
at org.apache.hive.org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62)
at org.apache.hive.service.rpc.thrift.TCLIService$Client.send_OpenSession(TCLIService.java:170)
at org.apache.hive.service.rpc.thrift.TCLIService$Client.OpenSession(TCLIService.java:162)
at org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:796)
... 5 more

I see a Spark2 alert where I got the initial error from. Seems like Thrift server is not working. I will appreciate if development team can look into the current state of the service offering. It simply not working for us. We are in Watson health team. I am not sure how to reach them internally

[gabydreckschmidt]

@tumblewind I'm sorry to hear that you are still having difficulties. I spoke to my development contact on Analytics Engine and she asks for you to contact the Analytics Engine dev support team on the Slack channel #iae-support (https://ibm-cloudplatform.slack.com/archives/C7DGE3CUV). She can see that the URL is still incorrect and says that you don't need to specify the .jks file. Your problem is no longer a documentation issue.