SparkSQL Connection error: v2.3.2

[binodsarkar]

SparkSQL Connection error
Spark sql connection through thrift s not working. Please find the error

Error while loading DB schema
Error: TApplicationException: Required field 'client_protocol' is unset!
Struct:TOpenSessionReq(client_protocol:null, username:anonymous, password:)
at Client.OpenSession (.../node_modules/jshs2/lib/HiveConnection.js:57:18)
at client._reqs.(anonymous function) (.../node_modules/thrift/lib/nodejs/lib/thrift/connection.js:151:13)
at exports.Client.TCLIServiceClient.recv_OpenSession (.../node_modules/jshs2/idl/Thrift_0.9.3_Hive_2.1.1/TCLIService.js:2327:12)
at .../node_modules/thrift/lib/nodejs/lib/thrift/connection.js:157:41
at .../node_modules/thrift/lib/nodejs/lib/thrift/buffered_transport.js:48:5
at frames.filter.map.f (.../node_modules/@cubejs-backend/hive-driver/driver/TSaslTransport.js:105:52)
at Array.map (<anonymous>)
at Socket.<anonymous> (.../node_modules/@cubejs-backend/hive-driver/driver/TSaslTransport.js:105:43)
at Socket.emit (events.js:198:13)
at addChunk (_stream_readable.js:287:12)

[tromika]

Thrift_0.9.3_Hive_2.1.1

Seems there is a version mismatch between the client and the thrift server.

[binodsarkar]

Thrift_0.9.3_Hive_2.1.1

Seems there is a version mismatch between the client and the thrift server.

In that case how to upgrade cubejs hive version? Because my warehouse version in 2.3.2 or any idea how to use sparksql jdbc driver?

And I have tested after downgrading hive spark etc on my local m/c but still not working

[tromika]

by default 2.3.4 bundled see https://github.com/cube-js/cube.js/tree/master/packages/cubejs-hive-driver/idl/Hive_2.3.4. So you have to replace the files with the 2.3.2 version inside your cubejs installation.

[tromika]

I guess it doesn't solve your problem but might solve someone's in the future, you can find older hive and thrift idls in https://github.com/imjuni/jshs2/tree/master/idl

[binodsarkar]

by default 2.3.4 bundled see https://github.com/cube-js/cube.js/tree/master/packages/cubejs-hive-driver/idl/Hive_2.3.4. So you have to replace the files with the 2.3.2 version inside your cubejs installation.

Connection error for bug fixing version change?

[binodsarkar]

I guess it doesn't solve your problem but might solve someone's in the future, you can find older hive and thrift idls in https://github.com/imjuni/jshs2/tree/master/idl

Sorry, the exact version 2.3.4 also not resolved my issue. I think I am missing any simple thing? And Any IDEA how to use spark sql jdbc because in my another node application jdbc works fine and I have also seen that there is a driver for sparksql in cubejs cubejs jdbc
https://github.com/cube-js/cube.js/blob/fd39c95136b00668026999faad5d19712c7df65d/packages/cubejs-jdbc-driver/driver/JDBCDriver.js#L60

[paveltiunov]

@binodsarkar Hey Binod! Hive is very sensitive with these version stuff. You should have correct IDL in place: https://github.com/cube-js/cube.js/tree/master/packages/cubejs-hive-driver#contributing-missing-hive-protocol.

On JDBC side: unfortunately node-maven isn't able to install it due to it doesn't support variables AFAIK.

[CBox]

Hey, just to make sure what version of SparkSQL is working with CubeJS?

[tromika]

Hi @CBox, we are on 2.3.x and it's working properly. I can't say anything about 2.4.x or 3.x unfortunately but keep in mind for the CubeJs and Spark integration the Hive versions matters.

[paveltiunov]

This issue is currently outside of our roadmap and we're looking for contributions here!