Fixing Spark & Hive jar mismatch for hive registration via hoodie-hive

[vinothchandar]

This is a hairy issue. Spark comes with its version of Hive, which could be different from what version of Hive itself being run. Ultimately since Hoodie is a spark app, this means any incompatibility here prevents hivesync being done inline from a tool like HoodieDeltaStreamer, and forced to run a separate task on a workflow engine..

We need a long term fix for this, to be able to seamlessly register Hive tables from a Hoodie datasource (like you can with HiveContext)

* Spark 2.1 still uses hive 1.2, which is different from the CDH version. Running into this (https://www.bountysource.com/issues/1366324-newer-versions-of-jdbc-driver-does-not-work-with-older-hiveserver2)
* What does Spark itself do? Thin client to talk to Hive? 

[yssharma]

Looks like Spark is having its own dependencies for hive - https://mvnrepository.com/artifact/org.spark-project.hive

[vinothchandar]

yes Spark bundles hive as well.

[prazanna]

This may not be fixable in Hoodie other than just working with Spark and Hive versions that are compatible with each other.

[vinothchandar]

#262 is related, in terms of API we want to package this under

[vinothchandar]

spark datasource can now sync to hive..