hoodie-hive-hundle don't have hive jars #736
Comments
|
We are revisiting jar/bundles ground up.. Will factor this in and get back. you can find some progress in the hackathon-0619 branch |
|
@cdmikechen |
|
@eisig #java -cp $HOODIE_HIVE_UBER_JAR:${HADOOP_HIVE_JARS}:${HADOOP_CONF_DIR} com.uber.hoodie.hive.HiveSyncTool "$@"
java -cp $HOODIE_HIVE_UBER_JAR:${HADOOP_HIVE_JARS}:${HADOOP_CONF_DIR}:${HIVE_HOME}/lib/* com.uber.hoodie.hive.HiveSyncTool "$@"and it can run. |
|
@cdmikechen these jars are in the hive installation, thats why we don't bundle them. is it possible the the script is not just picking them up? are you able to repro this on top of #751 and see if this still is an issue? |
|
@cdmikechen any updates on this? |
|
@vinothchandar |
|
No worries. |
|
Closing due to inactivity |
It's still same issue on 0.5.0 version |
|
https://github.com/apache/incubator-hudi/blob/master/hudi-hive/run_sync_tool.sh#L30 Adds in all the jars.. and we use the script in the docker setup successfully. Could you reproduce this in the docker setup we have and we can go from there? Trying to understand if this an HIVE_HOME config issue.. |
Hi I add below code inside run_sync_tool, then It works , but I did not try it in docker otherwise there will be throw this kind of exception Caused by: org.datanucleus.exceptions.NucleusException: Attempt to invoke the "dbcp-builtin" plugin to create a ConnectionPool gave an error : The specified datastore driver ("org.apache.derby.jdbc.EmbeddedDriver") was not found in the CLASSPATH. Please check your CLASSPATH specification, and the name of the driver.
at org.datanucleus.store.rdbms.ConnectionFactoryImpl.generateDataSources(ConnectionFactoryImpl.java:232)
at org.datanucleus.store.rdbms.ConnectionFactoryImpl.initialiseDataSources(ConnectionFactoryImpl.java:117)
at org.datanucleus.store.rdbms.ConnectionFactoryImpl.<init>(ConnectionFactoryImpl.java:82)
... 58 more
Caused by: org.datanucleus.store.rdbms.connectionpool.DatastoreDriverNotFoundException: The specified datastore driver ("org.apache.derby.jdbc.EmbeddedDriver") was not found in the CLASSPATH. Please check your CLASSPATH specification, and the name of the driver.
at org.datanucleus.store.rdbms.connectionpool.AbstractConnectionPoolFactory.loadDriver(AbstractConnectionPoolFactory.java:58)
at org.datanucleus.store.rdbms.connectionpool.DBCPBuiltinConnectionPoolFactory.createConnectionPool(DBCPBuiltinConnectionPoolFactory.java:49)
at org.datanucleus.store.rdbms.ConnectionFactoryImpl.generateDataSources(ConnectionFactoryImpl.java:213)
... 60 more |
|
I think the code assumes everything is in the hadoop conf In either case, still cant understand how adding conf to the classpath will resolve the driver not being found.. is there a direct link? i.e do you know why exactly adding the |
|
Hi Team, datanucleus packages are present in Any pointers ? |
|
This works for Hive 3 also, if we include all the jars in $HIVE_HOME/lib/* folder while running hive_sync.sh |
when using
run_sync_tool.shto sync a table like that:hoodie return this error:
I found this class is in in
hoodie-hive-bundlepom (hive-metastore), but when packaged ahoodie-hive-bundle-0.4.8-SNAPSHOT.jar, this class is missing. And inrun_sync_tool.sh, it doesn't include hive lib abouthive-metastoreand its dependencies lib.I think maybe we can include hive jars in
hoodie-hive-bundlepom.The text was updated successfully, but these errors were encountered: