[exchangis1.0.0]执行任务报错Run Error Message:java.lang.reflect.InvocationTargetException java.lang.reflect.InvocationTargetException: null #383
Comments
|
I found "Caused by: java.lang.NoClassDefFoundError: org/apache/hive/hcatalog/mapreduce/HCatOutputFormat 41. at org.apache.sqoop.mapreduce.DataDrivenImportJob.getOutputFormatClass(DataDrivenImportJob.java:178)" and "Caused by: java.lang.ClassNotFoundException: org.apache.hive.hcatalog.mapreduce.HCatOutputFormat" in your log. Have you installed hive environment correctly? |
|
The latest version of exchangis is 1.1.2, the latest dss version is 1.1.2, and the latest linkis version is 1.4.0 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
[exchangis-server (1).log](https://github.com/WeBankFinTech/Exchangis/files/9897566/exchangis-server.1.log) 1. Listening for transport dt_socket at address: 37946 2. 2022-10-31 10:34:06.234 INFO [EngineConn-Asyn-Thread-Thread-0] org.apache.linkis.engineconn.executor.listener.EngineConnAsyncListenerBus 41 info - EngineConn-Asyn-Thread-Thread-0 begin. 3. 2022-10-31 10:34:06.386 INFO [EngineConn-Asyn-Thread-Thread-0] org.apache.linkis.engineconn.acessible.executor.service.DefaultManagerService 41 info - success to send engine heartbeat report to localhost:9101,status:Running,msg:null 4. 2022-10-31 10:34:06.623 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 41 info - Try to execute params.{sqoop.args.password=Lucy867419., sqoop.args.driver=com.mysql.jdbc.Driver, sqoop.args.null.string=\\N, sqoop.args.hcatalog.partition.keys=ds, sqoop.args.columns=id,name,age,grade, sqoop.args.table=user, sqoop.args.where=1=1, sqoop.mode=import, sqoop.args.num.mappers=1, sqoop.args.hcatalog.table=user, sqoop.args.hcatalog.partition.values=1667097232847, sqoop.args.username=root, sqoop.args.fields.terminated.by=�, sqoop.args.null.non.string=\\N, sqoop.args.connect=jdbc:mysql://127.0.0.1:3306/test, sqoop.args.hcatalog.database=test_ind} 5. 2022-10-31 10:34:06.638 INFO [EngineConn-Asyn-Thread-Thread-0] org.apache.linkis.engineconn.acessible.executor.service.DefaultManagerService 41 info - success to send engine heartbeat report to localhost:9101,status:Busy,msg:null 6. 2022-10-31 10:34:06.655 INFO [EngineConn-Asyn-Thread-Thread-0] org.apache.linkis.engineconn.acessible.executor.service.DefaultManagerService 41 info - success to send engine heartbeat report to localhost:9101,status:Running,msg:null 7. 2022-10-31 10:34:06.770 WARN [Linkis-Default-Scheduler-Thread-1] org.apache.sqoop.tool.BaseSqoopTool 1021 applyCredentialsOptions - Setting your password on the command-line is insecure. Consider using -P instead. 8. 2022-10-31 10:34:06.772 WARN [Linkis-Default-Scheduler-Thread-1] org.apache.sqoop.tool.BaseSqoopTool 1530 validateHCatalogOptions - Output field/record delimiter options are not useful in HCatalog jobs for most of the output types except text based formats is text. It is better to use --hive-import in those cases. For non text formats, 9. 2022-10-31 10:34:06.818 WARN [Linkis-Default-Scheduler-Thread-1] org.apache.sqoop.ConnFactory 132 getManager - Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time. 10. 2022-10-31 10:34:06.824 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.sqoop.manager.SqlManager 98 initOptionDefaults - Using default fetchSize of 1000 11. 2022-10-31 10:34:06.824 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.sqoop.tool.CodeGenTool 92 generateORM - Beginning code generation 12. 2022-10-31 10:34:06.959 INFO [PollingServerListUpdater-0] com.netflix.config.ChainedDynamicProperty 115 checkAndFlip - Flipping property: linkis-cg-engineconnmanager.ribbon.ActiveConnectionsLimit to use NEXT property: niws.loadbalancer.availabilityFilteringRule.activeConnectionsLimit = 2147483647 13. 2022-10-31 10:34:07.343 INFO [PollingServerListUpdater-1] com.netflix.config.ChainedDynamicProperty 115 checkAndFlip - Flipping property: linkis-cg-linkismanager.ribbon.ActiveConnectionsLimit to use NEXT property: niws.loadbalancer.availabilityFilteringRule.activeConnectionsLimit = 2147483647 14. 2022-10-31 10:34:07.356 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.sqoop.manager.SqlManager 757 execute - Executing SQL statement: SELECT t.* FROM user AS t WHERE 1=0 15. 2022-10-31 10:34:07.365 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.sqoop.orm.CompilationManager 85 findHadoopJars - $HADOOP_MAPRED_HOME is not set 16. 2022-10-31 10:34:08.768 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.sqoop.orm.CompilationManager 330 jar - Writing jar file: /tmp/sqoop-hadoop/compile/60075ad13be812f20536b73e1ab8070e/user.jar 17. org.apache.linkis.engineconnplugin.sqoop.client.utils.JarLoader@1ffd1daa 18. 2022-10-31 10:34:08.784 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.sqoop.mapreduce.ImportJobBase 235 runImport - Beginning import of user 19. 2022-10-31 10:34:08.946 WARN [Linkis-Default-Scheduler-Thread-1] org.apache.hadoop.util.NativeCodeLoader 62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 20. 2022-10-31 10:34:08.956 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.hadoop.conf.Configuration.deprecation 1173 warnOnceIfDeprecated - mapred.jar is deprecated. Instead, use mapreduce.job.jar 21. 2022-10-31 10:34:08.972 ERROR [Linkis-Default-Scheduler-Thread-1] org.apache.linkis.engineconnplugin.sqoop.client.LinkisSqoopClient 69 run - Run Error Message:java.lang.reflect.InvocationTargetException java.lang.reflect.InvocationTargetException: null 22. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_342] 23. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_342] 24. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_342] 25. at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_342] 26. at org.apache.linkis.engineconnplugin.sqoop.client.LinkisSqoopClient.run(LinkisSqoopClient.java:67) ~[linkis-engineplugin-sqoop-1.1.1.jar:1.1.1] 27. at org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor$$anonfun$runSqoop$1.apply$mcI$sp(SqoopOnceCodeExecutor.scala:75) ~[linkis-engineplugin-sqoop-1.1.1.jar:1.1.1] 28. at org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor$$anonfun$runSqoop$1.apply(SqoopOnceCodeExecutor.scala:71) ~[linkis-engineplugin-sqoop-1.1.1.jar:1.1.1] 29. at org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor$$anonfun$runSqoop$1.apply(SqoopOnceCodeExecutor.scala:71) ~[linkis-engineplugin-sqoop-1.1.1.jar:1.1.1] 30. at org.apache.linkis.common.utils.Utils$.tryCatch(Utils.scala:40) ~[linkis-common-1.1.1.jar:1.1.1] 31. at org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor.runSqoop(SqoopOnceCodeExecutor.scala:76) ~[linkis-engineplugin-sqoop-1.1.1.jar:1.1.1] 32. at org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor$$anon$1.run(SqoopOnceCodeExecutor.scala:56) ~[linkis-engineplugin-sqoop-1.1.1.jar:1.1.1] 33. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_342] 34. at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_342] 35. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_342] 36. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.0_342] 37. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_342] 38. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_342] 39. at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_342] 40. Caused by: java.lang.NoClassDefFoundError: org/apache/hive/hcatalog/mapreduce/HCatOutputFormat 41. at org.apache.sqoop.mapreduce.DataDrivenImportJob.getOutputFormatClass(DataDrivenImportJob.java:178) ~[sqoop-1.4.6-hadoop200.jar:1.1.1] 42. at org.apache.sqoop.mapreduce.ImportJobBase.configureOutputFormat(ImportJobBase.java:94) ~[sqoop-1.4.6-hadoop200.jar:1.1.1] 43. at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:259) ~[sqoop-1.4.6-hadoop200.jar:1.1.1] 44. at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:673) ~[sqoop-1.4.6-hadoop200.jar:?] 45. at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:497) ~[sqoop-1.4.6-hadoop200.jar:?] 46. at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605) ~[sqoop-1.4.6-hadoop200.jar:?] 47. at org.apache.linkis.engineconnplugin.sqoop.client.Sqoop.run(Sqoop.java:157) ~[linkis-engineplugin-sqoop-1.1.1.jar:1.1.1] 48. at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) ~[hadoop-common-2.7.2.jar:?] 49. at org.apache.linkis.engineconnplugin.sqoop.client.Sqoop.runSqoop(Sqoop.java:190) ~[linkis-engineplugin-sqoop-1.1.1.jar:1.1.1] 50. at org.apache.linkis.engineconnplugin.sqoop.client.Sqoop.runTool(Sqoop.java:237) ~[linkis-engineplugin-sqoop-1.1.1.jar:1.1.1] 142. at org.apache.linkis.engineconnplugin.sqoop.client.Sqoop.runTool(Sqoop.java:297) ~[linkis-engineplugin-sqoop-1.1.1.jar:1.1.1] 143. at org.apache.linkis.engineconnplugin.sqoop.client.Sqoop.main(Sqoop.java:301) ~[linkis-engineplugin-sqoop-1.1.1.jar:1.1.1] 144. ... 18 more 145. Caused by: java.lang.ClassNotFoundException: org.apache.hive.hcatalog.mapreduce.HCatOutputFormat 146. at java.net.URLClassLoader.findClass(URLClassLoader.java:387) ~[?:1.8.0_342] 147. at java.lang.ClassLoader.loadClass(ClassLoader.java:418) ~[?:1.8.0_342] 148. at org.apache.linkis.engineconnplugin.sqoop.client.utils.JarLoader.loadClass(JarLoader.java:158) ~[linkis-engineplugin-sqoop-1.1.1.jar:1.1.1] 149. at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ~[?:1.8.0_342] 150. at org.apache.sqoop.mapreduce.DataDrivenImportJob.getOutputFormatClass(DataDrivenImportJob.java:178) ~[sqoop-1.4.6-hadoop200.jar:1.1.1] 151. at org.apache.sqoop.mapreduce.ImportJobBase.configureOutputFormat(ImportJobBase.java:94) ~[sqoop-1.4.6-hadoop200.jar:1.1.1] 152. at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:259) ~[sqoop-1.4.6-hadoop200.jar:1.1.1] 153. at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:673) ~[sqoop-1.4.6-hadoop200.jar:?] 154. at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:497) ~[sqoop-1.4.6-hadoop200.jar:?] 155. at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605) ~[sqoop-1.4.6-hadoop200.jar:?] 156. at org.apache.linkis.engineconnplugin.sqoop.client.Sqoop.run(Sqoop.java:157) ~[linkis-engineplugin-sqoop-1.1.1.jar:1.1.1] 157. at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) ~[hadoop-common-2.7.2.jar:?] 158. at org.apache.linkis.engineconnplugin.sqoop.client.Sqoop.runSqoop(Sqoop.java:190) ~[linkis-engineplugin-sqoop-1.1.1.jar:1.1.1] 159. at org.apache.linkis.engineconnplugin.sqoop.client.Sqoop.runTool(Sqoop.java:237) ~[linkis-engineplugin-sqoop-1.1.1.jar:1.1.1] 160. at org.apache.linkis.engineconnplugin.sqoop.client.Sqoop.runTool(Sqoop.java:297) ~[linkis-engineplugin-sqoop-1.1.1.jar:1.1.1] 161. at org.apache.linkis.engineconnplugin.sqoop.client.Sqoop.main(Sqoop.java:301) ~[linkis-engineplugin-sqoop-1.1.1.jar:1.1.1] 162. ... 18 more 163. 164. 2022-10-31 10:34:08.985 ERROR [Linkis-Default-Scheduler-Thread-1] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 62 error - SqoopOnceApp_0 has failed with old status Running, now stop it. 165. 2022-10-31 10:34:08.987 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 53 transition - Waitiing lock release, to change status Running=>Failed. 166. 2022-10-31 10:34:08.988 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 55 transition - Finished wait lock release, to change status Running=>Failed. 167. 2022-10-31 10:34:08.988 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 41 info - org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor@4aee166d changed status Running => Failed. 168. 2022-10-31 10:34:08.990 ERROR [Linkis-Default-Scheduler-Thread-1] org.apache.linkis.engineconn.once.executor.execution.OnceEngineConnExecution 58 error - Unknown reason. 169. 2022-10-31 10:34:08.991 ERROR [Linkis-Default-Scheduler-Thread-1] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 62 error - SqoopOnceApp_0 has failed with old status Failed, now stop it. 170. 2022-10-31 10:34:08.997 WARN [Linkis-Default-Scheduler-Thread-1] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 50 warn - Executor(SqoopOnceApp_0) exit by close. 171. 2022-10-31 10:34:08.997 WARN [Linkis-Default-Scheduler-Thread-1] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 50 warn - Executor(SqoopOnceApp_0) exit by close. 172. 2022-10-31 10:34:08.998 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 41 info - All codes completed, now to stop SqoopEngineConn. 173. 2022-10-31 10:34:08.999 INFO [Thread-31] com.netflix.loadbalancer.PollingServerListUpdater 53 run - Shutting down the Executor Pool for PollingServerListUpdater 174. 2022-10-31 10:34:09.000 INFO [EngineConn-Asyn-Thread-Thread-1] org.apache.linkis.engineconn.executor.listener.EngineConnAsyncListenerBus 41 info - EngineConn-Asyn-Thread-Thread-1 begin. 175. 2022-10-31 10:34:09.014 INFO [EngineConn-Asyn-Thread-Thread-0] org.apache.linkis.engineconn.acessible.executor.service.DefaultManagerService 41 info - success to send engine heartbeat report to localhost:9101,status:Failed,msg:null 176. 2022-10-31 10:34:09.018 INFO [EngineConn-Asyn-Thread-Thread-1] org.apache.linkis.engineconn.acessible.executor.service.DefaultManagerService 41 info - success to send engine heartbeat report to localhost:9101,status:Failed,msg:nullThe text was updated successfully, but these errors were encountered: