Skip to content

Conversation

@xiangfu0
Copy link
Contributor

@xiangfu0 xiangfu0 commented Jul 8, 2022

Seeing this exception when ingestion orc file:

2022/07/08 00:12:08.441 ERROR [TaskRunner] [TaskStateModelFactory-task_thread-1] Problem running the task, report task as FAILED.
java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FileSystem
	at java.lang.Class.getDeclaredConstructors0(Native Method) ~[?:?]
	at java.lang.Class.privateGetDeclaredConstructors(Class.java:3137) ~[?:?]
	at java.lang.Class.getConstructor0(Class.java:3342) ~[?:?]
	at java.lang.Class.getConstructor(Class.java:2151) ~[?:?]
	at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:356) ~[classes/:?]
	at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:325) ~[classes/:?]
	at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:306) ~[classes/:?]
	at org.apache.pinot.spi.data.readers.RecordReaderFactory.getRecordReaderByClass(RecordReaderFactory.java:147) ~[classes/:?]
	at org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl.getRecordReader(SegmentIndexCreationDriverImpl.java:125) ~[classes/:?]
	at org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl.init(SegmentIndexCreationDriverImpl.java:101) ~[classes/:?]
	at org.apache.pinot.plugin.ingestion.batch.common.SegmentGenerationTaskRunner.run(SegmentGenerationTaskRunner.java:118) ~[classes/:?]
	at org.apache.pinot.plugin.minion.tasks.segmentgenerationandpush.SegmentGenerationAndPushTaskExecutor.generateAndPushSegment(SegmentGenerationAndPushTaskExecutor.java:132) ~[classes/:?]
	at org.apache.pinot.plugin.minion.tasks.segmentgenerationandpush.SegmentGenerationAndPushTaskExecutor.executeTask(SegmentGenerationAndPushTaskExecutor.java:118) ~[classes/:?]
	at org.apache.pinot.minion.taskfactory.TaskFactoryRegistry$1.runInternal(TaskFactoryRegistry.java:113) ~[classes/:?]
	at org.apache.pinot.minion.taskfactory.TaskFactoryRegistry$1.run(TaskFactoryRegistry.java:89) ~[classes/:?]
	at org.apache.helix.task.TaskRunner.run(TaskRunner.java:71) [helix-core-0.9.8.jar:0.9.8]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:264) [?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java) [?:?]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) [?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
	at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.fs.FileSystem
	at jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581) ~[?:?]

@codecov-commenter
Copy link

codecov-commenter commented Jul 8, 2022

Codecov Report

Merging #9033 (7ad4e74) into master (de16a0a) will decrease coverage by 1.61%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master    #9033      +/-   ##
============================================
- Coverage     70.08%   68.47%   -1.62%     
+ Complexity     4957     4734     -223     
============================================
  Files          1827     1827              
  Lines         96064    96097      +33     
  Branches      14356    14360       +4     
============================================
- Hits          67327    65800    -1527     
- Misses        24092    25722    +1630     
+ Partials       4645     4575      -70     
Flag Coverage Δ
integration1 ?
integration2 24.69% <ø> (+0.03%) ⬆️
unittests1 66.84% <ø> (-0.03%) ⬇️
unittests2 15.34% <ø> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...pinot/minion/exception/TaskCancelledException.java 0.00% <0.00%> (-100.00%) ⬇️
...nverttorawindex/ConvertToRawIndexTaskExecutor.java 0.00% <0.00%> (-100.00%) ⬇️
...e/pinot/common/minion/MergeRollupTaskMetadata.java 0.00% <0.00%> (-94.74%) ⬇️
...plugin/segmentuploader/SegmentUploaderDefault.java 0.00% <0.00%> (-87.10%) ⬇️
.../transform/function/MapValueTransformFunction.java 0.00% <0.00%> (-85.30%) ⬇️
...ot/common/messages/RoutingTableRebuildMessage.java 0.00% <0.00%> (-81.82%) ⬇️
...data/manager/realtime/DefaultSegmentCommitter.java 0.00% <0.00%> (-80.00%) ⬇️
...verttorawindex/ConvertToRawIndexTaskGenerator.java 5.45% <0.00%> (-80.00%) ⬇️
...ache/pinot/common/lineage/SegmentLineageUtils.java 22.22% <0.00%> (-77.78%) ⬇️
...ore/startree/executor/StarTreeGroupByExecutor.java 0.00% <0.00%> (-77.78%) ⬇️
... and 151 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update de16a0a...7ad4e74. Read the comment docs.

@xiangfu0 xiangfu0 closed this Jul 9, 2022
@xiangfu0 xiangfu0 deleted the fix-orc-in-quickstart branch July 9, 2022 07:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants