-
Notifications
You must be signed in to change notification settings - Fork 86
Closed
Labels
bugSomething isn't workingSomething isn't working
Description
Describe the bug
Driver stacktrace:
at org.apache.spark.scheduler.DAGScheduler.failJobAndIndependentStages(DAGScheduler.scala:2898)
at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2(DAGScheduler.scala:2834)
at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2$adapted(DAGScheduler.scala:2833)
at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:2833)
at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1(DAGScheduler.scala:1253)
at
<199 frames>
Caused by: java.lang.NoSuchMethodError: 'java.lang.ClassLoader org.apache.logging.log4j.util.StackLocatorUtil.getCallerClassLoader(int)'
at org.apache.log4j.Logger.getLogger(Logger.java:35)
at za.co.absa.cobrix.spark.cobol.source.streaming.FileStreamer.<init>(FileStreamer.scala:42)
at za.co.absa.cobrix.spark.cobol.source.index.IndexBuilder$.getStreams(IndexBuilder.scala:249)
at za.co.absa.cobrix.spark.cobol.source.index.IndexBuilder$.generateIndexEntry(IndexBuilder.scala:208)
at za.co.absa.cobrix.spark.cobol.source.index.IndexBuilder$.$anonfun$buildIndexForVarLenReader$3(IndexBuilder.scala:148)
at scala.collection.Iterator$$anon$11.nextCur(Iterator.scala:486)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:492)
at scala.collection.Iterator.foreach(Iterator.scala:943)
Expected behavior
Use SLF4J interface everywhere which would support more mechanism types.
Context
- Cobrix version: 2.9.6
- Spark version: 3.5.8
- Scala version: 2.12
Copybook (if possible)
--
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working