Skip to content

Cobrix crashes with exception when logging is not done via Log4j #816

@yruslan

Description

@yruslan

Describe the bug

Driver stacktrace:
	at org.apache.spark.scheduler.DAGScheduler.failJobAndIndependentStages(DAGScheduler.scala:2898)
	at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2(DAGScheduler.scala:2834)
	at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2$adapted(DAGScheduler.scala:2833)
	at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
	at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
	at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:2833)
	at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1(DAGScheduler.scala:1253)
	at 
<199 frames>
Caused by: java.lang.NoSuchMethodError: 'java.lang.ClassLoader org.apache.logging.log4j.util.StackLocatorUtil.getCallerClassLoader(int)'
	at org.apache.log4j.Logger.getLogger(Logger.java:35)
	at za.co.absa.cobrix.spark.cobol.source.streaming.FileStreamer.<init>(FileStreamer.scala:42)
	at za.co.absa.cobrix.spark.cobol.source.index.IndexBuilder$.getStreams(IndexBuilder.scala:249)
	at za.co.absa.cobrix.spark.cobol.source.index.IndexBuilder$.generateIndexEntry(IndexBuilder.scala:208)
	at za.co.absa.cobrix.spark.cobol.source.index.IndexBuilder$.$anonfun$buildIndexForVarLenReader$3(IndexBuilder.scala:148)
	at scala.collection.Iterator$$anon$11.nextCur(Iterator.scala:486)
	at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:492)
	at scala.collection.Iterator.foreach(Iterator.scala:943)

Expected behavior

Use SLF4J interface everywhere which would support more mechanism types.

Context

  • Cobrix version: 2.9.6
  • Spark version: 3.5.8
  • Scala version: 2.12

Copybook (if possible)

--

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions