Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dependency conflict with slf4j #296

Closed
npetunin-clgx opened this issue Jan 14, 2021 · 1 comment
Closed

Dependency conflict with slf4j #296

npetunin-clgx opened this issue Jan 14, 2021 · 1 comment
Assignees

Comments

@npetunin-clgx
Copy link

When we are using this library, in Spark job, we see the following conflict:

Caused by: java.lang.LinkageError: loader constraint violation in interface itable initialization: when resolving method "com.google.cloud.spark.bigquery.BigQueryUtilScala$.org$apache$spark$internal$Logging$$log_()Lorg/slf4j/Logger;" the class loader (instance of org/apache/spark/util/ChildFirstURLClassLoader) of the current class, com/google/cloud/spark/bigquery/BigQueryUtilScala$, and the class loader (instance of sun/misc/Launcher$AppClassLoader) for interface org/apache/spark/internal/Logging have different Class objects for the type org/slf4j/Logger used in the signature 

We found that we have extra slf4j dependency in the bigquery connector dependency tree in Bigquery Connector version: 0.18.0. We tried to exclude slf4j dependency from arrow-vector, build it and try customized jar. And we didn't see error specified above.

We would like to exclude slf4j-api dependency from arrow-vector as follows:

"org.apache.arrow" % "arrow-vector" % "0.16.0" exclude("org.slf4j", "slf4j-api"), 

in Line 82 in build.sbt file.

@stsai-clgx
Copy link

hi @davidrabinowitz,
thank you for resolving this issue. When will next release become available?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants