NoClassDefFoundError: com/clickhouse/client/config/ClickHouseOption

[Aload]

It worked fine when I used Clickhouse-JDBC to run locally, but started reporting an error when the package went into the cluster. I also added the ClickHouse-JDBC JAR to the Spark jars directory.

env：
spark3.1.2
clickhouse-jdbc 0.3.2-patch8

[zhicwu]

Hi @Aload, it looks you did deploy all dependencies. Please use a shaded jar, the one with suffix like clickhouse-jdbc-0.3.2-patch8-http.jar or -all instead.

[Aload]

Hi @Aload, it looks you did deploy all dependencies. Please use a shaded jar, the one with suffix like clickhouse-jdbc-0.3.2-patch8-http.jar or -all instead.

So I don't think there's a bag? The same problem occurs when I use both. 0.3.2-Patch8, Clickhouse-HTTP-client

[zhicwu]

Sorry I meant you did NOT deploy all dependencies. In order to use shaded jar in maven, you need to specify classifier, for example:

<dependency>
    <!-- will stop using ru.yandex.clickhouse starting from 0.4.0  -->
    <groupId>com.clickhouse</groupId>
    <artifactId>clickhouse-jdbc</artifactId>
    <version>0.3.2-patch8</version>
    <!-- below is only needed when all you want is a shaded jar -->
    <classifier>http</classifier>
    <exclusions>
        <exclusion>
            <groupId>*</groupId>
            <artifactId>*</artifactId>
        </exclusion>
    </exclusions>
</dependency>

[Aload]

Sorry I meant you did NOT deploy all dependencies. In order to use shaded jar in maven, you need to specify classifier, for example:

<dependency>
    <!-- will stop using ru.yandex.clickhouse starting from 0.4.0  -->
    <groupId>com.clickhouse</groupId>
    <artifactId>clickhouse-jdbc</artifactId>
    <version>0.3.2-patch8</version>
    <!-- below is only needed when all you want is a shaded jar -->
    <classifier>http</classifier>
    <exclusions>
        <exclusion>
            <groupId>*</groupId>
            <artifactId>*</artifactId>
        </exclusion>
    </exclusions>
</dependency>

Sorry, I don't quite catch your meaning. I have also joined your POM. It doesn't really make a difference .

<dependency> <!-- will stop using ru.yandex.clickhouse starting from 0.4.0 --> <groupId>com.clickhouse</groupId> <artifactId>clickhouse-jdbc</artifactId> <version>0.3.2-patch8</version> <!-- below is only needed when all you want is a shaded jar --> <classifier>http</classifier> <exclusions> <exclusion> <groupId>*</groupId> <artifactId>*</artifactId> </exclusion> </exclusions> </dependency>

[zhilinli123]

@zhicwu @Aload May I ask how to solve the problem at last?

[Aload]

 <dependency>
            <groupId>com.clickhouse</groupId>
            <artifactId>clickhouse-jdbc</artifactId>
            <classifier>http</classifier>
            <version>0.3.2-patch7</version>
            <exclusions>
                <exclusion>
                    <groupId>*</groupId>
                    <artifactId>*</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

[gulldan]

I have the same problem in pyspark
java.lang.NoClassDefFoundError: com/clickhouse/client/config/ClickHouseOption

env :

1. sparkConf.set('spark.jars.packages',
   'com.github.housepower:clickhouse-spark-runtime-3.3_2.12:0.5.0,com.clickhouse:clickhouse-jdbc:0.4.0')
2. spark 3.3.1

[zhicwu]

com.clickhouse:clickhouse-jdbc:0.4.0

It looks like you missed the classifier all or http as shown in above.

[gulldan]

com.clickhouse:clickhouse-jdbc:0.4.0

It looks like you missed the classifier all or http as shown in above.

sorry im using pyspark and not familiar with java stack.
i dont understand how to use POM.

also i can find -all / -http in maven
https://repo1.maven.org/maven2/com/clickhouse/clickhouse-jdbc/0.4.0/clickhouse-jdbc-0.4.0-all.jar

but as i understand spark has spark.jars.ivySettings but ivy2 doesn't support classifier.

https://spark.apache.org/docs/latest/configuration.html

I tried to download and send it directly by adding

sparkConf.set('spark.jars', 'http://local-repo/bins/clickhouse-jdbc-0.4.0-all.jar')

but there is no changes

java.lang.NoClassDefFoundError: com/clickhouse/client/config/ClickHouseOption
If i try

$SPARK_HOME/bin/spark-sql \
  --conf spark.sql.catalog.clickhouse=xenon.clickhouse.ClickHouseCatalog \
  --conf spark.sql.catalog.clickhouse.host=127.0.0.1 \
  --conf spark.sql.catalog.clickhouse.protocol=http \
  --conf spark.sql.catalog.clickhouse.http_port=8123 \
  --conf spark.sql.catalog.clickhouse.user=default \
  --conf spark.sql.catalog.clickhouse.password= \
  --conf spark.sql.catalog.clickhouse.database=default \
  --packages com.github.housepower:clickhouse-spark-runtime-3.3_2.12:0.5.0,com.clickhouse:clickhouse-jdbc:0.4.0:all

i get
Exception in thread "main" java.lang.IllegalArgumentException: requirement failed: Provided Maven Coordinates must be in the form 'groupId:artifactId:version'. The coordinate provided is: com.clickhouse:clickhouse-jdbc:0.4.0:all

Could you give me some advices?

[TheHett]

Help for gradle (kotlin)

    // https://mvnrepository.com/artifact/com.clickhouse/clickhouse-jdbc
    implementation("com.clickhouse:clickhouse-jdbc:0.4.1:all")

[ivan-klass]

sbt:

("com.clickhouse" % "clickhouse-jdbc" % "0.4.1" classifier "all") % "runtime"