Skip to content

[SPARK-56010][BUILD] Upgrade snowflake-jdbc to 4.0.2#54831

Closed
dongjoon-hyun wants to merge 1 commit intoapache:masterfrom
dongjoon-hyun:SPARK-56010
Closed

[SPARK-56010][BUILD] Upgrade snowflake-jdbc to 4.0.2#54831
dongjoon-hyun wants to merge 1 commit intoapache:masterfrom
dongjoon-hyun:SPARK-56010

Conversation

@dongjoon-hyun
Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

This PR aims to upgrade snowflake-jdbc to 4.0.2.

Why are the changes needed?

To bring the latest bug fixed version for testing:

Does this PR introduce any user-facing change?

No. This is a test-only dependency.

How was this patch tested?

Pass the CIs.

Was this patch authored or co-authored using generative AI tooling?

No.

@dongjoon-hyun dongjoon-hyun changed the title [SPARK-56010] Upgrade snowflake-jdbc to 4.0.2 [SPARK-56010][BUILD] Upgrade snowflake-jdbc to 4.0.2 Mar 16, 2026
@dongjoon-hyun
Copy link
Copy Markdown
Member Author

Could you review this Snowflake JDBC Driver CVE patch, @huaxingao ?

$ build/sbt "sql/testOnly *.JDBCSuite"
...
[info] JDBCSuite:
[info] - SELECT * (1 second, 264 milliseconds)
[info] - SELECT * WHERE (simple predicates) (845 milliseconds)
[info] - SELECT COUNT(1) WHERE (predicates) (237 milliseconds)
[info] - SELECT * WHERE (quoted strings) (43 milliseconds)
[info] - SELECT first field (35 milliseconds)
[info] - SELECT first field when fetchsize is two (23 milliseconds)
[info] - SELECT second field (36 milliseconds)
[info] - SELECT second field when fetchsize is two (19 milliseconds)
[info] - SELECT * partitioned (28 milliseconds)
[info] - SELECT WHERE (simple predicates) partitioned (72 milliseconds)
[info] - SELECT second field partitioned (24 milliseconds)
[info] - SPARK-34843: columnPartition should generate the correct stride size and also realign the first partition for better distribution (21 milliseconds)
[info] - overflow of partition bound difference does not give negative stride (23 milliseconds)
[info] - Register JDBC query with renamed fields (10 milliseconds)
[info] - Basic API (30 milliseconds)
[info] - Missing partition columns (47 milliseconds)
[info] - Basic API with FetchSize (101 milliseconds)
[info] - Partitioning via JDBCPartitioningInfo API (22 milliseconds)
[info] - Partitioning via list-of-where-clauses API (18 milliseconds)
[info] - Partitioning on column that might have null values. (84 milliseconds)
[info] - Partitioning on column where numPartitions is zero (40 milliseconds)
[info] - Partitioning on column where numPartitions are more than the number of total rows (42 milliseconds)
[info] - Partitioning on column where lowerBound is equal to upperBound (39 milliseconds)
[info] - Partitioning on column where lowerBound is larger than upperBound (5 milliseconds)
[info] - SELECT * on partitioned table with a nullable partition column (21 milliseconds)
[info] - H2 integral types (37 milliseconds)
[info] - H2 null entries (19 milliseconds)
[info] - H2 string types (42 milliseconds)
[info] - H2 time types (69 milliseconds)
[info] - SPARK-34357: test TIME types (107 milliseconds)
[info] - SPARK-47396: TIME WITHOUT TIME ZONE preferTimestampNTZ (43 milliseconds)
[info] - test DATE types (53 milliseconds)
[info] - test DATE types in cache (63 milliseconds)
[info] - test types for null value (33 milliseconds)
[info] - H2 floating-point types (80 milliseconds)
[info] - SQL query as table name (39 milliseconds)
[info] - Remap types via JdbcDialects (32 milliseconds)
[info] - Map TINYINT to ByteType via JdbcDialects (30 milliseconds)
[info] - Default jdbc dialect registration (1 millisecond)
[info] - quote column names by jdbc dialect (1 millisecond)
[info] - compile filters (66 milliseconds)
[info] - Dialect unregister (1 millisecond)
[info] - Aggregated dialects (1 millisecond)
[info] - Aggregated dialects: isCascadingTruncateTable (0 milliseconds)
[info] - DB2Dialect type mapping (1 millisecond)
[info] - MySQLDialect catalyst type mapping (2 milliseconds)
[info] - SPARK-35446: MySQLDialect type mapping of float (1 millisecond)
[info] - PostgresDialect type mapping (1 millisecond)
[info] - DerbyDialect jdbc type mapping (0 milliseconds)
[info] - SPARK-39604: DerbyDialect catalyst type mapping (1 millisecond)
[info] - OracleDialect jdbc type mapping (0 milliseconds)
[info] - SPARK-42469: OracleDialect Limit query test (1 millisecond)
[info] - MsSqlServerDialect jdbc type mapping (1 millisecond)
[info] - SPARK-28152: MsSqlServerDialect catalyst type mapping (1 millisecond)
[info] - SPARK-42469: MsSqlServerDialect Limit query test (1 millisecond)
[info] - SPARK-42534: DB2Dialect Limit query test !!! IGNORED !!!
[info] - table exists query by jdbc dialect (0 milliseconds)
[info] - truncate table query by jdbc dialect (0 milliseconds)
[info] - SPARK-22880: Truncate table with CASCADE by jdbc dialect (1 millisecond)
[info] - Test DataFrame.where for Date and Timestamp (22 milliseconds)
[info] - SPARK-33867: Test DataFrame.where for LocalDate and Instant (30 milliseconds)
[info] - test credentials in the properties are not in plan output (23 milliseconds)
[info] - test credentials in the connection url are not in the plan output (11 milliseconds)
[info] - hide credentials in create and describe a persistent/temp table (116 milliseconds)
[info] - Hide credentials in show create table (51 milliseconds)
[info] - Replace CatalogUtils.maskCredentials with SQLConf.get.redactOptions (102 milliseconds)
[info] - SPARK 12941: The data type mapping for StringType to Oracle (1 millisecond)
[info] - SPARK-16625: General data types to be mapped to Oracle (0 milliseconds)
[info] - SPARK-15916: JDBC filter operator push down should respect operator precedence (66 milliseconds)
[info] - SPARK-16387: Reserved SQL words are not escaped by JDBC writer (11 milliseconds)
[info] - SPARK-18141: Predicates on quoted column names in the jdbc data source (313 milliseconds)
[info] - SPARK-18419: Fix `asConnectionProperties` to filter case-insensitively (6 milliseconds)
[info] - SPARK-16848: jdbc API throws an exception for user specified schema (3 milliseconds)
[info] - jdbc API support custom schema (37 milliseconds)
[info] - jdbc API custom schema DDL-like strings. (37 milliseconds)
[info] - SPARK-48399: TeradataDialect jdbc data mapping (0 milliseconds)
[info] - SPARK-38846: TeradataDialect catalyst type mapping (0 milliseconds)
[info] - Checking metrics correctness with JDBC (48 milliseconds)
[info] - unsupported types (5 milliseconds)
[info] - SPARK-47394: Convert TIMESTAMP WITH TIME ZONE to TimestampType (136 milliseconds)
[info] - SPARK-19318: Connection properties keys should be case-sensitive. (2 milliseconds)
[info] - SPARK-19318: jdbc data source options should be treated case-insensitive. (71 milliseconds)
[info] - SPARK-23856 Spark jdbc setQueryTimeout option (1 second, 83 milliseconds)
[info] - SPARK-24327 verify and normalize a partition column based on a JDBC resolved schema (19 milliseconds)
[info] - query JDBC option - negative tests (17 milliseconds)
[info] - query JDBC option (69 milliseconds)
[info] - SPARK-22814 support date/timestamp types in partitionColumn (137 milliseconds)
[info] - throws an exception for unsupported partition column types (3 milliseconds)
[info] - SPARK-24288: Enable preventing predicate pushdown (83 milliseconds)
[info] - SPARK-36574: pushDownPredicate=false should prevent push down filters to JDBC data source (91 milliseconds)
[info] - SPARK-26383 throw FAILED_JDBC.CONNECTION if wrong kind of driver to the given url (4 milliseconds)
[info] - support casting patterns for lower/upper bounds of TimestampType (65 milliseconds)
[info] - Add exception when isolationLevel is Illegal (2 milliseconds)
[info] - SPARK-28552: Case-insensitive database URLs in JdbcDialect (0 milliseconds)
[info] - SQLContext.jdbc (deprecated) (92 milliseconds)
[info] - SPARK-32364: JDBCOption constructor (0 milliseconds)
[info] - SPARK-34379: Map JDBC RowID to StringType rather than LongType (802 milliseconds)
[info] - SPARK-39339: Handle TimestampNTZType null values (426 milliseconds)
[info] - SPARK-39339: TimestampNTZType with different local time zones (3 seconds, 160 milliseconds)
[info] - SPARK-41990: Filter with composite name (35 milliseconds)
[info] - SPARK-44866: SnowflakeDialect BOOLEAN type mapping (0 milliseconds)
[info] - SPARK-45139: DatabricksDialect url handling (1 millisecond)
[info] - SPARK-45139: DatabricksDialect catalyst type mapping (0 milliseconds)
[info] - SPARK-45139: DatabricksDialect JDBC type mapping (0 milliseconds)
[info] - SPARK-45425: Mapped TINYINT to ShortType for MsSqlServerDialect (0 milliseconds)
[info] - SPARK-45475: saving a table via JDBC should work with observe API (110 milliseconds)
[info] - SPARK-47496: ServiceLoader support for JDBC dialects (0 milliseconds)
[info] - SPARK-47882: createTableColumnTypes need to be mapped to database types (5 milliseconds)
[info] - SPARK-50666: reading hint test (16 milliseconds)
[info] - FAILED_JDBC.CONNECTION (381 milliseconds)
[info] Run completed in 15 seconds, 303 milliseconds.
[info] Total number of tests run: 111
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 111, failed 0, canceled 0, ignored 1, pending 0
[info] All tests passed.
[success] Total time: 200 s (0:03:20.0), completed Mar 16, 2026, 1:11:43 PM

Copy link
Copy Markdown
Contributor

@huaxingao huaxingao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dongjoon-hyun
Copy link
Copy Markdown
Member Author

Thank you, @huaxingao ! 😄

@dongjoon-hyun
Copy link
Copy Markdown
Member Author

Merged to master for Apache Spark 4.2.0.

@dongjoon-hyun dongjoon-hyun deleted the SPARK-56010 branch March 16, 2026 23:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants