v2.1.2

This release contains a few improvements and bug fixes for Snowpipe Streaming:

[Bug Fix] Flaky tests are improved
[Bug Fix] Fixed an issue with failover across deployments
[Improvement] InsertRows Performance improvements
[Improvement] Added or Improved various logs for better observability
[Improvement] Fine tune channel and chunk sizes

What's Changed

• Add error log for flush task scheduling by @sfc-gh-alhuang in #760
• SNOW-1435061 Upgrade JDBC driver for domains fix by @sfc-gh-lsembera in #766
• no-snow Parallelize e2e jar tests by @sfc-gh-lsembera in #767
• SNOW-1357377 Add request Id in all streaming ingest APIs by @sfc-gh-japatel in #759
• no-snow Fix flaky test by @sfc-gh-lsembera in #772
• SNOW-1457523: Fix CVE for snowflake-ingest-java io.airlift:aircompressor 0.21 by @sfc-gh-tzhang in #774
• Various performance improvements in the path by @sfc-gh-psaha in #782
• SNOW-1457523: Upgrade Parquet dependencies version to fix CVE by @sfc-gh-tzhang in #779
• SNOW-1465503 Check row count in Parquet footer before committing by @sfc-gh-lsembera in #784
• SNOW-1512935 Retry SocketTimeoutException by @sfc-gh-lsembera in #789
• NO-SNOW Fix flaky test using multiple threads by @sfc-gh-lsembera in #790
• SNOW-1373151: Proactively refresh token to avoid token expiration exception from JDBC by @sfc-gh-tzhang in #780
• Reject new stage metadata if the deployment id does not match what the client was created with by @sfc-gh-psaha in #794
• SNOW-1512935 Reduce client socket timeout from 5 minutes to 1 minute by @sfc-gh-lsembera in #795
• SNOW-1545879 Reduce the max channel/chunk sizes by @sfc-gh-lsembera in #796
• NO-SNOW Upgrade protobuf package by @sfc-gh-xhuang in #793
• V2.1.2 Release by @sfc-gh-asen in #797

v2.1.1

What's Changed

• NO-SNOW updated version by @sfc-gh-dshah in #686
• SNOW-1165817 SNOW-1165815 Update vulnerable dependencies by @sfc-gh-lsembera in #693
• SNOW-1230527 Disable linkage checker by @sfc-gh-lsembera in #717
• SNOW-1196967 SNOW-1196966 Update vulnerable dependencies by @sfc-gh-lsembera in #720
• SNOW-1255447: Fix leaked connection due to error response by @sfc-gh-tzhang in #719
• SNOW-1000828 Expose specific error messages as part of the channel invalidation exception by @sfc-gh-alhuang in #721
• SNOW-993600 External OAuth2.0 Support by @sfc-gh-alhuang in #718
• NO-SNOW Reduce OAuth test time by @sfc-gh-alhuang in #732
• SNOW-1258064 Remove/Relax the limitations in order to generate bigger files by @sfc-gh-alhuang in #730
• [Snyk] SNOW-1327598: Fix for 3 vulnerabilities by @sfc-gh-snowflakedb-snyk-sa in #734
• SNOW-1167007: Validate no null values for not null cols by @sfc-gh-wtrefon in #739
• [Snyk] Security upgrade org.bouncycastle:bcpkix-jdk18on from 1.78 to 1.78.1 by @sfc-gh-snowflakedb-snyk-sa in #752
• SNOW-1361785 Run e2e jar tests with multiple Java versions by @sfc-gh-lsembera in #756
• No-SNOW Update fasterxml by @sfc-gh-xhuang in #740
• SNOW-1262359 Upgrade commons-configuration2 by @sfc-gh-lsembera in #761
• V2.1.1 Release by @sfc-gh-tzhang in #758

v2.1.0

What's Changed

• NO-SNOW Correct invalid use of javadoc by @sfc-gh-xhuang in #668
• SNOW-983635 Allow ZSTD compression algorithm by @sfc-gh-lthiede in #654
• SNOW-1032874: Fix an Overflow Issue for Integer-store Timestamp by @sfc-gh-tzhang in #673
• Snowpipe Streaming: send column ordinal back to GS with blob EPs to register by @sfc-gh-azagrebin in #674
• SNOW-979849: Add start offset token at batch level in the Client SDK by @sfc-gh-tzhang in #676
• SNOW-967235 Sends start and end offsets from the Client SDK by @sfc-gh-tjones in #675
• NO-SNOW: Two minor changes that address PR comments and fix code format by @sfc-gh-tzhang in #679
• SNOW-987122 Upgrade JDBC to 3.14.5 and Catch new exception type for r… by @sfc-gh-japatel in #677
• Allow clients to drop channel on close or blindly by @sfc-gh-psaha in #657
• Add more detailed javadoc for the drop related api to warn about the … by @sfc-gh-psaha in #681
• SNOW-979849: Support user to pass a user defined offset token verification logic as part of channel creation by @sfc-gh-tzhang in #680
• V2.1.0 Release by @sfc-gh-tzhang in #682

New Contributors

• @sfc-gh-psaha made their first contribution in #657

Full Changelog: v2.0.5...v2.1.0

v2.0.5

This release contains a few improvements and bug fixes for Snowpipe Streaming:

[Improvement] Added an optional offset token parameter for openChannel.
[Improvement] Added support for specifying compression algorithm to be used for BDEC Parquet files.
[Improvement] Updated to support customized URL and added Snowflake account name in request header.
[Improvement] Implemented a change to send spansMixedTables flag in blob registration requests.
[Improvement] Deprecated BUFFER_FLUSH_INTERVAL_IN_MILLIS parameter, instead use the MAX_CLIENT_LAG parameter.
[Improvement] Implemented the refresh of downscoped GCS tokens.
[Bug Fix] Reverted one change that updated public API for internal use case.
[Bug Fix] Fixed the end-to-end JAR test so it can run on all cloud platforms.

What's Changed

• Add an optional offset token parameter for openChannel by @sfc-gh-asen in #645
• support specifying compression algorithm to be used for BDEC Parquet files by @sfc-gh-gdoci in #579
• Disable Snowpipe flaky tests by @sfc-gh-tzhang in #647
• changed snapshot version from 2.0.4 to 2.0.5-SNAPSHOT by @sfc-gh-dshah in #650
• benchmark setup by @sfc-gh-lthiede in #655
• Update to support customized url and add customer name in request header by @sfc-gh-dwang in #656
• send spansMixedTables flag in blob registration requests by @sfc-gh-gdoci in #651
• Fix name of database the dataset is created in by @sfc-gh-lthiede in #658
• NO-SNOW: Deprecate BUFFER_FLUSH_INTERVAL_IN_MILLIS parameter by @sfc-gh-tzhang in #659
• NO-SNOW Run e2e jar tests on all clouds by @sfc-gh-lsembera in #666
• NO-SNOW: Revert one change that updates public API for internal use case by @sfc-gh-tzhang in #662
• GCS token refresh by @sfc-gh-lsembera in #665
• V2.0.5 release by @sfc-gh-tzhang in #664

Full Changelog: v2.0.4...v2.0.5

v2.0.4

This release contains a few improvements and bug fixes for Snowpipe Streaming:

• [Improvement] Support a new ON_ERROR option SKIP_BATCH, which will skip the entire batch if there is any issue and return all errors as part of the response
• [Improvement] Add row index information to all exceptions
• [Improvement] Upgrade snappy-java dependency
• [Improvement] Add a new interface to return the table schema information for a channel
• [Improvement] Add a new configuration option MAX_CLIENT_LAG which can contain the flush frequency, by default we flush every second
• [Bug Fix] Fix an issue when using snowflake-jdbc-fips
• [Bug Fix] Fix a rare ConcurrentModificationException issue
• [Bug Fix] Fix two issues in insertRows API that might cause wrong results in a very rare case
• [Bug Fix] Limit the max allowed number of chunks in blob to avoid the case when the request is too large

What's Changed

• Remove Public Preview note by @sfc-gh-lsembera in #582
• @no-snow refactor a few variables into ClientBufferParameters by @sfc-gh-gdoci in #581
• SNOW-914666 Adds MAX_CLIENT_LAG configuration option by @sfc-gh-tjones in #586
• SNOW-919423 Surfaces Table Schema for a Channel by @sfc-gh-tjones in #589
• NO-SNOW Remove wrong documentation by @sfc-gh-lthiede in #591
• updated SDK version to 2.0.4-SNAPSHOT by @sfc-gh-dshah in #588
• SNOW-924864 Upgrade snappy-java by @sfc-gh-lsembera in #592
• SNOW-918949: Add row index information to all exceptions by @sfc-gh-tzhang in #590
• SNOW-928802 Use concurrent hashmap to prevent ConcurrentModificationE… by @sfc-gh-japatel in #594
• SNOW-926149 Fix issues while using snowflake-jdbc-fips by @sfc-gh-lsembera in #596
• Update readme for JDBC version issues #599 by @sfc-gh-xhuang in #600
• SNOW-906224: Add new ON_ERROR=SKIP_BATCH option by @sfc-gh-tzhang in #597
• NO-SNOW: Fix two issues in insertRows API by @sfc-gh-tzhang in #602
• SNOW-902709 Limit the max allowed number of chunks in blob by @sfc-gh-lsembera in #580
• PRODSEC-3611 fix GHA parsing by @sfc-gh-jfan in #603
• no-snow Sort pom.xml in format.sh by @sfc-gh-lsembera in #604
• SNOW-936900 Use upgraded version of snappy-java by @sfc-gh-lsembera in #605
• SNOW-945927 SNOW-945928 Fix Snyk vulnerabilities by @sfc-gh-lsembera in #641
• NO-SNOW: fix row index issue for ON_ERROR=SKIP_BATCH by @sfc-gh-tzhang in #606
• V2.0.4 release by @sfc-gh-tzhang in #643

New Contributors

• @sfc-gh-lthiede made their first contribution in #591

Full Changelog: v2.0.3...v2.0.4

v2.0.3

This release contains a few improvements and bug fixes for Snowpipe Streaming:

• [Improvement] Support OAuth Authentication
• [Improvement] Remove exactly once related code for Snowpipe
• [Improvement] Support publishing unshaded snapshot release to Nexus repo
• [Improvement] Add retry logic for invalid JWT token
• [Improvement] Add warning for large batches in insertRows
• [Bug Fix] Fix a NPE issue caused by race condition

What's Changed

• SNOW-352846 OAuth Authentication: #1 Polymorph auth credential by @sfc-gh-alhuang in #532
• SNOW-693813 disable client info snowpipe tests, remove later by @sfc-gh-japatel in #536
• [SNOW-693813] Remove exactly once methods calling into server by @sfc-gh-rcheng in #539
• SNOW-855191 Race condition causing NPE by @sfc-gh-lsembera in #544
• SNOW-352846 OAuth Authentication: #2 OAuth Support by @sfc-gh-alhuang in #537
• SNOW-870386 Add default scheme parameters for test utils when profile is not specified by @sfc-gh-alhuang in #552
• SNOW-352846 OAuth Authentication: #3 Add error message by @sfc-gh-alhuang in #553
• NO-SNOW: Support publishing unshaded snapshot release to Nexus repo by @sfc-gh-tzhang in #561
• Update AWS test profile by @sfc-gh-alhuang in #564
• changed version from 2.0.2 to 2.0.3-SNAPSHOT by @sfc-gh-dshah in #565
• SNOW-877039 Ingest SDK retry for INVALID JWT Response by @sfc-gh-alhuang in #562
• Move testJWTRetries test to IT by @sfc-gh-alhuang in #572
• SNOW-861190 Add warning for large batches in insertRows by @sfc-gh-lsembera in #551
• Using jenkins sonatype username and password instead of my own by @sfc-gh-dshah in #571
• NO-SNOW: Add a few requested changes by @sfc-gh-tzhang in #576
• V2.0.3 release by @sfc-gh-tzhang in #577

New Contributors

• @sfc-gh-alhuang made their first contribution in #532

Full Changelog: v2.0.2...v2.0.3

v2.0.2

This release contains a few improvements for Snowpipe Streaming:

• [Improvement] Update dependencies based on Wiz and Snyk vulnerability scan result
• [Improvement] Improve retry logic on exceptions like SSLException
• [Improvement] Make the role as an optional input and support using of default role associated with the user
• [Improvement] Send uncompressed chunk length to server side for tracking purpose

What's Changed

• Remove CLA bot and update README.md to include cla and formatter requirements by @sfc-gh-rcheng in #510
• SNOW-841883 Upgrade snappy-java to 1.1.10.1 by @sfc-gh-lsembera in #533
• [Snyk] Security upgrade com.google.guava:guava from 31.1-jre to 32.0.0-jre by @sfc-gh-snyk-sca-sa in #534
• NO-SNOW Bump version to 2.0.2-SNAPSHOT by @sfc-gh-lsembera in #529
• Use later version of google guava, as recommended by CVE by @sfc-gh-japatel in #535
• [SNOW-843760] Upgrade netty-common in response to CVE vulnerability by @sfc-gh-rcheng in #538
• NO-SNOW Add note about dependencies by @sfc-gh-lsembera in #540
• SNOW-851351 Retry javax.net.ssl.SSLException by @sfc-gh-lsembera in #543
• [SNOW-843760] Update pom.xml corresponding to Wiz vulnerability scan by @sfc-gh-rcheng in #546
• SNOW-532834 Using default role when snowflake.role.name is not specified by @sfc-gh-alhuang in #507
• NO-SNOW Retry failed requests more aggressively by @sfc-gh-lsembera in #550
• v2.0.2 version upgrade by @sfc-gh-tzhang in #557
• @snow SNOW-835618 Snowpipe Streaming: send uncompressed chunk length from SDK to GS by @sfc-gh-azagrebin in #521
• reduce number of rows in BigFilesIT by @sfc-gh-gdoci in #559

Full Changelog: v2.0.1...v2.0.2

v2.0.1

This is a patch release on top of the v2.0.0 where we fix an unexpected dependency behavior for Snowflake JDBC

What's Changed

• @snow SNOW-837436 Streaming ingest: dedup var for support status code messages by @sfc-gh-azagrebin in #526
• NO-SNOW Specify dependencies in public_pom.xml by @sfc-gh-lsembera in #527
• V2.0.1 release by @sfc-gh-tzhang in #528

Full Changelog: v2.0.0...v2.0.1

v2.0.0

This release contains a few bug fixes and improvements for Snowpipe Streaming:

• [Improvement] Improving the dependencies shading and relocating logic, it introduces some behavior changes (e.g. JDBC) and that's why we bump the major version
• [Improvement] Make a few channel/chunk/file size limit parameters configurable
• [Improvement] Add more telemetries to track end2end latency
• [Improvement] Support GCS downscoped token
• [Improvement] Clean up all Arrow related code
• [Improvement] Add Attribution notice
• [Improvement] Enforce allowed DATE and TIMESTAMP range
• [Improvement] Expose more server side channel invalidation error message for customers to self-mitigate
• [Bug Fix] Fix an issue where some BG threads are not stopped during exception

What's Changed

• Adding a script to release unshaded version into S3 repo by @sfc-gh-hqin in #346
• Update protobuf version to 3.19.6 by @sfc-gh-dwang in #488
• SNOW-807102 Disable OpenManyChannelsIT by @sfc-gh-lsembera in #491
• SNOW-801398 Tidy up shaded jar by @sfc-gh-lsembera in #490
• SNOW-806047 fix logged uncompressedSize for Parquet by @sfc-gh-gdoci in #489
• SNOW-811371 Shade and Relocate Snowflake JDBC Classes by @sfc-gh-wfateem in #494
• NO-SNOW: Update MAX_CHUNK_SIZE_IN_BYTES default to be 32MB and make it configurable by @sfc-gh-tzhang in #498
• SNOW-758492: avoid combining channels with different schema by @sfc-gh-tzhang in #501
• Dshah snow 801462 snapshotdeploy by @sfc-gh-dshah in #496
• Revert "SNOW-811371 Shade and Relocate Snowflake JDBC Classes (#494)" by @sfc-gh-lsembera in #503
• Fix 'Connection reset' build errors by caching mvn dependencies by @sfc-gh-lsembera in #502
• [SNOW-766867] Add blob latency and start timestamps to registerBlobs call by @sfc-gh-rcheng in #429
• SNOW-768197 Attribution notice by @sfc-gh-lsembera in #506
• @snow SNOW-763199 Enable StreamingIngestIT.testTableColumnEvolution by @sfc-gh-azagrebin in #505
• NO-SNOW minor error text typo fix by @sfc-gh-xhuang in #508
• SNOW-768195: [Client Side] Support GCS downscoped token for Snowpipe Streaming by @sfc-gh-tzhang in #504
• SNOW-818891 Remove Arrow and other unneeded dependencies by @sfc-gh-lsembera in #497
• SNOW-747848: Cleanup BG threads with exceptions by @sfc-gh-tzhang in #479
• SNOW-830955 Use default SDK version in User-Agent by @sfc-gh-lsembera in #513
• NO-SNOW Exclude examples from code coverage scanning by @sfc-gh-lsembera in #515
• SNOW-830862 Maven build improvements by @sfc-gh-lsembera in #514
• SNOW-768197 Attribution notice for shaded dependencies by @sfc-gh-lsembera in #517
• NO-SNOW: Limit the chunk size to avoid potential OOM issue in the Parquet Reader by @sfc-gh-tzhang in #516
• NO-SNOW Enforce allowed DATE and TIMESTAMP range by @sfc-gh-lsembera in #520
• SNOW-720420 Enforce per-row size limits by @sfc-gh-lsembera in #519
• NO-SNOW Mention minimal JDBC driver version 3.13.30 by @sfc-gh-lsembera in #523
• V2.0.0 release by @sfc-gh-tzhang in #522
• NO-SNOW Exclude additional example from code coverage by @sfc-gh-xhuang in #524
• @snow SNOW-837436 Streaming ingest: get status code message in channel status response by @sfc-gh-azagrebin in #525

New Contributors

• @sfc-gh-hqin made their first contribution in #346
• @sfc-gh-dshah made their first contribution in #496
• @sfc-gh-xhuang made their first contribution in #508

Full Changelog: v1.1.3...v2.0.0

1.1.4

What's Changed

• NO-SNOW Upgrade JDBC to 3.13.30 by @sfc-gh-japatel in #500

Full Changelog: v1.1.3...v1.1.4