0.23.2

New Features

• PR #521: Added Arrow compression options to the spark-bigquery-with-dependencies_2.* connectors
• PR #526: Added the option to use parent project for the metadata/jobs API as well

Dependency Updtyes

• BigQuery API has been upgraded to version 2.3.3
• BigQuery Storage API has been upgraded to version 2.4.2
• gRPC has been upgraded to version 1.42.1
• Netty has been upgraded to version 4.1.70.Final

0.23.1

Bug Fixes

• Issue #501: Fixed using Apache Avro as an intermediate type for writing.

0.23.0

New Features

• New connector: This release adds a new connector (spark-2.4-bigquery) which have a better integration with the new Spark 2.4 internal APIs. This connector is implemented in Java, so it works for both Scala versions. This connector is released in preview version, comments are welcome.
• PR #469: Added support for the BigQuery Storage Write API, allowing faster writes (Spark 2.4 connector only). Activated by setting the option writeMethod to direct, this should give better write performance.
• Issue #481: Added configuration option to use compression from the READ API for Arrow

Dependency Updates

• BigQuery API has been upgraded to version 2.1.8
• BigQuery Storage API has been upgraded to version 2.1.2
• gRPC has been upgraded to version 1.41.0

0.22.2

Bug Fixes

• Issue #446: BigNumeric values are properly written to BigQuery
• Issue #452: Adding the option to clean BigQueryClient.destinationTableCache

Dependency Upgrades

• BigQuery API has been upgraded to version 2.1.8
• BigQuery Storage API has been upgraded to version 2.1.2
• gRPC has been upgraded to version 1.4.0

0.22.1

Bug Fixes

• Issue #444: allowing unpartitioned clustered table

0.22.0

New Features

• PR #404: Added support for BigNumeric
• PR #430: Added HTTP and gRPC proxy support

Bug Fixes

• Issue #273: Resolved the streaming write issue for spark 3.x

0.21.1

New Features

• PR #413: Pushing all filters to BigQuery Storage API

Bug Fixes

• Issue #412: Supporting WITH queries
• Issue #409: Allowing all whitespaces after the select
• PR #419: Fix a bug where background threads > 2 cases would miss pages (DSv2)
• PR #416: Moved zstd-jni library to be provided in order to solve Spark 2.4 compatibility (DSv2)
• PR #417: Added back column projection to DSv2

0.21.0

Bug Fixes

• Issue #354: users can query a view with different columns in select() and filter()
• Issue #367: Struct column order is fixed
• Issue #383: Fixed table metadata update when writing to a partitioned table
• Issue #390: Allowing additional white-space types in the query
• Issue #393: replacing avro.shaded dependency with guava
• PR #360: Removed redundant UNNEST when compiling IN condition

Dependency Updates

• BigQuery API has been upgraded to version 1.131.1
• BigQuery Storage API has been upgraded to version 1.22.0
• Guava has been upgraded to version 30.1.1-jre
• gRPC has been upgraded to version 1.37.1
• Netty has been upgraded to version 4.1.65.Final

0.20.0

New Features

• PR #375: Added support for pseudo column support - time partitioned table now support the _PARTITIONTIME and _PARTITIONDATE fields

Bug Fixes

• Issue #190: Writing data to BigQuery properly populate the field description
• Issue #265: Fixed nested conjunctions/disjunctions when using the AVRO read format
• Issue #326: Fixing netty_tcnative_windows.dll shading

Dependency Updates

• Arrow has een upgraded to version 4.0.0

0.19.1

Bug Fixes

• PR #324 - Restoring version 0.18.1 dependencies due to networking issues

Dependency Updates

• BigQuery API has been upgraded to version 1.123.2
• BigQuery Storage API has been upgraded to version 1.6.0
• Guava has been upgraded to version 30.0-jre
• Netty has been upgraded to version 4.1.51.Final