-
Notifications
You must be signed in to change notification settings - Fork 190
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add option to enable useAvroLogicalTypes option in LoadJobConfiguration #276
Add option to enable useAvroLogicalTypes option in LoadJobConfiguration #276
Conversation
Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). 📝 Please visit https://cla.developers.google.com/ to sign. Once you've signed (or fixed any issues), please reply here with What to do if you already signed the CLAIndividual signers
Corporate signers
ℹ️ Googlers: Go here for more info. |
@googlebot I signed it! |
We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google. ℹ️ Googlers: Go here for more info. |
19115be
to
3f40943
Compare
@googlebot I fixed it. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the contribution! Can you please update connector/src/test/java/com/google/cloud/spark/bigquery/SparkBigQueryConfigTest.java
as well?
Done |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Notice this is Google truth assertThat()
connector/src/test/java/com/google/cloud/spark/bigquery/SparkBigQueryConfigTest.java
Outdated
Show resolved
Hide resolved
connector/src/test/java/com/google/cloud/spark/bigquery/SparkBigQueryConfigTest.java
Outdated
Show resolved
Hide resolved
/gcbrun |
In order to be able to load a BigQuery table partitioned on a DATE column using Avro, the option
useAvroLogicalTypes
has to be activated inLoadJobConfiguration
. Otherwise, BigQuery reads DATE columns as INTEGER as documented here: https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-avro#logical_types