Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add support for INTERVAL data type to list_rows #840

Merged
merged 39 commits into from Oct 26, 2021

Conversation

tswast
Copy link
Contributor

@tswast tswast commented Jul 29, 2021

Implementation of most other features for INTERVAL is blocked by backend changes.

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Towards #826 馃

@tswast tswast requested a review from a team July 29, 2021 22:12
@tswast tswast requested a review from a team as a code owner July 29, 2021 22:12
@tswast tswast requested review from stephaniewang526 and removed request for a team July 29, 2021 22:12
@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery API. label Jul 29, 2021
@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Jul 29, 2021
@tswast tswast added the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Jul 29, 2021
@tswast
Copy link
Contributor Author

tswast commented Jul 29, 2021

Some of these commits are from #829

@tswast tswast mentioned this pull request Jul 29, 2021
4 tasks
google/cloud/bigquery/_helpers.py Show resolved Hide resolved
minutes = time_sign * int(parsed.group("minutes"))
seconds = time_sign * int(parsed.group("seconds"))
fraction = parsed.group("fraction")
microseconds = time_sign * int(fraction.ljust(6, "0")) if fraction else 0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't tell from https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types#interval_type if there are limits on any of these fields. 馃し

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. The docs don't really explain much about how this data format actually works. From what is documented and trial and error, I think these are the limits:

Min

  • years: -10000
  • months: -11 (I think?)
  • days: -3660000
  • hours: -87840000
  • minutes: -59 (I think?)
  • seconds: -59 (I think?)
  • microseconds: -999999 (I think?)

Max

  • years: 10000
  • months: 11 (I think?)
  • days: 3660000
  • hours: 87840000
  • minutes: 59 (I think?)
  • seconds: 59 (I think?)
  • microseconds: 999999 (I think?)

I don't think we'll need any client-side validation for this, but it does remind me that we should have some unit tests that exercise these limits.

BOOL = ScalarQueryParameterType("BOOL")
BOOLEAN = ScalarQueryParameterType("BOOL")
BIGDECIMAL = ScalarQueryParameterType("BIGNUMERIC")
BIGNUMERIC = ScalarQueryParameterType("BIGNUMERIC")
BYTES = ScalarQueryParameterType("BYTES")
INTEGER = ScalarQueryParameterType("INT64")
INT64 = ScalarQueryParameterType("INT64")
DATE = ScalarQueryParameterType("DATE")
DATETIME = ScalarQueryParameterType("DATETIME")
DECIMAL = ScalarQueryParameterType("NUMERIC")
FLOAT = ScalarQueryParameterType("FLOAT64")
FLOAT64 = ScalarQueryParameterType("FLOAT64")
NUMERIC = ScalarQueryParameterType("NUMERIC")
BIGNUMERIC = ScalarQueryParameterType("BIGNUMERIC")
DECIMAL = ScalarQueryParameterType("NUMERIC")
BIGDECIMAL = ScalarQueryParameterType("BIGNUMERIC")
BOOLEAN = ScalarQueryParameterType("BOOL")
BOOL = ScalarQueryParameterType("BOOL")
GEOGRAPHY = ScalarQueryParameterType("GEOGRAPHY")
TIMESTAMP = ScalarQueryParameterType("TIMESTAMP")
DATE = ScalarQueryParameterType("DATE")
INT64 = ScalarQueryParameterType("INT64")
INTEGER = ScalarQueryParameterType("INT64")
NUMERIC = ScalarQueryParameterType("NUMERIC")
STRING = ScalarQueryParameterType("STRING")
TIME = ScalarQueryParameterType("TIME")
DATETIME = ScalarQueryParameterType("DATETIME")
TIMESTAMP = ScalarQueryParameterType("TIMESTAMP")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you intend to include INTERVAL?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did but then reverted it because the query parameter support is incomplete. I could revert the alphabetization, but I figure we should do that at some point.

google/cloud/bigquery/query.py Outdated Show resolved Hide resolved
google/cloud/bigquery/query.py Outdated Show resolved Hide resolved
@tswast
Copy link
Contributor Author

tswast commented Aug 5, 2021

I think this is ready, but I'll keep the Do Not Merge label, as I think #829 should go in first.

docs/conf.py Outdated Show resolved Hide resolved
@tswast tswast requested review from plamut and removed request for jimfulton September 30, 2021 18:38
@tswast tswast added the owlbot:run Add this label to trigger the Owlbot post processor. label Sep 30, 2021
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Sep 30, 2021
@tswast tswast added the owlbot:run Add this label to trigger the Owlbot post processor. label Oct 25, 2021
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Oct 25, 2021
@tswast
Copy link
Contributor Author

tswast commented Oct 25, 2021

@plamut Did you have any additional thoughts on this? It'd be good to get in before v3. I added some proposed unit tests with trailing ., but kept the regex for separating seconds from microseconds due to rounding errors discovered with the divmod approach.

Copy link
Contributor

@plamut plamut left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Went over this again, it looks fine (just one nit).

Good that we caught the issue in the divmod-based implementation, keeping the regex is the right way to go.

google/cloud/bigquery/_helpers.py Outdated Show resolved Hide resolved
@tswast tswast requested a review from plamut October 26, 2021 15:20
plamut
plamut approved these changes Oct 26, 2021
@tswast tswast added the automerge Merge the pull request once unit tests and other checks pass. label Oct 26, 2021
@tswast tswast merged commit e37380a into googleapis:main Oct 26, 2021
13 checks passed
@gcf-merge-on-green gcf-merge-on-green bot removed the automerge Merge the pull request once unit tests and other checks pass. label Oct 26, 2021
@tswast tswast deleted the issue826-interval branch October 26, 2021 16:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants