Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: unnest failed in some cases (with table references failed when there were no other references to refrenced tables in a query) #290

Merged
merged 11 commits into from Aug 25, 2021

Conversation

@jimfulton
Copy link
Contributor

@jimfulton jimfulton commented Aug 20, 2021

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #283 馃

@google-cla google-cla bot added the cla: yes label Aug 20, 2021
@jimfulton jimfulton marked this pull request as ready for review Aug 20, 2021
@jimfulton jimfulton requested a review from as a code owner Aug 20, 2021
@jimfulton
Copy link
Contributor Author

@jimfulton jimfulton commented Aug 23, 2021

Reviewer:

This PR is explained pretty well, I think, by the comment here:
https://github.com/googleapis/python-bigquery-sqlalchemy/pull/290/files#diff-efe1987a73ae5b54aa817dc8c70541025df89d9d08601c8ab33fa84ad8c09de7R254-R272

Mostly unrelated, I added the missing ARRAY type to the list of types, to facilitate a test. I took the opportunity to sort the types, which contributes a quite a few changed lines, but the change is trivial.

@jimfulton jimfulton changed the title fix: unnest with table references failed when there were no other references to refrenced tables fix: unnest failed in some cases (with table references failed when there were no other references to refrenced tables in a query) Aug 23, 2021
}

# By convention, dialect-provided types are spelled with all upper case.
STRING = _type_map["STRING"]
BOOL = _type_map["BOOL"]
ARRAY = _type_map["ARRAY"]
Copy link
Collaborator

@tswast tswast Aug 24, 2021

Does the new array type count as a feature? Perhaps it'd look better in the changelog as a separate PR?

If it is a feature and it doesn't make sense to split out, we could also try the multi-change commit message feature when we squash and merge this: https://github.com/googleapis/release-please#what-if-my-pr-contains-multiple-fixes-or-features

Copy link
Contributor Author

@jimfulton jimfulton Aug 24, 2021

It's not a new feature. It was just missing from the types list.

SQLAlchemy has basic array support that works with BigQuery (thanks to some previous effort on our part).

sqlalchemy_bigquery/base.py Outdated Show resolved Hide resolved
sqlalchemy_bigquery/base.py Show resolved Hide resolved
@@ -34,6 +34,7 @@
import datetime
import decimal

sqlalchemy_version_info = tuple(map(int, sqlalchemy.__version__.split(".")))
Copy link
Collaborator

@tswast tswast Aug 24, 2021

This works, though I've been trying to use https://packaging.pypa.io/en/latest/version.html which is the canonical version parser. We already have it pulled in via a transitive dependency through setuptools, I believe.

Copy link
Contributor Author

@jimfulton jimfulton Aug 24, 2021

Nice. I'll switch to that.

WRT how we get it, IMO, anything we import should be a direct dependency.

Copy link
Collaborator

@tswast tswast Aug 24, 2021

Sounds good. I'm fine explicitly including it in our dependencies / test dependencies.

Copy link
Contributor Author

@jimfulton jimfulton Aug 24, 2021

Done

@jimfulton
Copy link
Contributor Author

@jimfulton jimfulton commented Aug 25, 2021

@tswast are you cool with this? :)

@jimfulton
Copy link
Contributor Author

@jimfulton jimfulton commented Aug 25, 2021

I have to more small PRs coming that will build on this.

tswast
tswast approved these changes Aug 25, 2021
@jimfulton jimfulton merged commit 9b5b002 into googleapis:master Aug 25, 2021
9 checks passed
@jimfulton jimfulton deleted the fix-unnest-283 branch Aug 25, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

2 participants