Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BigQuery: Valid google-resumable-media versions conflict with Cloud Storage #9786

Closed
daniel-humu opened this issue Nov 13, 2019 · 5 comments · Fixed by #9788
Closed

BigQuery: Valid google-resumable-media versions conflict with Cloud Storage #9786

daniel-humu opened this issue Nov 13, 2019 · 5 comments · Fixed by #9788
Assignees
@tseaver @tswast
Labels
api: bigquery Issues related to the BigQuery API. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@daniel-humu
Copy link

daniel-humu commented Nov 13, 2019 •
edited

It looks like #9572 introduced a dependency conflict for google-resumable-media between the latest google-cloud-storage and google-cloud-bigquery releases.

Environment details

  1. Python 3.7

Steps to reproduce

  1. pip install google-cloud-storage (latest release is 1.23.0)
    2.pip install google-cloud-bigquery (latest release is 1.21.0)
  2. Try to import both libraries in the same python

Issue

Today, google-cloud-storage 1.23.0 was released. It depends on google-resumable-media >= 0.5.0, < 0.6dev. The current google-cloud-bigquery version, depends on google-resumable-media >= 0.3.1, != 0.4.0, < 0.5.0dev

Therefore, the setup.py files for these two libraries will 2nd level dependency conflict when you attempt to pip install.

This can be fixed by releasing a new google-cloud-bigquery version. (to pick up the changes in #9572)
@joshmeek joshmeek mentioned this issue Nov 13, 2019
Merged
3 tasks
@prha prha mentioned this issue Nov 13, 2019
Closed
prha added a commit to dagster-io/dagster that referenced this issue Nov 13, 2019
@prha
pin google-cloud-storage until they fix deps
e42caa5 
Summary:
googleapis/google-cloud-python#9786

tracking github issue to remove once there is a new bigquery release: #1907

Test Plan: bk

Reviewers: #ft, alangenfeld, natekupp

Reviewed By: #ft, natekupp

Subscribers: alangenfeld, max

Differential Revision: https://dagster.phacility.com/D1414
@tseaver tseaver added api: bigquery Issues related to the BigQuery API. type: question Request for information or clarification. Not an issue. labels Nov 13, 2019
@tseaver
Copy link
Contributor

tseaver commented Nov 13, 2019

@tswast Are we at a good spot to release a new version of google-cloud-bigquery?

@tseaver tseaver added priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. and removed type: question Request for information or clarification. Not an issue. labels Nov 13, 2019
@tseaver
Copy link
Contributor

tseaver commented Nov 13, 2019

@daniel-humu Thanks for the report!

@tseaver tseaver mentioned this issue Nov 13, 2019
Merged
samanvp added a commit to samanvp/gcp-variant-transforms that referenced this issue Nov 14, 2019
@samanvp
Try to resolve a google-resumable-media package conflict
c8dc5a0 
Most likely this is casued by the conflict between google-cloud-storage
and google-cloud-bigquery (which is required by BEAM).

googleapis/google-cloud-python#9786
@samanvp samanvp mentioned this issue Nov 14, 2019
Closed
samanvp added a commit to samanvp/gcp-variant-transforms that referenced this issue Nov 14, 2019
@samanvp
Resolve google-resumable-media package conflict
d773708 
This is casued by the conflict between:
* google-cloud-storage 1.23.0
* and google-cloud-bigquery < 1.22.0

More details here:
googleapis/google-cloud-python#9786

Currently Beam is pinned to google-cloud-bigquery 1.17.0

https://github.com/apache/beam/blob/edf01c456d4ffbd17e5eee9a0dc2acb5693ed1f6/sdks/python/setup.py#L162

We have to pin to google-cloud-storage<1.23.0 until Beam updates to
google-cloud-bigquery<=1.22.0
samanvp added a commit to googlegenomics/gcp-variant-transforms that referenced this issue Nov 15, 2019
@samanvp
Resolve google-resumable-media package conflict (#529)
be516c7 
This is casued by the conflict between:
* google-cloud-storage 1.23.0
* and google-cloud-bigquery < 1.22.0

More details here:
googleapis/google-cloud-python#9786

Currently Beam is pinned to google-cloud-bigquery 1.17.0

https://github.com/apache/beam/blob/edf01c456d4ffbd17e5eee9a0dc2acb5693ed1f6/sdks/python/setup.py#L162

We have to pin to google-cloud-storage<1.23.0 until Beam updates to
google-cloud-bigquery<=1.22.0
@icymai
Copy link

icymai commented Dec 2, 2019

Hello Daniel! Thank you for providing the solution. I have a small question related to this error. What does it mean "google-resumable-media >= 0.3.1, != 0.4.0, < 0.5.0dev"? Why does this description mention 3 different versions?

@daniel-humu
Copy link
Author

@icymai Are you asking how version specifiers work? The dependency google-resumable-media >= 0.3.1, != 0.4.0, < 0.5.0dev means that the package depended on versions greater than or equal to 0.3.1, less than 0.5.0dev and excluding the 0.4.0 version.

You can read more documentation on version specification on the Python website.

@icymai
Copy link

icymai commented Dec 3, 2019

Cool! I haven't read this doc before. Thank you :D

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tseaver tseaver

@tswast tswast

Labels
api: bigquery Issues related to the BigQuery API. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

chore(bigquery): release 1.22.0
4 participants
@tseaver @tswast @icymai @daniel-humu