Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix argument ordering and type of bucket and object #15738

Merged
merged 1 commit into from May 9, 2021

Conversation

sjvanrossum
Copy link
Contributor

This fixes the argument ordering of bucket_name and object_name in GCSHook.download.
All other methods defined on GCSHook have bucket_name before object_name except download.
GoogleCloudStorageHook.download in Airflow 1.0 also defined bucket_name first instead of object_name.

Using object_name, bucket_name causes an issue in BigQueryCreateExternalTableOperator and mlengine_operator_utils.create_evaluate_ops where the parameters to GCSHook.download in those operators are passed positionally.

Additionally, bucket_name is not an optional parameter in the underlying API and is not represented as such in other methods of GCSHook.

Found this while debugging a workflow using BigQueryCreateExternalTableOperator with @FedericaLionetto and stumbling on this log line:

google.api_core.exceptions.NotFound: 404 GET https://storage.googleapis.com/download/storage/v1/b/OBJECT_NAME_REDACTED/o/BUCKET_NAME_REDACTED?alt=media: Not Found: ('Request failed with status code', 404, 'Expected one of', <HTTPStatus.OK: 200>, <HTTPStatus.PARTIAL_CONTENT: 206>

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

@boring-cyborg boring-cyborg bot added area:providers provider:google Google (including GCP) related issues labels May 8, 2021
@potiuk potiuk merged commit 3711a29 into apache:master May 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:providers provider:google Google (including GCP) related issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants