Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: allow no project in client methods using storage emulator #703

Merged
merged 4 commits into from
Feb 4, 2022

Conversation

cojenco
Copy link
Contributor

@cojenco cojenco commented Jan 25, 2022

When using a storage emulator, allow no project/fake project marker in client.create_bucket() and client.list_buckets()

Alongside, this removes client-side validation and falls back to API validations. In this case, if a project is not set, instead of raising a ValueError in the client, the GCS server will return a BadRequest with message Required parameter: project

Fixes #699

@product-auto-label product-auto-label bot added the api: storage Issues related to the googleapis/python-storage API. label Jan 25, 2022
@cojenco cojenco marked this pull request as ready for review January 25, 2022 04:09
@cojenco cojenco requested review from a team as code owners January 25, 2022 04:09
if project is None:
raise ValueError("Client project not set: pass an explicit project.")
# Use no project if STORAGE_EMULATOR_HOST is set
if _get_storage_host().find("storage.googleapis.com") < 0:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Three points to make here:

I think if "storage.googleapis.com" in _get_storage_host() is a better idiom than using find().

We should avoid hardcoding strings like "storage.googleapis.com" when possible. Perhaps we should have a constant, and/or a helper function that answers the question consistently and globally of whether we are using the emulator?

Is there no way to avoid needing the client to be aware of the emulator at all? Perhaps it's the emulator that needs to change here?

Copy link
Contributor Author

@cojenco cojenco Jan 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review! Updated according to your comments.

The user ask is to be able to run the same code against a storage emulator as well as the live server (without writing extra conditions or wrappers), often seen in testing.

client.create_bucket() and client.list_buckets() are the only 2 library methods that explicitly expect a project. I think it makes sense to allow these 2 methods to detect the emulator and use a fake project marker.

google/cloud/storage/client.py Outdated Show resolved Hide resolved
@cojenco cojenco requested a review from andrewsg January 28, 2022 18:53
Copy link
Contributor

@unforced unforced left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM; aside from the comments but I might just be missing something there.

google/cloud/storage/client.py Show resolved Hide resolved
google/cloud/storage/client.py Show resolved Hide resolved
@cojenco cojenco merged commit bcde0ec into googleapis:main Feb 4, 2022
@cojenco cojenco deleted the emu699 branch November 16, 2023 22:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: storage Issues related to the googleapis/python-storage API.
Projects
None yet
3 participants