Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skip unnecessary gcs bucket metadata requests. #1931

Merged
merged 1 commit into from Oct 16, 2019

Conversation

@jmcarp
Copy link
Contributor

jmcarp commented Oct 13, 2019

What changes are proposed in this pull request?

Use bucket and blob methods in gcs sdk rather than get_bucket and get_blob methods. This saves a handful of unnecessary api calls (since we don't use the bucket and blob metadata) and drops the iam permissions required to use the gcs artifact store: after this patch, users only require access to objects, not to the bucket itself.

How is this patch tested?

Unit tests

Release Notes

Is this a user-facing change?

  • No. You can skip the rest of this section.
  • Yes. Give a description of this change to be included in the release notes for MLflow users.

(Details in 1-2 sentences. You can just refer to another PR with a description if this PR is part of a larger change.)

What component(s) does this PR affect?

  • UI
  • CLI
  • API
  • REST-API
  • Examples
  • Docs
  • Tracking
  • Projects
  • Artifacts
  • Models
  • Scoring
  • Serving
  • R
  • Java
  • Python

How should the PR be classified in the release notes? Choose one:

  • rn/breaking-change - The PR will be mentioned in the "Breaking Changes" section
  • rn/none - No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" section
  • rn/feature - A new user-facing feature worth mentioning in the release notes
  • rn/bug-fix - A user-facing bug fix worth mentioning in the release notes
  • rn/documentation - A user-facing documentation change worth mentioning in the release notes
Copy link
Contributor

smurching left a comment

Thanks @jmcarp, this looks good - were you able to manually QA this patch & verify that e.g. logging & downloading artifacts works?

Copy link
Collaborator

dbczumar left a comment

LGTM! Thank you for your contribution @jmcarp.

@smurching I QA'ed these changes by ensuring that the following behaviors are exhibited:

  1. Artifact logging to an existing bucket with appropriate access credentials succeeds
  2. Artifact logging to an existing bucket with insufficient permissions fails with an informative error message
  3. Artifact logging to a nonexistent bucket fails with an informative error message
  4. Listing/downloading artifacts from an existing bucket with appropriate access credentials succeeds
  5. Listing/downloading artifacts from an existing bucket with insufficient permissions fails with an informative error message
  6. Listing/downloading artifacts from a nonexistent bucket fails with an informative error message.
@dbczumar dbczumar merged commit e550928 into mlflow:master Oct 16, 2019
1 check passed
1 check passed
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@sueann sueann added the rn/none label Oct 29, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.