Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

closes #2496 and #2552 #2603

Merged
merged 1 commit into from Jun 11, 2018
Merged

closes #2496 and #2552 #2603

merged 1 commit into from Jun 11, 2018

Conversation

yuwaMSFT2
Copy link
Contributor

Update Azure SDK with release v16.2.1
Update Azure autorest SDK with release v10.8.1

Signed-off-by: Yu Wang yuwa@microsoft.com

@yuwaMSFT2
Copy link
Contributor Author

@nim-nim @dmcgowan @sajayantony @sivagms
I did some manual test for image pull/push.
Also all storage driver tests passed.

@dmcgowan
Copy link
Collaborator

dmcgowan commented Jun 5, 2018

Try updating the vndr tool locally and rerunning, it looks like that is the reason behind the circleci failure

@GordonTheTurtle
Copy link

Please sign your commits following these rules:
https://github.com/moby/moby/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:

$ git clone -b "newazuresdk" git@github.com:yuwaMSFT2/distribution.git somewhere
$ cd somewhere
$ git rebase -i HEAD~842354362440
editor opens
change each 'pick' to 'edit'
save the file and quit
$ git commit --amend -s --no-edit
$ git rebase --continue # and repeat the amend for each commit
$ git push -f

Amending updates the existing PR. You DO NOT need to open a new one.

@codecov
Copy link

codecov bot commented Jun 5, 2018

Codecov Report

Merging #2603 into master will decrease coverage by 9.83%.
The diff coverage is 0%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2603      +/-   ##
==========================================
- Coverage   60.92%   51.09%   -9.84%     
==========================================
  Files         129      129              
  Lines       11876    11901      +25     
==========================================
- Hits         7236     6081    -1155     
- Misses       3739     5057    +1318     
+ Partials      901      763     -138
Impacted Files Coverage Δ
registry/storage/driver/azure/azure.go 0.7% <0%> (-0.07%) ⬇️
registry/storage/driver/gcs/gcs.go 0.39% <0%> (-68.67%) ⬇️
registry/storage/driver/oss/oss.go 0.56% <0%> (-56.91%) ⬇️
registry/storage/driver/s3-goamz/s3.go 0.5% <0%> (-51.14%) ⬇️
registry/storage/driver/s3-aws/s3.go 4.15% <0%> (-50.98%) ⬇️
registry/storage/cache/cache.go 50% <0%> (-25%) ⬇️
...egistry/storage/cache/cachedblobdescriptorstore.go 45.83% <0%> (-10.42%) ⬇️
registry/client/transport/transport.go 69.69% <0%> (-9.1%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f0cc927...6279723. Read the comment docs.

@yuwaMSFT2
Copy link
Contributor Author

@dmcgowan Thanks! I updated the vendor package and ran dep-validate locally.

@@ -118,7 +119,7 @@ func (d *driver) GetContent(ctx context.Context, path string) ([]byte, error) {

// PutContent stores the []byte content at a location designated by "path".
func (d *driver) PutContent(ctx context.Context, path string, contents []byte) error {
if limit := 64 * 1024 * 1024; len(contents) > limit { // max size for block blobs uploaded via single "Put Blob"
if limit := 256 * 1024 * 1024; len(contents) > limit { // max size for block blobs uploaded via single "Put Blob"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This limit was changed on the Azure side? Was this documented somewhere, I know this hard coded value has been confusing in the past

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. The new SDK is using a newer version header when sending request. The new version supports 256MB. This will allow all driver tests to pass (previously there was one failed test due to the size limit here).
https://docs.microsoft.com/en-us/rest/api/storageservices/put-blob#remarks

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated the PR with comments.

@dmcgowan
Copy link
Collaborator

dmcgowan commented Jun 5, 2018

Much cleaner API. The change looks good, I just had one question about the PutBlob size, and maybe a request to clarify in the comment. Otherwise LGTM, can you also get someone on the Azure side to LGTM here before merging.

@yuwaMSFT2
Copy link
Contributor Author

@marstr @shizhMSFT @colemickens
can you help to review the change?

Copy link

@marstr marstr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it were me, I'd be using this as an opportunity to take advantage of the new repository:
https://github.com/Azure/azure-storage-blob-go

It is the replacement for the blob functions that you're referencing today.

if limit := 64 * 1024 * 1024; len(contents) > limit { // max size for block blobs uploaded via single "Put Blob"
// max size for block blobs uploaded via single "Put Blob" for version after "2016-05-31"
// https://docs.microsoft.com/en-us/rest/api/storageservices/put-blob#remarks
if limit := 256 * 1024 * 1024; len(contents) > limit {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] I'd declare this as:

const limit = 256 * 1024 * 1024
if len(contents) > limit {
    // stuff
}

Copy link
Contributor Author

@yuwaMSFT2 yuwaMSFT2 Jun 6, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated the const declaration.

@yuwaMSFT2
Copy link
Contributor Author

@marstr Regarding the new storage repo, how reliable is it? I saw only prelease builds for github.com/Azure/azure-storage-blob-go ( as well as the referenced package Azure/azure-pipeline-go). This is a bit uncomfortable. Would prefer a stable version. Do you think it's Ok to move to azure-storage-blob-go later once it starts to generate official release?

@marstr
Copy link

marstr commented Jun 7, 2018

@yuwaMSFT2 says:
Regarding the new storage repo, how reliable is it? ... This is a bit uncomfortable. Would prefer a stable version.

Totally fair, there have been a few breaking changes introduced. In terms of functionality, it's pretty robust, and checking in a vendored copy should protect you from having breaking changes forced on you. However, I can totally understand waiting for 1.0.

@yuwaMSFT2
Copy link
Contributor Author

yuwaMSFT2 commented Jun 7, 2018

@marstr Thanks. I will check back and update the reference once the new blob storage sdk has 1.0.
Other than this, any other comments? Please LGTM if you are Ok with it:)

Update Azure SDK with release v16.2.1
Update Azure autorest SDK with release v10.8.1

Signed-off-by: Yu Wang <yuwa@microsoft.com>
@yuwaMSFT2
Copy link
Contributor Author

Thanks @marstr
@dmcgowan I just squashed the commits. Could you please help to merge it? Thanks!

@dmcgowan
Copy link
Collaborator

LGTM

@dmcgowan dmcgowan merged commit 749f6af into distribution:master Jun 11, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants