Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UploadStreamToBlockBlob: file doesn't appear in Azure Storage #17131

Closed
ItalyPaleAle opened this issue Feb 21, 2022 · 6 comments
Closed

UploadStreamToBlockBlob: file doesn't appear in Azure Storage #17131

ItalyPaleAle opened this issue Feb 21, 2022 · 6 comments
Assignees
Labels
Client Storage

Comments

@ItalyPaleAle
Copy link
Member

@ItalyPaleAle ItalyPaleAle commented Feb 21, 2022

Bug Report

  • import path of package in question: github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v0.3.0
  • output of go version: go version go1.17.5 linux/arm64

Using UploadStreamToBlockBlob to upload a block blob from a stream doesn't seem to work. The method returns no error, but the file doesn't appear in Azure Storage.

This is an extract of the code that I'm using, and I'm connected to an Azure Storage account running on Azure (no emulator). The connection is successful, as I'm able to, for example, create containers.

blob := containerClient.NewBlockBlobClient(name)
ifNoneMatch := "*"
opts := azblob.UploadStreamToBlockBlobOptions{
	BufferSize: 5 * 1024 * 1024,
	MaxBuffers: 2,
	BlobAccessConditions: &azblob.BlobAccessConditions{
		ModifiedAccessConditions: &azblob.ModifiedAccessConditions{
			IfNoneMatch: &ifNoneMatch,
		},
	},
	Metadata: metadata,
}
res, err := blob.UploadStreamToBlockBlob(ctx, in, opts)

Note that in is a io.Reader readable stream, and metadata is a map[string]string.

Running the code above, I don't get any error (err is nil), and res shows a 201 response. It also contains a valid ETag. But nothing is created in the Storage Account.

PS: I have to use UploadStreamToBlockBlob rather than Upload because the stream doesn't implement a seeker (would be great if that wasn't a requirement in the first place, as that API seems more "complete").

@msftbot msftbot bot added the needs-triage label Feb 21, 2022
@jhendrixMSFT jhendrixMSFT added the Storage label Feb 22, 2022
@msftbot msftbot bot removed the needs-triage label Feb 22, 2022
@jhendrixMSFT jhendrixMSFT added the Client label Feb 22, 2022
@zezha-msft
Copy link
Member

@zezha-msft zezha-msft commented Feb 23, 2022

Hi @ItalyPaleAle, thanks for reaching out!

Receiving a successful HTTP status indicates that the blob was most likely created successfully. How are you checking the existence of the blob btw? If it's with the portal or Azure Storage Explorer, could you please refresh just to double check?

Please let us know and we'll open an investigation accordingly upon your confirmation. Thanks.

@ItalyPaleAle
Copy link
Member Author

@ItalyPaleAle ItalyPaleAle commented Feb 23, 2022

@zezha-msft I'm looking at the Azure Portal and the blob doesn't appear in the container. I have tried refreshing the page.

@ItalyPaleAle
Copy link
Member Author

@ItalyPaleAle ItalyPaleAle commented Feb 23, 2022

I've been trying to repro this outside of my own code's test environment but I can't repro it correctly yet. I'm going to close this for now assuming that's a bug in my code. Will reopen if I can create a reliable repro environment :)

@ItalyPaleAle
Copy link
Member Author

@ItalyPaleAle ItalyPaleAle commented Feb 23, 2022

@zezha-msft I figured out what the issue is. There is a bug in the SDK that I can now repro, although it's different that I originally thought. The file is created, but my tests were then deleting it because of a bug somewhere else.

In essence, when using UploadStreamToBlockBlob, the blob access conditions are ignored. You can see in my code above that I use IfNoneMatch: &ifNoneMatch to only create the blob if it doesn't exist. That was working fine with the old ("track 1") SDK, but it's not working with the new one. That caused the tests to overwrite the file, and then the next test was deleting it making it look like it was never there. I'm going to open a separate issue for that with full repro code.

@zezha-msft
Copy link
Member

@zezha-msft zezha-msft commented Feb 23, 2022

Thanks @ItalyPaleAle for the confirmation, much appreciated.

@mohsha-msft
Copy link
Member

@mohsha-msft mohsha-msft commented Apr 20, 2022

Hey @ItalyPaleAle ,

azblob v0.4.0 is now publically available. I have fixed the issue here and here.

Please reach out if it still persists.

Thanks a lot for your feedbacks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Client Storage
Projects
None yet
Development

No branches or pull requests

4 participants