Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] copyFromUrlWithResponse fails with 404 CannotVerifyCopySource if blob name contains %20 #11204

Closed
daolis opened this issue May 15, 2020 · 6 comments
Assignees
Labels
customer-reported Issues that are reported by GitHub users external to the Azure organization. question The issue doesn't require a change to the product in order to be resolved. Most issues start as that Storage Storage Service (Queues, Blobs, Files)

Comments

@daolis
Copy link

daolis commented May 15, 2020

Scanario

  • Copying Container of Blobstore A to Container of Blobstore B.
  • A Blob exists with the name my%20testfile.txt. You see this name in the AzurePortal (Its not the URL Encoded name)

Application functionality:

  1. Get list of all Blobs from Blobstore A
  2. Create a BlockBlobClient with the name from the source blob (containerClient.getBlobClient(name)
  3. Add a SASToken to the BlobURL
  4. Trigger server side copy process to create Blob in Blobstore B with URL from source blob

Example:

  1. Get blob with name my%20testfile.txt
  2. Create BlockBlobClient:
    URL is e.g. https://xyz.blob.core.windows.net/container/my testfile.txt (URL Encoding)
  3. Add SAS Token
  4. Trigger server side copy: -> 404, because it tries to copy the file my testfile.txt

Error
Exception: com.azure.storage.blob.models.BlobStorageException: Status code 404
Errorcode: CannotVerifyCopySource
Message: The specified blob does not exist.

Reproduce

  • Create a blob with %20 in its name.
  • Create a BlockBlobClient with this name
  • Optional: Add a SAS Token if not public readable
  • Trigger Copy Process with: targeBlockBlobClient.copyFromUrlWithResponse(...)

Expected behavior
Successful copy of the file

Workaround
.. for now is to replace the character % with %20.

  1. my%20testfile.txt to my%2020testfile.txt
  2. URL of BlockBlobClient: https://xyz.blob.core.windows.net/container/my%20testfile.txt
  3. Copy successful
@ghost ghost added needs-triage This is a new issue that needs to be triaged to the appropriate team. customer-reported Issues that are reported by GitHub users external to the Azure organization. question The issue doesn't require a change to the product in order to be resolved. Most issues start as that labels May 15, 2020
@alzimmermsft alzimmermsft added the Storage Storage Service (Queues, Blobs, Files) label May 15, 2020
@ghost ghost removed the needs-triage This is a new issue that needs to be triaged to the appropriate team. label May 15, 2020
@alzimmermsft
Copy link
Member

@gapra-msft @rickle-msft I have a suspicion that this is related to #11183

@rickle-msft
Copy link
Contributor

@daolis Thank you for reporting this.

@alzimmermsft I suspsect you are right about the connection. @gapra-msft Can you be sure to test this case for blobs while you're working on files?

@gapra-msft gapra-msft self-assigned this May 15, 2020
@gapra-msft
Copy link
Member

Hi @daolis

I'm having a little trouble reproducing the issue you are seeing based off the steps you provided. Could you please provide a code sample so I can further investigate?

@daolis
Copy link
Author

daolis commented Jun 2, 2020

Hi @gapra-msft
I created a repo reproducing this issue.
https://github.com/daolis/azureBlobCopy404Example

@gapra-msft
Copy link
Member

Hi @daolis, Thank you for your repro code.

I believe the issue is that when you call listBlobs, it returns the raw, not encoded blob name. However, the getBlobClient method expects a url encoded blob name (especially when the blob name could possibly look url encoded).

Our recommendation is for you to first url encode the blob name before you pass it in the getBlobClient method. We have a utility method that does just that if you'd like to use it. - Utility.urlEncode

Please let me know if you have further questions/concerns.

@rickle-msft
Copy link
Contributor

I am going to close this issue as we have given a recommendation on how to appropriately resolve the problem and there has been no follow up.

Please feel free to reach back out to us if you need further support.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
customer-reported Issues that are reported by GitHub users external to the Azure organization. question The issue doesn't require a change to the product in order to be resolved. Most issues start as that Storage Storage Service (Queues, Blobs, Files)
Projects
None yet
Development

No branches or pull requests

4 participants