-
Notifications
You must be signed in to change notification settings - Fork 86
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
File delete fails for files in the format <GUID>##9999.csv #7916
Comments
@maps-mike My guess is the double hash (##) is what's causing the issue. Some characters need special handling. Strange, though, as the URL in the error message looks correct. Can you confirm that URL of the blob used by Storage Explorer is the same one used by the Azure portal? You can check this by looking at the blob's properties both in Storage Explorer and the portal. |
@maps-mike I am not able to reproduce this issue, either by deleting one or multiple blobs. Can you add any more details about what exactly you were doing? When you refresh the explorer view, are the blobs you're trying to delete still present? |
@maps-mike Have you had a chance to reproduce this issue and gather more information? Anything you can provide can help us investigate further. |
Hi Craig – I’ve just done a test this morning, the exact process is:
* Upload 2 CSV files to a file path “landing/Transactional/error”
* Note path is an ADLSG2 file system, NOT Blob storage
* Performance: Standard
* Replication: LRS
* Account Kind: StorageV2 (general purpose v2)
* Firewall IS enabled.
* Files:
* 35ba9a88-09d6-4c9b-bf7f-3231a283c443##7580.csv
* 2237f762-77f7-4039-a531-968a442359b5##7581.csv
* Files contain 2 lines of CSV data (header & 1 row)
* File size is just over 1K
* I click “delete” in Storage Explorer
* Storage explorer confirms “are you sure” with correct file name
* I click “Yes”
* Failure message returned, file still present in filesystem.
***@***.***
Copy AZ command to clipboard gives me this (XXX for sensitive info)
$env:AZCOPY_CONCURRENCY_VALUE = "AUTO";
$env:AZCOPY_CRED_TYPE = "SharedKey";
$env:ACCOUNT_NAME = "XXX";
$env:ACCOUNT_KEY = "XXX";
./azcopy.exe remove https://XXX.dfs.core.windows.net/landing/Transactional/error/35ba9a88-09d6-4c9b-bf7f-3231a283c443%23%237580.csv --from-to=BlobFSTrash --recursive --log-level=INFO;
$env:AZCOPY_CONCURRENCY_VALUE = "";
$env:AZCOPY_CRED_TYPE = "";
$env:ACCOUNT_NAME = "";
$env:ACCOUNT_KEY = "";
Log contains this:
2024/05/13 06:20:02 AzcopyVersion 10.23.0
2024/05/13 06:20:02 OS-Environment windows
2024/05/13 06:20:02 OS-Architecture amd64
2024/05/13 06:20:02 Log times are in UTC. Local time is 13 May 2024 07:20:02
2024/05/13 06:20:03 ISO 8601 START TIME: to copy files that changed before or after this job started, use the parameter --include-before=2024-05-13T06:19:57Z or --include-after=2024-05-13T06:19:57Z
2024/05/13 06:20:03 Authenticating to destination using SharedKey
2024/05/13 06:20:03 Authenticating to source using SharedKey
2024/05/13 06:20:03 Job-Command remove https://XXX.dfs.core.windows.net/landing/Transactional/error/35ba9a88-09d6-4c9b-bf7f-3231a283c443%23%237580.csv --output-type=json --cancel-from-stdin --from-to=BlobFSTrash --recursive --log-level=INFO
2024/05/13 06:20:03 Number of CPUs: 8
2024/05/13 06:20:03 Max file buffer RAM 4.000 GB
2024/05/13 06:20:03 Max concurrent network operations: will be dynamically tuned up to 3000 (Based on auto-tuning limit. Set AZCOPY_CONCURRENCY_VALUE environment variable to override)
2024/05/13 06:20:03 Check CPU usage when dynamically tuning concurrency: true (Based on hard-coded default. Set AZCOPY_TUNE_TO_CPU environment variable to true or false override)
2024/05/13 06:20:03 Max concurrent transfer initiation routines: 64 (Based on hard-coded default. Set AZCOPY_CONCURRENT_FILES environment variable to override)
2024/05/13 06:20:03 Max enumeration routines: 16 (Based on hard-coded default. Set AZCOPY_CONCURRENT_SCAN environment variable to override)
2024/05/13 06:20:03 Parallelize getting file properties (file.Stat): false (Based on AZCOPY_PARALLEL_STAT_FILES environment variable)
2024/05/13 06:20:03 Max open files when downloading: 2147479959 (auto-computed)
2024/05/13 06:20:03 Final job part has been created
2024/05/13 06:20:03 Trying 4 concurrent connections (initial starting point)
2024/05/13 06:20:03 Final job part has been scheduled
2024/05/13 06:20:03 WARN: [P#0-T#0] Displayed file count will be either 1 or based upon list-of-files entries, and thus inaccurate, as deletes are performed recursively service-side.
2024/05/13 06:20:04 ==> REQUEST/RESPONSE (Try=1/396.6515ms, OpTime=619.7381ms) -- RESPONSE SUCCESSFULLY RECEIVED
HEAD https://XXX.blob.core.windows.net/landing/Transactional%2Ferror%2F35ba9a88-09d6-4c9b-bf7f-3231a283c443%23%237580.csv
X-Ms-Request-Id: [774e3d22-101e-0087-25fd-a44878000000]
2024/05/13 06:20:04 ==> REQUEST/RESPONSE (Try=1/39.9697ms, OpTime=260.7882ms) -- RESPONSE SUCCESSFULLY RECEIVED
DELETE https://XXX.dfs.core.windows.net/landing/Transactional/error/35ba9a88-09d6-4c9b-bf7f-3231a283c443?recursive=false#%237580.csv
X-Ms-Request-Id: [19fc7ae5-301f-0016-05fd-a42dca000000]
2024/05/13 06:20:04 ERR: [P#0-T#0] https://XXX.dfs.core.windows.net/landing/Transactional/error/35ba9a88-09d6-4c9b-bf7f-3231a283c443%23%237580.csv: 404: DELETE ERROR -404 The specified path does not exist.. X-Ms-Request-Id:19fc7ae5-301f-0016-05fd-a42dca000000
2024/05/13 06:20:04 JobID=d2cd78e0-5bab-0444-6f05-3f610e137603, Part#=0, TransfersDone=1 of 1
2024/05/13 06:20:04 all parts of entire Job d2cd78e0-5bab-0444-6f05-3f610e137603 successfully completed, cancelled or paused
2024/05/13 06:20:04 is part of Job which 1 total number of parts done
2024/05/13 06:20:05 PERF: primary performance constraint is Unknown. States: T: 0, GRs: 4
2024/05/13 06:20:05 100.0 %, 0 Done, 1 Failed, 0 Pending, 0 Skipped, 1 Total,
2024/05/13 06:20:05 Closing Log
* Using portal to try delete:
* Go to Stroage account
* Choose Storage Browser
* Blob containers -> <container>
* Click tick box next to file
* Click delete, get confirmation message:
* ***@***.***
* Click Yes, file deleted in Portal & Storage explorer
Comparing URL in Storage Explorer & portal:
Portal
https://PATH/35ba9a88-09d6-4c9b-bf7f-3231a283c443##7580.csv
Storage Explorer:
https://PATH/35ba9a88-09d6-4c9b-bf7f-3231a283c443%23%237580.csv
I agree that its got to be the ## characters – I’m not convinced the correct escape sequence is being used in the command, or maybe its my version of AZCopy?
Anyways, hope this helps.
Mike
|
@maps-mike For testing purposes, can you upload and delete some files with some variations? For example, does the issue still reproduce if:
Strictly speaking, "#" characters are not legal in a URL path. Azure portal might be un-escaping characters for display convenience. The fact that I still can't repo the issue suggests we are escaping the "#" characters correctly:
|
Hi Craig – results & observations below:
* You only have one "#" character.
* [Mike] Same problem – error deleting.
* You have no "#" characters
* [Mike] Success – deletes as expected.
* You change the directory structure, such as moving the file to the root
* [Mike] Same problem – error deleting (from root & other folder paths)
* You delete multiple files at once
* [Mike] Same problem – error deleting.
I have observed the problem only occurs on ADLS Gen2 Storage accounts – if I upload ## files to a Blob storage account the delete works as expected.
Note – I copied & renamed the same file in all tests so all other factors should be consistent.
Short file name cush as “3#.csv” fail as well.
No worries if this one goes down as “cannot reproduce”, I have a workaround, though I’m equally happy to try more tests.
Mike
|
@maps-mike Thank you for that information! Ah ha! Sorry, I should have tried this first: I was able to reproduce the issue using our 1.33.0 release. It doesn't reproduce in our working builds now, so I suspect the issue has already been addressed (by updating our version of AzCopy, probably). Here is a link to a private build. Can you download, install, and give it a try to see if the problem persists? [LINK REMOVED] |
Thanks Craig, I’ll do that – the install needs .Net 8 (& admin rights) so I’ll need to get IT to do that install so might take a little while to get back but sounds like it’ll be sorted!
Mike
|
@maps-mike Any progress? If you need a new link, please let me know. |
Hi Craig – sorry I was a bit slow, got the new version installed now & can confirm the issue is resolved, I can delete these ## files as expected!
Thank You!
|
Excellent! Our next release (1.34.0) contains an AzCopy update, so that means the next release should resolve your issue. |
Preflight Checklist
Storage Explorer Version
1.33.1
Regression From
No response
Architecture
x64
Storage Explorer Build Number
20240410.2
Platform
Windows
OS Version
Windows 10
Bug Description
File delete fails for files in the format ##9999.csv
Steps to Reproduce
Create ADLS G2 storage account
create folder
add a file called ab9f89ba-f069-44b1-84a6-7dda8da39147##7587.csv
try to delete file using storage explorer
Actual Experience
Error message in storage explorer:
Deletion of 'ab9f89ba-f069-44b1-84a6-7dda8da39147##7587.csv' from 'landing/Transactional/upload_api_sandbox/20240430/' failed: 0 completed, 1 failed (used name and key)
Started at: 01/05/2024 08:17, Duration: 4 seconds
Error in log:
2024/05/01 07:17:14 ERR: [P#0-T#0] https://adlsdatahubraw.dfs.core.windows.net/landing/Transactional/upload_api_sandbox/20240430/ab9f89ba-f069-44b1-84a6-7dda8da39147%23%237587.csv: 404: DELETE ERROR -404 The specified path does not exist.. X-Ms-Request-Id:4666f2af-401f-000c-0997-9b4c15000000
Expected Experience
File is deleted
Additional Context
Other files can be deleted without issue it is just files in the format ([GUID]##9999.csv) above that cannot be deleted.
The problem files can still be deleted by other mechanisms - I usually use the Azure portal.
The text was updated successfully, but these errors were encountered: