-
Notifications
You must be signed in to change notification settings - Fork 212
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
azcopy sync fails with "AuthenticationErrorDetail: se is mandatory", however the se is definitely included #122
Comments
Also confirmed on Alpine Linux, reproduction case: Reproduction case with Docker:
|
So, it turns out the tokens generated are not complete!
Notice: Unfortunately the current version of this tool does not know how to parse the URL correctly and figure out what's wrong. |
Good catch on the cause there, however, that's not totally satisfactory because when I did my initial test, I created the SAS token directly using the az tool like this:
However, I grant you I think "az" is a separate project from this one, so if it's generating invalid URLs, maybe the issue should be against that project, although improving azcopy's URL parsing would be nice as well, I think we need the devs to chime in. |
Hi @angrygreenfrogs @hyperized, I apologize for the delayed response. I have created an issue on the CLI repo and will follow up with them to resolve this problem. |
@zezha-msft can we perhaps reopen this as the error given is not exactly helpful and actually incorrect? Can you maybe verify the individual argument fields in the tool before sending it out to the endpoint? |
@hyperized sure! Let me take a closer look and get back to you. |
Hi @angrygreenfrogs @hyperized, thanks for your patience. After some investigation and help from the CLI Team, I've confirmed that the problem is actually in our Go Blob SDK. Despite what the doc says, an expiration time like "2030-01-01" is actually accepted by the Storage service. It is the Go SDK not parsing the time properly. I've logged an item in our backlog to fix this issue. Thanks! |
@zezha-msft Thank you, that's great to hear and appreciate the response |
Part of the bug is also the documentation of the az tool. It specifies:
But that format is not that strict. If I use Y-m-d'T'H:M:S'Z' instead (%Y-%m-%dT%H:%M:%SZ in strftime) it works correctly. |
Hi @silvester747, thanks for the feedback! You are indeed right that the Storage service is not very strict in terms of the time format. But I think the CLI's documentation is fine, since a suggested format should be given so that it's easier for users to understand. The bug is in the Go Blob SDK, while fails to parse the variation of formats. |
The Azure Node.js SDK also generates the SAS token with time format |
I'm having a similar issue. The REST API allows SAS tokens to be generated that work in other tools (e.g. Storage Explorer), but result in "Authentication Failed" when using AzCopy. To replicate, generate some SAS tokens via the "Try It" option here: https://docs.microsoft.com/en-us/rest/api/storagerp/storageaccounts/listaccountsas This caught me out as I'm generating a SAS token from an ARM template using the listAccountSas function, and passing it into a container as an environment variable which is later used by AzCopy. Here's the payload I'm using to generate a SAS token:
And here's an example of how I am trying to use it (real resources, but now nuked):
Things I've noticed:
So, while the SAS token APIs (accessible via REST or AZ CLI) are quite happy to generate tokens with different date formats, I suspect that AzCopy isn't handling them verbatim and is breaking the signature as a result. |
@nathankitchen, thanks for reaching out! Yes, this is a known issue, we'll try to schedule the work item as soon as possible. In the mean time, please adjust the time formats to work around it. We are sorry for the inconvenience! |
@zezha-msft Are there any updates on this issue? |
@raidlman No updates just yet, sorry. Once release 10.3 is out (soon) we'll take a look at outstanding work and check priorities. |
@raidlman sorry we haven't picked up this item yet, since there are workarounds. We'll try to schedule it as soon as the more prioritized items are done. |
Apologies for being a little slow on the uptake here, but exactly what are the work-arounds? I've tried modifying the expiry time stamp in every way I can imagine with no luck. The sas I generate works fine in az commands, so I don't think that's the problem. Is there a know good format for expiry to use in, say, az storage container generate-sas', that would provide a valid (for azCopy) sig? Thanks! |
@zezha-msft can you note the workarounds here pls? |
Hi @mifydnu, thanks for reaching out. As noted earlier in this thread, the documented format works fine, e.g. 2019-10-16T21:50:18Z. |
As fate would have it, I just figured it out with help from another github post on the issue (sorry I can't find it now to credit the author). It was as simple as including a literal in the date format for '00Z':
I do not know that this is exactly what will work for others but after many hours of hair-pulling it's worked for me. @zezha-msft thanks for the link just now. Doesn't seem to load, but thanks! |
@mifydnu glad to hear you figured it out! We are really sorry for the inconvenience. We'll try to schedule this work item as soon as possible. |
This is also a problem when using SAS tokens with containers and storage policies (no explicit times in the signature). For example, this does not work on OSX, but does work on Windows using v10.3.1 on both.
|
I've been banging my head for a while to no avail, happy I stumbled on this GH Issue, I'm having the same problem. Eagerly awaiting the bug fix. Can confirm the workaround of specifying the format as the Linux command |
Has anyone managed to get SAS tokens generated from AZ CLI to work with 'azcopy copy' I've tried generating tokens with all of the mentioned datetime formats mentioned here as well as the format from the AZ CLI docs and azcopy spits them all back. Examples tried:
Using this AZ command to generate the SAS:
For reference I've compared a valid SAS token generated via the Azure Portal and a token generated from AZ CLI with %3A changed to : and they are more or less identical in formatting so this is very confusing as to what is going on with azcopy. The only difference I can see is firstly the parameters of the SAS token are in a different order (Shouldn't matter though) e.g portal generated token starts with NOTE: These are not valid tokens I've changed the values in them so don't worry about redacting Az CLI Generated Token (With re-arranging): Az CLI Generated Token (Without re-arranging): I opened a ticket with the AZ CLI team as well regarding this issue. |
After more testing I eventually got SAS tokens from AZ CLI to work.
|
This eventually worked for me as well. Azcopy seems to need the seconds in start and expiry in order for it to work, however, this is unfortunately not documented anywhere. |
It's been over a year since this bug was found, has there been any progress in fixing it? |
No, sorry. We have been getting a lot of high priority feature requests from a lot of different sources, and have not been able to work on this one yet. I agree that's not ideal. We have not forgotten about it. |
Can you please confirm how to apply the date format workaround when calling the listAccountSas rest api directly (through postman for example), and not from the Azure CLI? I have tried the format below in my request, which returns a token that works with azcopy make, but not azcopy copy (same issue as above): |
I don't know the answer, sorrry. @adreed-msft Any ideas for lflemmingweb's question above? |
@lflemingweb I don't think there's any way to get the REST API to produce a working sas. The problem is that even if you pass it a datetime that works with azcopy, the API will return a sas that includes decimals in the seconds field, which fails. |
I spent way more time digging into this than I expected last night. I was able to produce a working local version of azcopy by making changes purely to Blob Storage Go SDK. It looks like azcopy has forks of some of the files I was changing, but me making my edits there had seemingly no effect. Unsure if it's dead code, or hidden behind a switch. I'm going to try to prepare a PR for the Go SDK this week, to help speed things along. It'll be my first change to a Go codebase! 💪 |
@AtOMiCNebula, I had an email about this same topic from @adreed-msft this morning? Are you already in touch with her about this? |
I hadn't been...sounds like a coincidence! 😇 |
This has been affecting the Service Fabric diagnostic support case for a while as mentioned in this bug on ASE which consumes azcopy: We use Azure Storage Explorer to download logs from storage accounts via SAS and since azcopy does not decode the SAS key correctly we are having to use an older version of ASE which doesn't appear to use azcopy: Transfer of 'fabriclogs-bec4a403-0026-4589-... by Alfredo Santamaria Gomez |
+1 to fixing docs . In particular, for CLI users, an end-to-end example showing the happy path. For us, it was a combo of (a) timezone issue and (b) auth mode (needing to use account-key instead of auth-mode login).
|
I am having the same problem with azure-storage-deploy and can't figure out what the correct format is. I have tried
Still, there is no hope! The actual
It works locally with AzCopy 10.5.0 just fine. Ist there any workaround in the meantime? |
I needed to use After trying out most suggestions, the only thing that worked for me was what @lmeyerov suggested, however, I don't need to use the For others that find this issue, I did it with: - task: Bash@3
displayName: Install azcopy
inputs:
targetType: 'inline'
script: |
mkdir $(Agent.ToolsDirectory)/azcopy && cd "$_"
wget -O azcopy_v10.tar.gz https://aka.ms/downloadazcopy-v10-linux
tar -xf azcopy_v10.tar.gz --strip-components=1
- task: AzureCLI@2
displayName: Download using azcopy
inputs:
azureSubscription: my-vmssagents-service-connection
scriptType: bash
scriptLocation: inlineScript
inlineScript: |
export STORE_NAME="data"
export CONTAINER_NAME="data"
NOW=`date +"%Y-%m-%dT%H:%M:00Z"` \
EXPIRY=`date -d "$NOW + 1 day" +"%Y-%m-%dT%H:%M:00Z"` \
&& export SAS_TOKEN=$( az storage container generate-sas \
--account-name $STORE_NAME \
--name $CONTAINER_NAME \
--start $NOW \
--expiry $EXPIRY \
--permissions acdlrw \
--output tsv )
$(Agent.ToolsDirectory)/azcopy/azcopy copy \
"https://${STORE_NAME}.blob.core.windows.net/${CONTAINER_NAME}/${{ parameters.folder }}/?${SAS_TOKEN}" \
"." --recursive --include-pattern "*c_*b.nc;left.nc;right.nc" # <-- my specific pattern |
How do I tell azcopy NOT to drop
|
and azcopy expects decoded one:
azcopy shows funny errors to cheer you up and make you think! |
The original time parsing issue is resolved. Please open new issues if you are still experiencing problems with your SAS tokens, but first please double check that you are passing the token in correctly. |
AzCopy 10.0.4-Preview on Ubuntu 14.04
Command:
azcopy sync 'https://production8858.blob.core.windows.net/wad-iis-logfiles?se=2030-01-01&sp=rl&sv=2018-03-28&sr=c&sig=REDACTED' /backup --recursive
SAS was generated using:
The "se" value is definitely included, but azcopy fails.
Result:
The text was updated successfully, but these errors were encountered: