New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AzureFileCopyV2 failed when uploading to Azure Storage Static Website #7611

Closed
alaconda opened this Issue Jun 30, 2018 · 43 comments

Comments

@alaconda
Copy link

alaconda commented Jun 30, 2018

Hi,

With the new release of Azure Static Website. I can't seem to find a way to upload to its default container of $web. Same setting would work if I switch to a different container.

Upload to container: '$web' in storage account: '<account name>' with blob prefix: '' failed with error: 'AzCopy.exe exited with non-zero exit code while uploading files to blob storage.' For more info please refer to https://aka.ms/azurefilecopyreadme

Is there something extra I need to do? Or is it not supported yet? And if so, is it in the pipeline to be released soon?

Thanks!

@alaconda alaconda changed the title AzureFileCopyV2 failed when uploading to Azure Storge Static Website AzureFileCopyV2 failed when uploading to Azure Storage Static Website Jun 30, 2018

@rajatagrawal-dev

This comment has been minimized.

Copy link
Member

rajatagrawal-dev commented Jul 2, 2018

Hi @alaconda
Thanks for reporting this. AzureFileCopyV2 task uses AzCopy v7.1.0 to copy files. This is the latest version of AzCopy available and it does not support Azure Static Websites. Refer this issue

A new AzCopy version will be released soon. We'll update our task to use that latest version of AzCopy which should support static websites. Depending on when the new AzCopy version will be made available, it will take at least a few weeks for the updated AzureFileCopyV2 task to reach all accounts.

@onpaws

This comment has been minimized.

Copy link

onpaws commented Jul 2, 2018

HI @rajatagrawal-dev I'm curious.
For VSTS users who are interested in testing out the new release of AzureFileCopyV2, is there a way to opt-in before public release? I'd be happy to help test.
I've already opted into the other 'preview' VSTS features but curious about VSTS Tasks specifically.
Thanks

@rajatagrawal-dev

This comment has been minimized.

Copy link
Member

rajatagrawal-dev commented Jul 2, 2018

@onpaws AzureFileCopyV2 (using AzCopy v7.1.0) is already publicly available. When you add Azure File Copy task in your build/release definition, you should see an option of selecting the version of the task. You can select version 2 to test the preview version.

image

@alaconda

This comment has been minimized.

Copy link

alaconda commented Jul 2, 2018

Hi @rajatagrawal-dev,

Thanks for the link. Please keep us posted and let us know if there is anything else we can do to help.

@rajatagrawal-dev

This comment has been minimized.

Copy link
Member

rajatagrawal-dev commented Jul 2, 2018

Sure. I'll update when we've made progress on this.

@nathanvv

This comment has been minimized.

Copy link

nathanvv commented Jul 4, 2018

The file copy with AzureFileCopyV2 doesnt seem to copy the file into the "$web" folder . Logs :

2018-07-04T16:35:34.3835676Z Uploading files from source path: 'C:\agent\_work\46\s\allure-publish\migration\prod\2018.7.4.22\index.html' to storage account: 'somerandomname' in container: '$web' with blob prefix: ''
2018-07-04T16:35:34.3990429Z ##[command] & "AzCopy\AzCopy.exe" /Source:"C:\agent\_work\46\s\allure-publish\migration\prod\2018.7.4.22" /Dest:"https://somerandomname.blob.core.windows.net/$web" /DestKey:"*****" /XO /Y /SetContentType /Z:"AzCopy" /V:"AzCopy\AzCopyVerbose_03702a45-1109-4fdd-9066-5ee504653e34.log" /S /Pattern:"index.html"
2018-07-04T16:35:34.5146935Z [2018/07/04 16:35:34][WARNING] The command line ""C:\agent\_work\_tasks\AzureFileCopy_eb72cb01-a7e5-427b-a8a1-1b31ccac8a43\2.0.1\AzCopy\AzCopy.exe" /Source:C:\agent\_work\46\s\allure-publish\migration\prod\2018.7.4.21 /Dest:https://somerandomname.blob.core.windows.net//migration /DestKey:****** /XO /Y /SetContentType /Z:AzCopy /V:AzCopy\AzCopyVerbose_a7e93134-e33d-4219-a3ff-a502ec4e6d51.log /S /Pattern:index.html" in the journal file "AzCopy\AzCopy.jnl" is different from your input.
2018-07-04T16:35:34.5147946Z [2018/07/04 16:35:34][WARNING] Incomplete operation with different command line detected at the journal directory "AzCopy".

notice how the " /Dest:https://somerandomname.blob.core.windows.net//migration" is missing the $web. The file is however being copied to the $root folder when I check it thru explorer

@rajatagrawal-dev

This comment has been minimized.

Copy link
Member

rajatagrawal-dev commented Jul 4, 2018

Currently the task does not support static websites, because AzCopy does not support static websites. Container names cannot contain '$' symbol in them (unless storage account is configured as static website) and so $web is invalid container name. Also, as you pointed out, there is bug in task which misbehaves when $web is specified, and it missed in the URL. AzCopy defaults the container to $root. This will be fixed soon.

@nathanvv

This comment has been minimized.

Copy link

nathanvv commented Jul 4, 2018

I have the storage account configured as static website, just running into the issue of $web missing in the url and hence the files being copied into $root. Appreciate the help. Thanks a lot again.

@onpaws

This comment has been minimized.

Copy link

onpaws commented Jul 5, 2018

Hi @rajatagrawal-dev,
Thanks for posting back on how to select the 2.x FileCopy task.
I'm actually already using that with no luck on the $web as other are reporting.
Was offering to help test out the next release after 2.x (3.x or whatever it's called?)
Either way thanks for your efforts, I know at least a couple people at my company who will be excited to get this

@rajatagrawal-dev

This comment has been minimized.

Copy link
Member

rajatagrawal-dev commented Jul 5, 2018

@onpaws Support for $web for static websites will be available a few after AzCopy tool is updated. It will be a minor version update only, so it will also be version 2.x. I'll let you know the exact version of task you should see when we have made the changes.

@sonu2alok

This comment has been minimized.

Copy link

sonu2alok commented Jul 10, 2018

I tried with 2.x FileCopy task but still getting the same error:

Upload to container: '$web' in storage account: 'azaueastdevXXXXX' with blob prefix: '' failed with error: 'AzCopy.exe exited with non-zero exit code while uploading files to blob storage.' For more info please refer to https://aka.ms/azurefilecopyreadme

@SumiranAgg

This comment has been minimized.

Copy link
Contributor

SumiranAgg commented Jul 10, 2018

@sonu2alok The task currently doesn't support static website. We will update the thread once the enhancement is in place.

@kenotron

This comment has been minimized.

Copy link
Member

kenotron commented Jul 10, 2018

Looks like the azcopy 7.30 just came out. Can you please update this task to support that version?

@rajatagrawal-dev

This comment has been minimized.

Copy link
Member

rajatagrawal-dev commented Jul 12, 2018

We've updated the AzCopy version to v7.3.0. The task should now be able to handle upload to $web container.
The fix will roll out in the next deployment cycle. So it will take a couple of weeks to reach all accounts.
The fix will be available in Azure File Copy task v2.0.6 and above.

@Jaxwood

This comment has been minimized.

Copy link

Jaxwood commented Jul 12, 2018

@rajatagrawal-dev how can I tell which version my account is running?

@rajatagrawal-dev

This comment has been minimized.

Copy link
Member

rajatagrawal-dev commented Jul 12, 2018

@Jaxwood One way is to run the task. If you see the release logs of your task, you'll find the version of the task near the top of the logs.
image

Another way is to get data from tasks api. Just enter the following URL in your browser (assuming you are logged in to your account):
https://{you_account_name}.visualstudio.com/_apis/Distributedtask/tasks.
This will give you a json of task versions in your account. Search for "AzureFileCopy" (with quotes) in that json and you should find 2 search results (for V1 and V2 of the task). Version will be there right after the search match locations.

@nathanvv

This comment has been minimized.

Copy link

nathanvv commented Jul 12, 2018

Just curious, looks like we are still at v2.0.1. Is there a release log where I could check to get the date of release of v2.0.1 ?. More curious to get a rough idea oh how long it would take for us to get this patch

image

@rajatagrawal-dev

This comment has been minimized.

Copy link
Member

rajatagrawal-dev commented Jul 12, 2018

@nathanvv Patch versions of tasks are updated on almost every check-in of code. So the patch versions can easily jump multiple versions between 2 releases. The latest release hasn't been rolled out to all accounts so you might see v2.0.4 in a few days.
For v2.0.6 and above, it will take at least 2-3 weeks more for all accounts to get updated.

@denis1stomin

This comment has been minimized.

Copy link

denis1stomin commented Jul 12, 2018

Great job!
I've just checked new AzCopy today and it works as expected with $web container.
Will wait till new version will be available for my account. Thanks!

@alaconda

This comment has been minimized.

Copy link

alaconda commented Jul 13, 2018

@rajatagrawal-dev, thanks for the update. Looking forward to > v2.0.6 once it rolled out to my account.

@markusdresch

This comment has been minimized.

Copy link

markusdresch commented Jul 25, 2018

now it does transfer files from the root directory to $root (instead of $web) and then it fails with:

2018-07-25T09:59:33.5323085Z Uploading files from source path: 'D:\a\1\s\src\xxx\wwwroot' to storage account: 'xxx' in container: '$web' with blob prefix: ''
2018-07-25T09:59:33.5545963Z ##[command] & "AzCopy\AzCopy.exe" /Source:"D:\a\1\s\src\xxx\wwwroot" /Dest:"https://xxx.blob.core.windows.net/$web" /@:"D:\a\_temp\xxx" /XO /Y /SetContentType /Z:"AzCopy" /V:"AzCopy\AzCopyVerbose_xxx.log" /S
2018-07-25T09:59:39.2893962Z [2018/07/25 09:59:39][ERROR] Error parsing destination location "https://xxx.blob.core.windows.net/": The remote server returned an error: (400) Bad Request. For more details, please type "AzCopy /?:Dest" or use verbose option /V.
2018-07-25T09:59:39.2916948Z [2018/07/25 09:59:39][ERROR] HttpStatusMessage:The requested URI does not represent any resource on the server.
2018-07-25T09:59:39.2917359Z RequestId:xxx
2018-07-25T09:59:39.2919288Z Time:Wed, 25 Jul 2018 09:59:37 GMT
2018-07-25T09:59:39.2934672Z [2018/07/25 09:59:39] Transfer summary:
2018-07-25T09:59:39.2934932Z -----------------
2018-07-25T09:59:39.2935843Z Total files transferred: 2
2018-07-25T09:59:39.2936898Z Transfer successfully:   2
2018-07-25T09:59:39.2937949Z Transfer skipped:        0
2018-07-25T09:59:39.2938749Z Transfer failed:         0
2018-07-25T09:59:39.2939543Z Elapsed time:            00.00:00:05
2018-07-25T09:59:40.7827898Z ##[error]Upload to container: '$web' in storage account: 'xxx' with blob prefix: '' failed with error: 'AzCopy.exe exited with non-zero exit code while uploading files to blob storage.' For more info please refer to https://aka.ms/azurefilecopyreadme

subdirectories are omitted.

@rajatagrawal-dev

This comment has been minimized.

Copy link
Member

rajatagrawal-dev commented Jul 25, 2018

@markusdresch In what version of the task are you seeing this behavior? You'll find the task version near the top of the logs.

@markusdresch

This comment has been minimized.

Copy link

markusdresch commented Jul 25, 2018

2018-07-25T09:58:15.8633365Z Task         : Azure File Copy
2018-07-25T09:58:15.8633592Z Description  : Copy files to Azure blob or VM(s)
2018-07-25T09:58:15.8633884Z Version      : 2.0.4

ah ok, so i'll wait for 2.0.6. i just got excited since the output somehow changed from last time.

@rajatagrawal-dev

This comment has been minimized.

Copy link
Member

rajatagrawal-dev commented Aug 2, 2018

Deployment is a bit delayed. It is planned to complete by next week. I'll post more updates when I have any.

@markusdresch

This comment has been minimized.

Copy link

markusdresch commented Aug 10, 2018

works now with v2.0.7

@rajatagrawal-dev

This comment has been minimized.

Copy link
Member

rajatagrawal-dev commented Aug 15, 2018

Static website support should be available in all accounts now. Please check if you are seeing Azure File Copy v2.0.7 in your builds/releases and whether you are able to deploy to $web container successfully.

@jonathantower

This comment has been minimized.

Copy link

jonathantower commented Aug 16, 2018

It's getting close to working, but what's that backtick doing in there?

image

@rajatagrawal-dev

This comment has been minimized.

Copy link
Member

rajatagrawal-dev commented Aug 16, 2018

@jonathantower You can ignore that backtick. We add it because when we actually execute this command, AzCopy resolves all strings starting with '$'. So it would convert $web null/empty string because it would treat $web as a powershell variable. Hence, we escape the '$'. The actual command that is executed does not contain this backtick.

You can add a variable named 'system.debug' with value 'true' in your release to get the debug logs. In those logs, you will find the AzCopy logs included and it will show the exact AzCopy command executed.
If your upload to $web container is not working as expected, please send us these debug logs at RM_Customer_Queries [at] microsoft [dot] com and we will try to debug the issue.

@johnhydemtm365

This comment has been minimized.

Copy link

johnhydemtm365 commented Aug 16, 2018

Hi still not working, unless I have mis-configured something... :-(

Version is 2.0.7
2018-08-16T08:18:28.4657458Z ##[section]Starting: AzureBlob File Copy
2018-08-16T08:18:28.4665256Z
2018-08-16T08:18:28.4665631Z Task : Azure File Copy
2018-08-16T08:18:28.4665961Z Description : Copy files to Azure blob or VM(s)
2018-08-16T08:18:28.4666259Z Version : 2.0.7
2018-08-16T08:18:28.4666564Z Author : Microsoft Corporation
2018-08-16T08:18:28.4666921Z Help : More Information
2018-08-16T08:18:28.4667301Z

Error:
2018-08-16T08:20:02.4436158Z ##[command] & "AzCopy\AzCopy.exe" /Source:"D:\a\1\s\public" /Dest:"https://***.blob.core.windows.net/`$web" /@:"D:\a_temp\stnrkcbu06m5c9msjptfyldi" /S
2018-08-16T08:20:04.4304443Z [2018/08/16 08:20:04][ERROR] Error parsing destination location "https://
.blob.core.windows.net/$web": Failed to validate destination. The remote server returned an error: (403) Forbidden.
2018-08-16T08:20:04.4305539Z HttpStatusMessage:This request is not authorized to perform this operation.
2018-08-16T08:20:04.4306049Z RequestId:b6273607-101e-0017-5139-355a77000000
2018-08-16T08:20:04.4306550Z Time:Thu, 16 Aug 2018 08:20:03 GMT For more details, please type "AzCopy /?:Dest" or use verbose option /V.

Thanks
John

@rajatagrawal-dev

This comment has been minimized.

Copy link
Member

rajatagrawal-dev commented Aug 16, 2018

@johnhydemtm365 Does this only happen with $web container or with any container name? Can you re-execute with some other container name like "testContainer" instead of "$web"?

Also, please enable debug logging. Add a variable in your release definition called 'system.debug' and set it's value to 'true'. Then create a new release to get debug logs.

@johnhydemtm365

This comment has been minimized.

Copy link

johnhydemtm365 commented Aug 16, 2018

Hi nope, does not work with a normal container. Updated AzCopy Locally and it all works fine in a windows batch file.
Log File:
Vsts_AzcopyTaskFailure.txt

@rajatagrawal-dev

This comment has been minimized.

Copy link
Member

rajatagrawal-dev commented Aug 16, 2018

@johnhydemtm365 Since the issue does not seem to be related to static website support but rather related to permissions, can you open a new issue for this to get the right traction. Please attach the logs in that as well.

@jonathantower

This comment has been minimized.

Copy link

jonathantower commented Aug 16, 2018

@rajatagrawal-dev I figured out my issue, and it is definitely something you'll want to fix. In the "Azure File COpy (Preview)" task, I had mistakenly specified the Source to be "/", thinking it was a relative path. I noticed in the logs that it was copying from c:\ on the underlying VM! When I changed Source to $(Build.SourcesDirectory), it worked, but I thought you'd want to know about this issue.

@rajatagrawal-dev

This comment has been minimized.

Copy link
Member

rajatagrawal-dev commented Aug 16, 2018

@jonathantower We expect you to give absolute paths in source path input or use pre-defined build/release variables like $(Build.SourcesDirectory) and $(System.ArtifactsDirectory). This is also specified in the help text of the input.
"/" and "\" paths in Windows OS usually correspond to the root C:\ path only . If you execute cd / from a command prompt in Windows it takes you to "C:\". So this behaviour is expected by design.

@ociata

This comment has been minimized.

Copy link

ociata commented Oct 10, 2018

Sorry for posting in this thread, however I've got the very same issue as original issuer. What is frustrating me, is how hard is to debug the problem.

Attached is my full error log, however the interesting part is:
2018-10-10T07:05:57.4949064Z [2018/10/10 07:05:57][ERROR] Error parsing source location "D:\a\r1\a\Internal-Blog": Failed to enumerate directory D:\a\r1\a\Internal-Blog\ with file pattern *. The system cannot find the path specified. (Exception from HRESULT: 0x80070003) For more details, please type "AzCopy /?:Source" or use verbose option /V.
I guess it is related to my setup of Source:
http://prntscr.com/l4brys

Yet I'm completely lost of how to explore and/or download my build artefact to see how to properly setup this path. Other option will be to somehow list/browse/explore the machine to see what is on D:\a\r1\a\I. I've found following link describing how to do just that:
https://docs.microsoft.com/en-us/azure/devops/pipelines/build/artifacts?view=vsts

... yet my UI is completely different and I cannot found aforementioned options:
http://prntscr.com/l4btiw

Any help will be highly appreciated, and if this is not the proper thread to ask please advise how to proceed.

Regards,

@kmkumaran kmkumaran reopened this Oct 11, 2018

@kmkumaran kmkumaran assigned chshrikh and unassigned SumiranAgg Oct 11, 2018

@ociata

This comment has been minimized.

Copy link

ociata commented Oct 11, 2018

Okay, I figured it out - yet I wish the documentation was a little bit more helpful.

So I was missing publish artefacts from my azure-pipeline yaml:

pool:
  vmImage: 'Ubuntu 16.04'

steps:
- task: NodeTool@0
  inputs:
    versionSpec: '8.x'
  displayName: 'Install Node.js'

- script: |
    cd srv
    npm install
    npm run build
  displayName: 'npm install and build'

# This was missing
- task: PublishBuildArtifacts@1
  inputs:
    PathtoPublish: '$(System.DefaultWorkingDirectory)/srv/public'

After I added this step, my UI now properly displays the published artefact and I can download and/or explore it:
http://prntscr.com/l4pemh

One more important thing I found the hard way - you can only deploy to AzureBlob from Windows Hosted machine (I had mine setup with Ubuntu). The throwing error is something like

you have script build for Windows only such as powershell

I haven't, but obviously the uploading job has :)

Hope this is useful to someone in the future.

Regards,

@omeshp

This comment has been minimized.

Copy link
Contributor

omeshp commented Oct 12, 2018

@ociata Which task are you using to deploy to AzureBlob which is giving the powershell error?

@ociata

This comment has been minimized.

Copy link

ociata commented Oct 15, 2018

@ociata Which task are you using to deploy to AzureBlob which is giving the powershell error?

AzureBlob File Copy (Preview)

@omeshp

This comment has been minimized.

Copy link
Contributor

omeshp commented Oct 15, 2018

@ociata ok, it's a powershell based task hence the error. @kmkumaran Do you know if we have plans to support this task on linux or if there is an alternative?

@rajatagrawal-dev

This comment has been minimized.

Copy link
Member

rajatagrawal-dev commented Oct 15, 2018

@ociata You can try using the Azure CLI task to upload to blob on linux agents.
Please refer this for details on using Azure CLI to upload to blob.

@timtlm

This comment has been minimized.

Copy link

timtlm commented Oct 16, 2018

I am using this feature to deploy my static website using Azure DevOps, and I have it working properly with v2, but I want to be able to delete everything before deploying. Currently, if I remove a file from my project, it will still remain in the folder after this pipeline task.
There is some mention of being able to do this here: https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-file-copy
Release pipelines don't support yaml, so it isn't very helpful. However, I tried placing /cleanTargetBeforeCopy:true in the optional arguments and it says the option is not recognized.
According to the readme here, it should just be an option somewhere in the task, but I don't see it anywhere. https://github.com/Microsoft/azure-pipelines-tasks/tree/master/Tasks/AzureFileCopyV2

@rajatagrawal-dev

This comment has been minimized.

Copy link
Member

rajatagrawal-dev commented Oct 17, 2018

@timtlm The cleanTargetBeforeCopy option is for when you are copying to a VM. It is not for copying to blob storage.
Right now AzCopy tool (which Azure FIle Copy task uses internally) doesn't seem to provide the option to delete the container. A workaround will be to add a Azure Powershell task before the Azure File Copy task in your release pipeline and execute the following command:
Remove-AzureRmStorageContainer -ResourceGroupName "myResourceGroup" -AccountName "myStorageAccount" -ContainerName "myContainer"
Refer this.
This will delete the entire container. The container will be automatically recreated when Azure File Copy task is executed.

@timtlm

This comment has been minimized.

Copy link

timtlm commented Oct 17, 2018

@rajatagrawal-dev Thanks for the follow-up and the info. I ended up using the Azure CLI task with the following command:
az storage blob delete-batch --account-name <storage_account_name> --source $web
It worked as needed. I felt a little uncomfortable removing the $web container as it seems to have some special properties that go along with it, but I'm guessing your solution would work as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment