New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Azure File Copy Task Fails with Azure PS Version 1.4.0 #1660

Open
thoughtentity opened this Issue May 8, 2016 · 20 comments

Comments

Projects
None yet
@thoughtentity
Copy link

thoughtentity commented May 8, 2016

When a build agent has Azure PS Cmdlets version 1.4.0 installed the Azure File Copy Task will fail with the following error:

Cannot bind argument to parameter 'storageKey' because it is an empty string.

The previous version of Azure PS Cmdlets 1.3.2 works fine.


Raw Task Logs with Azure PS 1.4.0

2016-05-08T06:24:48.9801943Z Executing the powershell script: C:\Agent\tasks\AzureFileCopy\1.0.52\AzureFileCopy.ps1
2016-05-08T06:24:49.1676994Z Looking for Azure PowerShell module at C:\Program Files (x86)\Microsoft SDKs\Azure\PowerShell\ServiceManagement\Azure\Azure.psd1
2016-05-08T06:24:49.9958542Z AzurePSCmdletsVersion= 1.4.0
2016-05-08T06:24:50.0583559Z Get-ServiceEndpoint -Name ******** -Context Microsoft.TeamFoundation.DistributedTask.Agent.Worker.Common.TaskContext
2016-05-08T06:24:50.0739805Z tenantId= ********
2016-05-08T06:24:50.0739805Z azureSubscriptionId= ********
2016-05-08T06:24:50.0739805Z azureSubscriptionName= ********
2016-05-08T06:24:50.2146097Z Add-AzureRMAccount -ServicePrincipal -Tenant ******** -Credential System.Management.Automation.PSCredential
2016-05-08T06:24:50.7982308Z Select-AzureRMSubscription -SubscriptionId ******** -tenantId ********
2016-05-08T06:24:51.0235517Z ##[debug]Loading AzureUtilityGTE1.1.0.ps1
2016-05-08T06:24:57.3994077Z ##[error]Cannot bind argument to parameter 'storageKey' because it is an empty string.

Raw Task Logs with Azure PS 1.3.2

2016-05-08T06:56:10.0177511Z Executing the powershell script: C:\Agent\tasks\AzureFileCopy\1.0.52\AzureFileCopy.ps1
2016-05-08T06:56:10.2052547Z Looking for Azure PowerShell module at C:\Program Files (x86)\Microsoft SDKs\Azure\PowerShell\ServiceManagement\Azure\Azure.psd1
2016-05-08T06:56:11.0646512Z AzurePSCmdletsVersion= 1.3.2
2016-05-08T06:56:11.1271528Z Get-ServiceEndpoint -Name ******** -Context Microsoft.TeamFoundation.DistributedTask.Agent.Worker.Common.TaskContext
2016-05-08T06:56:11.1427777Z tenantId= ********
2016-05-08T06:56:11.1427777Z azureSubscriptionId= ********
2016-05-08T06:56:11.1427777Z azureSubscriptionName= ********
2016-05-08T06:56:11.2834062Z Add-AzureRMAccount -ServicePrincipal -Tenant ******** -Credential System.Management.Automation.PSCredential
2016-05-08T06:56:11.7897892Z Select-AzureRMSubscription -SubscriptionId ******** -tenantId ********
2016-05-08T06:56:11.9616688Z ##[debug]Loading AzureUtilityGTE1.1.0.ps1
2016-05-08T06:56:14.5176791Z Uploading files from source path: 'D:\Agent\Work\326ad65b5\********.Backend.CI\Backend\artifacts\package.zip' to storage account: '********' in container: '********' with blobprefix: 'deployments/********'
2016-05-08T06:56:16.7210196Z Uploaded files successfully from source path: 'D:\Agent\Work\326ad65b5\********.Backend.CI\Backend\artifacts\package.zip' to storage account: '********' in container: '********' with blobprefix: 'deployments/********'

@kmkumaran kmkumaran added the deployment label May 9, 2016

@kmkumaran

This comment has been minimized.

Copy link
Member

kmkumaran commented May 9, 2016

Azure File Copy task has not been signed off yet with Azure SDK 1.4.0. You can refer the "Supported Azure and Azure RM module versions" section in ReadME https://github.com/Microsoft/vsts-tasks/blob/master/Tasks/AzureFileCopy/README.md

@kmkumaran kmkumaran self-assigned this May 9, 2016

@antmeehan

This comment has been minimized.

Copy link
Contributor

antmeehan commented May 10, 2016

@kmkumaran Do you have an idea when it may be ready? We are very keen to move to Azure PS 1.4.0 to fix Azure/azure-powershell#2120

@kmkumaran

This comment has been minimized.

Copy link
Member

kmkumaran commented May 12, 2016

We have the sing-off planned sometime next sprint. I will post back to this thread once we have it ready.

@jirisykora83

This comment has been minimized.

Copy link

jirisykora83 commented Jun 1, 2016

Do you have any news about this issue?

@antmeehan

This comment has been minimized.

Copy link
Contributor

antmeehan commented Jun 2, 2016

While waiting for the fix, we've manually patched AzureFileCopy\1.0.54\AzureUtilityGTE1.0.ps1 as follows:

function Get-AzureStorageKeyFromARM
{
    param([string]$storageAccountName)

    if (-not [string]::IsNullOrEmpty($storageAccountName))
    {
        # get azure storage account resource group name
        $azureResourceGroupName = Get-AzureStorageAccountResourceGroupName -storageAccountName $storageAccountName

        Write-Verbose "[Azure Call]Retrieving storage key for the storage account: $storageAccount in resource group: $azureResourceGroupName"
        $storageKeyDetails = Get-AzureRMStorageAccountKey -ResourceGroupName $azureResourceGroupName -Name $storageAccountName -ErrorAction Stop
        $storageKey = $storageKeyDetails[0].Value
        Write-Verbose "[Azure Call]Retrieved storage key successfully for the storage account: $storageAccount in resource group: $azureResourceGroupName"

        return $storageKey
    }
}

The line:

        $storageKey = $storageKeyDetails[0].Value

is what has been changed.

@Jacob-McKay

This comment has been minimized.

Copy link

Jacob-McKay commented Jul 8, 2016

@antmeehan I have no clue how you figured that out, but seems to have worked for me too...

@nelsonmorais

This comment has been minimized.

Copy link

nelsonmorais commented Jul 21, 2016

This fix also worked for me but my ps1 file was located in tasks\AzureFileCopy\1.0.62 (more recent version I presume), so I guess the reported problem is not fixed yet.

Can someone tell me how the tasks that are present on the agent folder are updated and when? I don't see them in the .zip agent that is downloaded from VSTS.

@antmeehan

This comment has been minimized.

Copy link
Contributor

antmeehan commented Jul 22, 2016

@nelsonmorais The agent must ping VSTS periodically and download the latest agent tasks, or when starting a task the version number of the task might be specified (and it downloads if it doesn't have them).

Luckily it doesn't do a checksum or anything to make sure the source hasn't been modified :)

[I don't work for MS, I'm just guessing based on observed behaviour]

@nelsonmorais

This comment has been minimized.

Copy link

nelsonmorais commented Jul 22, 2016

@antmeehan Thank you for your comment.
@kmkumaran Maybe someone from MS could also clarify how the process works, and also give more information when this problem will be fixed

@nelsonmorais

This comment has been minimized.

Copy link

nelsonmorais commented Jul 29, 2016

I got this problem again, now with the task version 1.0.69
@kmkumaran Any plans to have this fixed?

@edtheshed

This comment has been minimized.

Copy link

edtheshed commented Aug 9, 2016

I also have this issue now with 1.0.69. Surely this shouldn't be closed...

@vincentdass vincentdass assigned vincentdass and unassigned kmkumaran Aug 18, 2016

@vincentdass vincentdass reopened this Aug 18, 2016

vincentdass added a commit that referenced this issue Aug 18, 2016

vincentdass added a commit that referenced this issue Aug 18, 2016

@vincentdass

This comment has been minimized.

Copy link
Member

vincentdass commented Aug 19, 2016

Latest Supported Azure Module version is 1.3.2. [https://github.com/Microsoft/vsts-tasks/blob/master/Tasks/AzureFileCopy/README.md#supported-azure-and-azurerm-module-versions]
You can download the Azure Module version 1.3.2 for fix. This bug will be resolved in upcoming sprint.

@stack111

This comment has been minimized.

Copy link

stack111 commented Aug 24, 2016

@vincentdass Do you have any date to when this will be available? When will 2.x be supported?

@vincentdass

This comment has been minimized.

Copy link
Member

vincentdass commented Sep 1, 2016

@stack111 , It's currently in progress. Will be updating the status soon.

@stevematdavies

This comment has been minimized.

Copy link

stevematdavies commented Aug 25, 2017

I don't fully get this, were not using any manual commands, we are running inside VSTS builds thus and deploying directly to Azure, but all of a sudden our deployment task has failed with this nonsensical error. We have been using the same build definitions for months without change, but just today we are receiving:

Cannot bind argument to parameter 'files' because it is an empty string.

Id like to know (clearly and simply) what has changed in Azure for this to happen. Our Azure admin says nothing has changed. We are only deploying from VSTS on a hook when a repo is merged. So nothing to do with build scripts etc:

Anyone can offer a plain English solution?

@stack111

This comment has been minimized.

Copy link

stack111 commented Aug 26, 2017

this the last I remember on this thread since it is so old. I believe these VSTS tasks depend on Azure Poweshell and in our agents some of them had higher version(s) of AzurePs installed and the task was not compatible with the higher version.

I hope that helps provide a starting point to investigate.

@arjgupta

This comment has been minimized.

Copy link
Member

arjgupta commented Aug 28, 2017

Hi @stevematdavies , please send the debug logs to RM_Customer_Queries@microsoft.com . Also, are you using a Hosted Agent? If yes, then which Pool? "Hosted" or "Hosted VS2017" . If not, then can you tell what is the version of Azure PowerShell on your private agent.

@KrishnaAdityaB

This comment has been minimized.

Copy link
Contributor

KrishnaAdityaB commented Aug 30, 2017

@mkusmiy

This comment has been minimized.

Copy link

mkusmiy commented Sep 21, 2018

After upgrading AzureRM from v5.7.0 to v6.8.1 we ran into this very problem with Azure File copy task in our TFS. Issue can be fixing by merging following line in the very same file from v2 of the task:
$azureStorageAccountResourceDetails = (Get-AzureRMResource -ErrorAction Stop) | Where-Object { ($.ResourceType -eq $ARMStorageAccountResourceType) -and ($.Name -eq $storageAccountName)}
https://github.com/Microsoft/vsts-tasks/blob/master/Tasks/AzureFileCopyV2/AzureUtilityGTE1.0.ps1

@soccerjoshj07

This comment has been minimized.

Copy link

soccerjoshj07 commented Oct 31, 2018

@vincentdass This shouldn't be closed - still an issue in Azure DevOps pipelines as @mkusmiy alluded to.

A fix that worked for me is by modifying line 27 of AzureUtilityGTE1.0.ps1 (@mkusmiy , you had a small typo in your snippet above, missing the "_" in $.ResourceType and $.Name)

from
$azureStorageAccountResourceDetails = (Get-AzureRMResource -ErrorAction Stop) | Where-Object { ($_.ResourceType -eq $ARMStorageAccountResourceType) -and ($_.ResourceName -eq $storageAccountName)}

to
$azureStorageAccountResourceDetails = (Get-AzureRMResource -ErrorAction Stop) | Where-Object { ($_.ResourceType -eq $ARMStorageAccountResourceType) -and ($_.Name -eq $storageAccountName)}

Edit: V2 of the task (in preview) also worked.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment