Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using pipeline variables in Azure CLI task #9416

Closed
erikcartman opened this issue Jan 25, 2019 · 23 comments

Comments

Projects
None yet
@erikcartman
Copy link

commented Jan 25, 2019

Hello, in Azure Devops pipeline i am using Azure CLI task with inline script on Hosted Ubuntu 1604 agent, is there any way to pass pipeline variables to scipt? as example i want to use this command with variable az resource list -g MYVARIABLE --query "[].name" -o tsv? if so could you provide example. Thanks.

@erikcartman erikcartman changed the title Use pipeline variables in Azure CLI Using pipeline variables in Azure CLI task Jan 25, 2019

@FireDrunk

This comment has been minimized.

Copy link

commented Jan 28, 2019

az storage account keys list --account-name $(accountName)
Works fine?

@dikhakha

This comment has been minimized.

Copy link
Member

commented Jan 29, 2019

You can use pipeline variables in the script by passing them as arguments. Example:
Arguments input : $(MYVARIABLE)
Inline script: az resource list -g $1 --query "[].name" -o tsv

where MYVARIABLE is your pipeline variable. Please let me know if this helps.

@dikhakha

This comment has been minimized.

Copy link
Member

commented Feb 1, 2019

@erikcartman Did you get a chance to try out the above suggestion ?

@erikcartman

This comment has been minimized.

Copy link
Author

commented Feb 2, 2019

nope $(mypipelinevariable) didn't worked for me. I changed my approach, but thanks anyway

@erikcartman erikcartman closed this Feb 2, 2019

@tomasaschan

This comment has been minimized.

Copy link

commented Feb 5, 2019

@dikhakha FWIW, I had a working inline script that made use of pipeline variables that worked fine. When I moved the script, unchanged, to a file in the source repository, exposed it as a build artifact and reconfigured the Azure CLI task to use a script path instead, it didn't do variable substitution anymore.

The script uses some 20-ish variables (most of them to set app settings) so I would really like to avoid having to specify them all as script arguments.

@kmkumaran kmkumaran reopened this Feb 6, 2019

@dikhakha

This comment has been minimized.

Copy link
Member

commented Feb 6, 2019

@tomasaschan you should be able to access the pipeline variables in the script as well. All pipeline variables are made available to the script as environment variables, except secret variables. I am wondering if you changed the Job Agent by any chance ? This is because variable names are converted to UPPER CASE and stored so if you are using Ubuntu agent, you might not be getting the values due to case sensitivity. Can you please tell me which job agent are you using ?

@tomasaschan

This comment has been minimized.

Copy link

commented Feb 6, 2019

@dikhakha It's running on Ubuntu 16.04. The variables have names like $(WebApp.Name) - what would that translate to as an environment variable name?

Also, a majority of the variables are imported from Azure KeyVault. They are not marked as secret in the pipeline variables configuration, but they do get masked when included in log outputs. Are they made available as well?

@dikhakha

This comment has been minimized.

Copy link
Member

commented Feb 6, 2019

WebApp.Name will be available as WEBAPP_NAME (' ' and '.' is converted to '_'). You can refer to this doc about custom variables.

I did not completely understand the second scenario. If a variable is not marked as secret, it will be available. however, it could be getting masked in the logs because those variables might contain values which are considered as secret like your Azure service connection details.

@tomasaschan

This comment has been minimized.

Copy link

commented Feb 6, 2019

The second scenario is basically this:

I have a variable group that is linked to an Azure KeyVault, and the secrets from there are imported as variables. I then have pipeline variables that refer to the variables from the variable group, in order to map the secret names in Key Vault to something that makes sense for the pipeline, and to handle scopes.

So, for example, say the Key Vault has two secrets; foo-bar-test and foo-bar-prod. They are made available under the same names in the variable group. In the pipeline, I have a pipeline variable Foo.Bar with the value $(foo-bar-test) scoped to Test, and Foo.Bar with the value $(foo-bar-prod) scoped to Prod. In the pipeline variable configuration, Foo.Bar is not marked secret.

So, given the following statement:

All pipeline variables are made available to the script as environment variables, except secret variables.

the question was whether I will have a FOO_BAR environment variable available in the script, or if being imported from Key Vault counts as being a "secret variable".

@dikhakha

This comment has been minimized.

Copy link
Member

commented Feb 6, 2019

You can refer to this doc for using variables imported from KeyVault. Secret variables and variables from key vault are considered as secrets therefore they will not be available in the scripts.

@dikhakha

This comment has been minimized.

Copy link
Member

commented Feb 12, 2019

Closing this issue. Please feel free to reopen if you need more help.

@dikhakha dikhakha closed this Feb 12, 2019

@mydiemho

This comment has been minimized.

Copy link
Member

commented Mar 8, 2019

You can use pipeline variables in the script by passing them as arguments. Example:
Arguments input : $(MYVARIABLE)
Inline script: az resource list -g $1 --query "[].name" -o tsv

where MYVARIABLE is your pipeline variable. Please let me know if this helps.

I probably misunderstood you but your instruction didn't work when I try to add the variable in the Arguments sections. It did work when I reference it directly when calling the script

clitaskvariable

@sachinma sachinma reopened this Mar 8, 2019

@bishal-pdMSFT

This comment has been minimized.

Copy link
Contributor

commented Mar 8, 2019

@mydiemho in the task UI, you should use the same name as pipeline variable name(without changing case and '.' etc).
Only when accessing the variables as environment variable inside a script you should transform the name.

@uniquelau

This comment has been minimized.

Copy link

commented Mar 9, 2019

I'd like to add that I'd also experienced some issues with variables in inline scripts.
I am seeing unreliable behaviour in one pipeline out of many that I have logged here:
Azure/azure-cli#8722 (comment)

@mydiemho

This comment has been minimized.

Copy link
Member

commented Mar 11, 2019

@mydiemho in the task UI, you should use the same name as pipeline variable name(without changing case and '.' etc).
Only when accessing the variables as environment variable inside a script you should transform the name.

I am using the same case. My variable name is all capitalize

@issacnitin issacnitin assigned issacnitin and unassigned issacnitin Mar 12, 2019

@issacnitin

This comment has been minimized.

Copy link
Member

commented Mar 12, 2019

@mydiemho I stumbled upon something similar, it happened because of trying to reference argument by $1 while running the task in a windows machine. Could you confirm if it's the same case with you?
In windows it should work when you use %1 to reference the argument supplied to Inline script.

@mydiemho

This comment has been minimized.

Copy link
Member

commented Mar 12, 2019

@mydiemho I stumbled upon something similar, it happened because of trying to reference argument by $1 while running the task in a windows machine. Could you confirm if it's the same case with you?
In windows it should work when you use %1 to reference the argument supplied to Inline script.

I am running this on a Linux agent.

@issacnitin

This comment has been minimized.

Copy link
Member

commented Mar 13, 2019

@mydiemho Could you share the logs with system.debug variable set to true? You can share them here or at RM_Customer_Queries@microsoft.com

@mydiemho

This comment has been minimized.

@issacnitin

This comment has been minimized.

Copy link
Member

commented Mar 14, 2019

@mydiemho Looks like the account value is getting printed correctly when $1 is used. The issue is with azcopy asking for user input when script is ran. You could use --quiet argument along with azcopy to suppress user inputs. Let us know in case of more queries!

@mydiemho

This comment has been minimized.

Copy link
Member

commented Mar 18, 2019

@issacnitin I don't think we're talking about the same thing. I know $1 works (but only when I pass the variable to the script). $1 doesn't work when I set it in the Arguments sections

@issacnitin

This comment has been minimized.

Copy link
Member

commented Mar 21, 2019

@mydiemho

From my understanding, you're referring to $1 in your script.
If that's the case then you need to explicitly pass an argument to your script if "Inline" mode is used. You can do that by:

  1. Directly passing $(AZURE_STORAGE_ACCOUNT) in Inline script,
    OR
  2. Passing $(AZURE_STORAGE_ACCOUNT) in the Arguments section and then passing reference further on to your script using $1

So your script would look like:

./scripts/push-report.sh $1 $2
Arguments: $(AZURE_STORAGE_ACCOUNT) $(AZURE_STORAGE_KEY)

Alternately, you can use "Script Path" mode and avoid passing argument explicitly.

Please refer to the images below, these are the working versions of the task:

Passing directly in Inline script
2

Passing through Arguments in Inline script
1

Using Script Path
3

Let me know if this solves your problem. If you're having more trouble, could you post the release logs for the failure case (with system.debug mode set to True)? And if possible, do share a screenshot of your task as well.

Thanks!

@amaljg

This comment has been minimized.

Copy link
Member

commented Apr 4, 2019

@mydiemho - I assume your issue is resolved using @issacnitin's suggestions above. If you still face the issue, feel free to reopen the issue or open a new one.

@amaljg amaljg closed this Apr 4, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.