New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Secret passed in to Bash task as argument only works with $(MYSECRET) not $MYSECRET #9363

Closed
GrahamDSmith opened this Issue Jan 19, 2019 · 15 comments

Comments

Projects
None yet
4 participants
@GrahamDSmith
Copy link

GrahamDSmith commented Jan 19, 2019

In Azure Pipelines I can pass normal variables to a v3 Bash script (Shell Script task) as an argument using $MYVARIABLE. However, secrets (which can only be passed in as arguments) do not get passed in using the $MYSECRET form but they do get passed in using the $(MYSECRET) form. I've not found this documented anywhere. Is this a bug?

@bryanmacfarlane

This comment has been minimized.

Copy link
Member

bryanmacfarlane commented Jan 23, 2019

You have to map secrets into the env block of the step. @ericsciple can point to example

@GrahamDSmith

This comment has been minimized.

Copy link
Author

GrahamDSmith commented Jan 23, 2019

Thanks @bryanmacfarlane. Just to clarify, I have this working fine, however the issue is that I've found secrets need to be passed to a script with parentheses round the word, ie $(MYSECRET), as opposed to anything that isn't a secret which can be passed without parentheses around the word ie $MYPLAINVARIABLE. I haven't seen this need for the parentheses for secrets documented anywhere and am wondering if it's a bug because I'm sure it will trip some people up.

@ericsciple

This comment has been minimized.

Copy link
Member

ericsciple commented Jan 23, 2019

That is intentional. Secrets are not automatically added into the environment block. They must be explicitly mapped in. Either into the env block, or into one of the task inputs. Examples are here.

@GrahamDSmith

This comment has been minimized.

Copy link
Author

GrahamDSmith commented Jan 23, 2019

Thanks @ericsciple , but that's not the issue. Please see previous comments.

@ericsciple

This comment has been minimized.

Copy link
Member

ericsciple commented Jan 24, 2019

@GrahamDSmith what's the issue? The behavior you are describing aligns with the documentation I linked.

@GrahamDSmith

This comment has been minimized.

Copy link
Author

GrahamDSmith commented Jan 29, 2019

The problem @ericsciple is one of consistency. If I can pass non-secrets to a script using $MYPLAINVARIABLE then for consistency I should be able to pass secrets using $MYSECRET. This lack of consistency will trip some people up which is what I'm trying to address. So the fix is one of these:

  • Allow secrets to be passed in as $MYSECRET to be consistent with $MYPLAINVARIABLE
  • Force all arguments to be passed in as $(MYPLAINVARIABLE) to be consistent with $(MYSECRET)
  • Make the documentation very clear that secrets have to be passed in to a script in a very specific way, ie $(MYSECRET) works but $MYSECRET doesn't.

The second option is obviously a breaking change so one or three would be preferable.

Cheers!

@dylan-smith

This comment has been minimized.

Copy link
Member

dylan-smith commented Jan 30, 2019

Are you sure it's doing what you think. When you pass in a non-secret using the $MYPLAINVARIABLE i suspect that it's just getting passed to your task as a string (AzDO is not converting it to the value), then in the script $MYPLAINVARIABLE resolves to the environment variable so things work.

I wouldn't expect AzDO to process anything that doesn't have the $() syntax.

@GrahamDSmith

This comment has been minimized.

Copy link
Author

GrahamDSmith commented Jan 30, 2019

Hi @dylan-smith

Thanks for the reply. I've tested this as follows:

  • New variable created as Name = Graham, Value = Smith
  • the Graham variable is passed to the bash script as a second parameter as $GRAHAM
  • In the script I have echo "TEST is $2"
  • In the log output I have TEST is Smith

I'm fairly new to Bash scripts and passing parameters to them in Azure DevOps but this does feel like the $MYPLAINVARIABLE syntax is (unexpectedly!) working or have I missed something?

Cheers - Graham

@dylan-smith

This comment has been minimized.

Copy link
Member

dylan-smith commented Jan 30, 2019

I think bash is just being too smart. Here's what I see:

I created a build with 2 variables $username and $foo (values of dylan and blah). I have a bash task that looks like this:
image

test.sh is:

echo "USERNAME is $1"
echo "FOO is $2"

It prints out:

USERNAME is dylan
FOO is blah

Which is the same behavior you are seeing, but if you look at the build log more closely you will see this:

Generating script.
[command]"C:\Program Files\Git\bin\bash.exe" --noprofile --norc -c pwd
/d/a/1/s
Formatted command: . '/d/a/1/s/test.sh' $USERNAME blah

so it really is passing in "$USERNAME" as a literal string, and bash is just doing something "smart" with it

@GrahamDSmith

This comment has been minimized.

Copy link
Author

GrahamDSmith commented Jan 31, 2019

What happens if test.sh is:

echo "TEST is $1"

ie USERNAME is not used in the script?

Whatever is happening though, the net effect for end users is the same, ie a lack in consistency which will trip some people up.

@ericsciple

This comment has been minimized.

Copy link
Member

ericsciple commented Feb 5, 2019

This section contains a table which describes where the azure-pipelines macro syntax $( ) can be used, and describes alternate syntaxes from accessing environment variables within a script (batch, powershell, and bash).

And this section talks about working with secret variables which intentionally must by mapped-in using macro syntax. You are correct it's inconsistent, however we intentionally chose that behavior to help prevent accidental leaking of secrets (crash dumps, etc).

I think the confusion is, the filename + arguments that you specify are written to a temporary script. And bash is used to run that command. So when you are using bash variable-syntax (e.g. $FOO) in the arguments input, those arguments are getting interpreted by bash. Many customers desire this behavior and it makes the task more consistent with what they type on the command line.

If you have any specific feedback for the docs, I'm happy to pass it along.

@GrahamDSmith

This comment has been minimized.

Copy link
Author

GrahamDSmith commented Feb 5, 2019

Not sure if this is specific feedback, but I think the whole scenario of passing secrets to Bash scripts via arguments needs to be explained explicitly and clearly, and be easy to stumble upon.

@ericsciple

This comment has been minimized.

Copy link
Member

ericsciple commented Feb 6, 2019

Thanks, makes sense. Personally I agree, as I've seen it be a very common stumbling point for customers in the past.

I'll relay the feedback and follow up with appropriate folks tomorrow.

@ericsciple ericsciple closed this Feb 6, 2019

@ericsciple

This comment has been minimized.

Copy link
Member

ericsciple commented Feb 6, 2019

@ericsciple

This comment has been minimized.

Copy link
Member

ericsciple commented Feb 6, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment