-
Notifications
You must be signed in to change notification settings - Fork 927
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support conditions for templates #30
Comments
conditions are evaluated at runtime, not during plan compilation. I dont think we should mix the two |
Either the template should be taught the right condition (most common case?) or it should be parameterized if the user of the template needs to be in control. |
@ericsciple, Are you saying that a -template is effectively a #include of the underlying template into the existing YAML file? If so then why, on the original issue, did you say the syntax (I'm trying to use below) is supported? @vtbassmatt, Why was this issue closed when the original issue this was based upon (before the migration) is still valid and still not working? What do you mean by "taught the right condition"? Based upon your second statement of "parameterized" it seems like you're trying to suggest that a template should have some sort of "enabled" parameter that should be set by the code. That doesn't make sense to me because none of the existing tasks do this because they support the condition clause. The ask on this issue is that we can use condition on a template. Why is this different than an existing task? Even if it currently doesn't support it, the recommended solution of using a template expression isn't working for people, myself included. When I try this (from Eric's original solution under the original issue). variables:
someVar: true
steps:
- ${{ if ne(variables['someVar'], 'true') }}:
- template: mytemplate.yml I get the error: |
what does your template look like? |
When I run this: pool: hosted ubuntu 1604
variables:
someVar: true
steps:
- ${{ if ne(variables['someVar'], '') }}:
- template: mytemplate.yml
- script: echo hi I don't get any error. I think the error about the directive not allowed must be coming from your template file. I would expect the error to include the file name and line and column. Does it not? |
@ericsciple I think you're right. I have been updating the templates to use the inline template expression stuff and I finished updating them all. I had temporarily commented out that expression to get past it. I added it back and I don't get the error anymore. I cannot confirm it is working as expected yet because the build is failing in an earlier step but it queues now. Thanks. |
Think of template expansion like a Jinja or Handlebars template (though it doesn't operate on pure text - it operates on the DOM of the YAML file). The conditional-inclusion stuff there is Both of them support roughly the same capabilities in the expression language. If we're missing something there, let us know so we can schedule it. |
@vtbassmatt , the template expression stuff is mostly hacky to me. I would argue that the goal should be to be able to copy/paste as much existing logic from a YAML file to a template and it still work. As such I would expect to be able to reference parameters inside a condition just like I can reference variables and env vars. I don't believe templates currently support variable declarations but if they did I would expect to be able to use them just like any other non-template YAML file. As a user of YAML I shouldn't have to be concerned with how the parser is implemented just to use it for "normal" things. My focus should be on getting my standalone scripts running inside the engine. That is hard enough as it is without the added complexity of figuring out the rules of the YAML parser. |
guys, i seem to encounter a bit of an issue with a condition, which i tried setting in multiple ways :) my current approach on setting it on a template yaml reference is as you said above:
indentation seems to be fine, the template line is one tab more to the right that the condition line and the condition lines starts at the same indentation as the 'jobs' line the thing is that i don't get any error, but it doesn't execute either, although i'm sure that i do have a variable 'TestRun' (settable at queue time, but it has a default value on it) // does it influence that this variable is declared in the UI side of the Build Definition and not in the YAML (i know that some time ago you couldn't declare in the yaml that the variable should be settable at queue time) ? i tried in a few other ways with basically the same main expression 'in(variables['TestRun'], 'SmokeTests', 'AllTests')', but the above seemed to get me most far :) only that it is always false :) btw, is it normal behavior that if the condition set a on a template reference evaluates to false to not even see that phase in the Build Execution ? (as opposed to the message 'skipped' that i can see when a condition is not met on a specific step) appreciate any kind of help, thank you! |
@carct
Template expansion is just a DOM-manipulation step. Think of text templating like Jinja or C preprocessor macros, slightly enlightened about YAML structures so that you don't have to worry as much about spacing. It's not aware of most of the other features of the YAML syntax, including [edit] user-provided variables. It only considers |
@vtbassmatt it is a bit confusing: "variables aren't visible at template expansion time" also, your line is again confusing as I successfully manage to pass parameters values to the template with the help of variables another thing that i did notice while working with these YAML Templates is that inside a template i used in the condition of a step the following line "condition: eq(variables['myVar'], 'true')", nothing weird until now, just that 'myVar' is actually a parameter of the template and does not exist as a Var, so again.. a bit confusing :) coming back to my issue, what would your suggestion be regarding it ? thanks for your time ! LATER EDIT: LATER EDIT x2:
and of course that i had to convert my template YAML to be a steps template and not a jobs one as it was initially in the end it's not so bad or impossible to do, but proper documentation is needed as i had to do a lot of tryouts to finally make it do my bidding :) and i don't consider my use-case so niched, i just wanted to execute a template based on a truly simple condition that uses a variable in it... |
Hey, sorry, I was wrong about what's in scope in the various steps. Can you paste in your whole YAML (or if not comfortable sharing here, you can email me: mattc at xbox dot com)? |
well, i did manage to make it work as per my latest edit to the post the only thing is that it was a bit confusing and to be frank with you, i would like to still be able to set such a condition on Jobs Template
but unable to actually use some variables in that if execution condition :) in my current case, i can be ok with the conversion to a Steps Template, but in other future cases i might not be so keen on that :)) if you think that we can get around the discussed limitations, we can take this to private, but from what i understood from you this is kinda' the way to do it... |
Thanks. This is something we'll consider for a future enhancement. I don't know if there's a good reason why variables aren't available in that context, or if it's just an implementation quirk. |
@vtbassmatt, do predefined build variables available here? I would like to do something like this. pool: hosted ubuntu 1604
variables:
someVar: true
steps:
- ${{ if in(variables['Build.Reason'], 'Manual', 'Schedule')) }}:
- template: mytemplate.yml
- script: echo hi I am getting the following error:
For the completeness; the Thank you! Edit: I have updated the error message. Edit2: Someone else has already reported this (or similar) but the original GH issue has been closed. |
I am a bit puzzled; the example for the conditional templates
pool: hosted ubuntu 1604
variables:
someVar: true
steps:
- script: echo someVar is $(someVar)
displayName: 'Check someVar Script'
- script: 'echo someVar is $(someVar) in a conditional script'
condition: ne(variables['someVar'], '')
displayName: 'Conditional Script (Should run)'
- script: 'echo someVar is $(someVar) in a conditional script'
condition: eq(variables['someVar'], '')
displayName: 'Conditional Script (Should NOT run)'
- ${{ if ne(variables['someVar'], '') }}:
- template: mytemplate.yml
- script: echo hi
displayName: 'Always Run Script'
steps:
- script: echo from mytemplate.yml
displayName: 'Template Script' And the build steps from the template are not executed: Can someone please help me figure out what's wrong with my build? Thank you! |
There is another problem, perhaps this could go to a separate GH issue. Let assume, I would like to run different jobs when the build is manually triggered or scheduled (CRON) than when it is a rolling (individual commits/pushes plus the batched ones). Formally:
Since the conditions ( I ended up having the following configuration:
jobs:
- job: rolling_macOS
displayName: 'Rolling Build (macOS)'
pool:
vmImage: 'macOS-10.13'
condition: notIn(variables['Build.Reason'], 'Manual', 'Schedule'))
steps:
- template: rolling-build-steps.yml
- job: release_macOS
displayName: 'Release Build (macOS)'
pool:
vmImage: 'macOS-10.13'
condition: in(variables['Build.Reason'], 'Manual', 'Schedule'))
steps:
- template: release-build-steps.yml
- job: release_windows
displayName: 'Release Build (Windows)'
pool:
vmImage: 'vs2017-win2016'
condition: in(variables['Build.Reason'], 'Manual', 'Schedule'))
steps:
- template: release-build-steps.yml
steps:
- script: echo release build
displayName: 'Release Build Script'
steps:
- script: echo rolling build
displayName: 'Rolling Build Script' Error:
A working solution 🎉: Transform the
jobs:
- job: rolling_macOS
displayName: 'Rolling Build (macOS)'
pool:
vmImage: 'macOS-10.13'
condition: and(ne(variables['Build.Reason'], 'Manual'), ne(variables['Build.Reason'], 'Schedule'))
steps:
- template: rolling-build-steps.yml
- job: release_macOS
displayName: 'Release Build (macOS)'
pool:
vmImage: 'macOS-10.13'
condition: or(eq(variables['Build.Reason'], 'Manual'), eq(variables['Build.Reason'], 'Schedule'))
steps:
- template: release-build-steps.yml
- job: release_windows
displayName: 'Release Build (Windows)'
pool:
vmImage: 'vs2017-win2016'
condition: or(eq(variables['Build.Reason'], 'Manual'), eq(variables['Build.Reason'], 'Schedule'))
steps:
- template: release-build-steps.yml |
@kittaakos the error about the ) is correct. You have a single paren on the left and 2 parens on the right so it is invalid. Try this. steps:
- ${{ if in(variables['Build.Reason'], 'Manual', 'Schedule') }}: And in last post you have the same issue. condition: notIn(variables['Build.Reason'], 'Manual', 'Schedule') The parens are for the notIn function. You have an extra paren that isn't needed. For the template expression around your template the syntax looks correct. I have a similar expression and it is working fine. I have noticed that YAML is picky about spacing though. Indentation is important and the docs make it clear that you should avoid tabs. So I would recommend that you remove all the whitespace from the template line and then add enough spaces until the - lines up with the $ above it. I'm not sure it is the cause of the issue here but I've had problems with indentation in my YAML. |
This. Sorry for the noise, I was blind. |
So, according to this comment system variables work. That means (as I understand it) that only the strict subset of system variables can be used. I created another issue to request support for agent variables too, which would be quite helpful IMHO. For reference, my non-working template: template:
pipeline:
Results in: Not working (there's no install/provision step there, and the powershell step renders |
Clarifies for others facing microsoft/azure-pipelines-yaml#30 or microsoft/azure-pipelines-yaml#297
You will need to add two steps to the pipe and provide a "condition":
And then inside your template:
I know it's not elegant and it will always show one step as skipped, but it's the only thing I could get working. By the way, you can also use |
Hey @denravonska, thanks for your suggestion but unfortunately no luck on this Pipeline run start errors:
This would essentially explain why: Thanks for your help regardless, I will place the task step for failure directly into my pipeline for now. |
Strange, I use that very setup myself:
|
@denravonska A possibility could be that the template I am consuming is in a different repository from the Primary pipeline? |
@sl-NZ I don't think it matters. If I understand it correctly the template parameter expansion just pastes whatever is passed to it into where it's used. So the final, expanded results should be
Did you put quotes around the condition?
instead of:
? Not sure if it matters though. |
@sl-NZ template expressions are evaluated at "compilation" time and not while the pipeline is running. To me they are glorified #if statements. This diagram shows the process. If you want to change the parameters passed to a task based upon parameters at compile time then you can do that with a single step and use template expressions to conditionally include/exclude values. Something like this (not tested). - template: ../SlackMessageTasks/send-slack-message.yml@templates
parameters:
slack_message: ':x-cross: Deploy to `${{ parameters.test_env_num }}` failed for ClientLogin, Build `$(Build.BuildNumber)`, Branch `$(Build.SourceBranch)` for `$(Build.RequestedFor)`'
SlackChannelConfig: 'test-environments-$(EnvNum)'
- ${{ if ne(variables['Agent.JobStatus'], 'Failed') }}:
slack_step_tite: 'Success${{ parameters.test_env_num }}'
- ${{ if eq(variables['Agent.JobStatus'], 'Failed') }}:
slack_step_tite: 'Failed ${{ parameters.test_env_num }}' In my experience if you have only a single value to swap then it isn't bad. If the task itself changes you need multiple steps. Alternatively I tend to prefer variables when possible (templates make this hard). So instead of using template expressions everywhere I'll define a variable (if possible) or step(s) that set env vars that later steps just use directly. So, in your example, I'd probably use a variable for the messages and then have a single template expression above this to set the values I want to use in the task. Again, this isn't always possible. Templates in particular don't currently allow variables to be declared so I tend to use env variables (ugly but it works). If you only need the values once though then inline template expressions are easier. |
@CoolDadTx That won't work as the job status is not available at template compile time. He needs to pass an evaluation expression to the template, as in my example, so the task is able to evaluate and conditionally run or skip itself at runtime. |
@denravonska I wasn't focusing on the expression itself. I was addressing the posters comment about when template expressions are evaluated and the need for multiple steps. For the very specific case of checking the status of the agent then condition is fine. |
@CoolDadTx No, your example is faulty :) You cannot do a static expression on a dynamic condition. |
@denravonska I don't know what you mean by dynamic condition but template expressions can reference any values that can be determine at the point the template is "compiled". That includes build variables and template parameters. We do this heavily in our highly customized YAML builds and we have no problems whatsoever. Of course getting all this debugged and working was entirely different story but you can use some variables like build variables in template expressions. Variables whose values change during the build won't work correctly however so you have to switch to env variables (or perhaps YAML variables). Nevertheless your experience may be different than mine and what works for our builds might not work for yours. This is one of the currently annoying features of YAML in that it is very finicky. Even in our own templates we have found that certain expressions or syntax works fine in one template but not in another. Although most of our issues are related to how the PS tasks are generated. Still I've shared my feedback on this issue and I'm not going to argue with you about what does and doesn't work. This is a closed topic anyway. The OP should open a new one if they are having issues with template expressions. |
how about just a 'condition:' line under "- template: xxxxx' ?? |
Hi Just a update from my experience which is working smoothly for me now.
=========== Pipeline Definition =============
=========== YML Template =============
|
Just want to point out to folks reaching this point, this comment here was very helpful for me. @vtbassmatt is your doc available somewhere? User-defined variables are part of the runtime execution right? |
@spygi the doc was published a while ago here: https://docs.microsoft.com/azure/devops/pipelines/process/runs. Sorry that wasn't clear! |
Hi @CoolDadTx, I'm trying to use sth like that:
I would like to be able to pass extraparam to template as parameter when var2==True. Thanks, |
@andrzej-jedrzejewski You're trying to apply a condition on a parameter to a template. We don't do that so I cannot confirm it'll work. Personally I don't think it makes sense to conditionally include a parameter or not because the template defines the parameters it needs anyway. So making it conditional when calling it doesn't change the template itself. What we do is just set the parameter to a value. If the parameter cannot have a reasonable default value then we include an extra parameter indicating whether to use it or not. For example in our master YAML file we have a template that initializes our build process but must support .NET Framework and Core apps, publishing, etc so we make these parameters to the template. Since we are at the master YAML level we can set the values directly but you could use template expressions if you wanted I suspect. - template: yaml/initialize-build.yml@templates
parameters:
packageBuild: true
publishBuild: true We do have a couple of templates we only want to call if certain conditions are set. - ${{ if eq(parameters.metadataEnabled, 'true') }}:
- template: add-props-metadata.yml
parameters:
assemblyTypeName: ${{ parameters.metadataTypeName }}
isPublic: ${{ parameters.metadataIsPublic }} I suspect your issue is that you're trying to use - template: /templates/template.yaml
parameters:
extraparam: 0
useextraparam: ${{ eq(variables.var2, 'true') }} |
Thank you for your response. We have a template with a parameter that has a default value. Default value is valid for 99% cases but there is one case that requires us to override it with variable. We also use very similar code to your examples. I just don't understand why if statement is not able evaluate properly very simple condition like here: - template: /templates/template.yaml
parameters:
build: ${{ parameters.param1}}
repo: ${{ variables.var1 }}
${{ if eq(variables.var2, 'True') }}:
extraparam: 0 |
As I mentioned the conditional template expression is evaluated when the template is loaded and therefore it'll use the value of the variable at the point the template is loaded into memory, not whatever value you eventually assign it. Parameters work because parameters are set when the template is loaded but variables not so much. That is one reason why our templates tend to include an "enable" parameter to allow callers to determine whether to enable functionality or not rather than the template itself. We also sometimes use multi-step templates which can sometimes help to work around issues, but not with conditions. As a last resort we call to Powershell to get the exact behavior we want. |
The first time I wanted to use templates and I immediately ran into that issue. What a poor design. Users should not need to care about when and how variables are expanded. That is just an implementation detail. It should be as easy as this:
The pipeline should then make 2 evaluation passes:
How hard can it be? |
I am having the same experience. |
I think the docs are good but there is something which needs to be spelled out with examples better IMHO. It says over and over again that 'variables arent available at template expansion time', but there are circumstances which make it APPEAR as though they are, but they arent. (excuse the syntax) variables: steps:
template.yml parameters:
steps:
But during the template expansion step, all instances of parameters.publishNow will be replaced by a string literal "$(publishNowVariable)" for the purposes of template expression evaluation.
BUT! gets expanded to at template expansion time but not evaluated as part of a template expression. When we get to runtime, the varaible is dereferenced as we get: So it kind of appears as though somehow variables are available to templates, but its an illusion created by no properly understanding steps of the two phases. It took me 6 hours of googling until I eventually stumbled upon the initializeLog.txt file in the 'download logs' zip in the Portal (this could be better documented!) and then the penny dropped. |
I'm no longer working on Azure DevOps. |
Hi @RDavis3000 I would suggest to open a ticket on https://developercommunity.visualstudio.com/search?space=21 for this question - this repo is mostly for questions regarding templates located here, but devcom portal would be a better place for generic ADO questions. |
Please add a 'condition' property to a template. It solves soooo many problems. Some use cases which are now impossible:
Error unrecognised value 'stageDependencies' |
Hi Team I am running this pipeline and getting the same error directive are not supported for the expression that are embedded within the string Directive are only supported when the entire value is an expression. Below is the pipeline script trigger:
pool: parameters:
variables: Set the suffix of the version number depending on whether this is master, pr, or other branch${{ if eq(variables['Build.SourceBranch'], 'refs/heads/master') }}: steps:
|
Fast forward 5 years later same old error, Nice! |
If any one had the same issue as I did which is Conditions in Variables Template by Parameters ( ChatGPT Gave me wrong answer that it's not possible sadly ..) It's possible like explained above simply under variables call the template and pass parameters to it:
and inside the Template
for some reason such simple action took few hours to find answer to, and chatGPT wasn't helping. |
copy from: microsoft/azure-pipelines-agent#1749
Have you tried trouble shooting?
Queuing a build generates an error. Unexpected value 'condition'.
Agent Version and Platform
Version of your agent? 2.102.0/2.100.1/...
OS of the machine running the agent? Windows
VSTS Type and Version
VSTS
If VisualStudio.com, what is your account name? https://fsmb.visualstudio.com
What's not working?
I am trying to set up a YAML build. We have a lot of builds and our existing definitions use task groups. I am moving the task group logic into a template so they can be shared across build definitions. The templates are stored in a separate repository. All this is working correctly.
There is one step that shouldn't execute when it is a PR build so I tried to do this.
parameters:
value: true
condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest'))
The build won't queue because of the error - Unexpected value 'condition'.
It would be nice if templates could be conditional like tasks are. Right now the workaround would be to either use a parameter or specify the condition inside the template but the template may not always know the correct condition to use.
Agent and Worker's Diagnostic Logs
It is not queuing so there is no log.
Related Repositories
Please ensure you are logging issues to the correct repository in order to get the best support.
Tasks Repository - contains all of the inbox tasks we ship with VSTS/TFS. If you are having issues with tasks in Build/Release jobs (e.g. unreasonable task failure) please log an issue here.
Hosted Agent Image Repository - contains the VM image used in the VSTS Hosted Agent Pool. If you are having Build/Release failures that seems like they are related to software installed on the Hosted Agent (e.g. the DotnetSDK is missing or the AzureSDK is not on the latest version) please log an issue here.
If you are hitting a generic issue about VSTS/TFS, please report it to the Developer Community
The text was updated successfully, but these errors were encountered: