Bicep Experimental Test Framework

[sydkar]

Bicep Experimental Test Framework

Introduction

We began prototyping experimental test features early in summer 2023, and we’ve been encouraged by the level of customer interest in the project. While we are still early in exploring what the future of testing looks like in Bicep, we are eager to get early feedback. If you choose to enable the feature, beware the functionality is still experimental and subject to change at any time.

Read the full proposal for the testing framework here.

How to get started with experimental test framework

1. Enable assertions and testFramework experimental feature flags in a bicepconfig.json file. Both feature flags must be enabled for expected functionality.
2. Create a new .bicep file to author your tests, eg. tests.bicep. This must be a separate file from the .bicep template you want to test. Author a test block with the new test keyword, referencing a .bicep template you would like to test on the client-side (without deploying).
3. Author assert statements to validate parameters, variables, and resource names within the .bicep template file you would like to test that you referenced in your test blocks.
4. In your CLI, run the command bicep test <filepath_to_test_file> to run test blocks and their corresponding assert statements without deploying.

1. Enable experimental features

Enable testing experimental features in bicepconfig.json

1. Create bicepconfig.json file within your Bicep project repo

2. Within bicepconfig.json, define experimentalFeaturesEnabled and set testFramework: true and assertions: true

2. Test blocks

test block syntax:

• test keyword
• Name of test block (eg. testMain)
• Reference to Bicep file to test (eg. ‘main.bicep’)
• params object mocking parameter values from the Bicep file (eg. params: {env: 'prod'})

Example:

Lmitations:

• You must pass in a parameter object, you cannot reference an existing parameter file currently
• Test blocks must be in a separate file from the .bicep template you want to test
• Test result output summary is not standardized

3. Assert statements

assert statement syntax:

• assert keyword
• Name of assert statement (eg. appName)
• equals boolean validation expression (eg. = contains(appServiceApp.name, "${env}")

assert statements can be any boolean expression that references parameters, variables, or resource names. These boolean expressions can include Bicep functions such as "contains()", "length()", etc.

Examples:

Limitations:

• You can only assert on parameters, variables, and resource names with current functionality
  • You cannot assert on resource size, location, or other propreties yet as these depend on runtime deployment functionality

[dciborow]

what version of bicep cli do I need for the test command?

And the asserts would be more useful if they could be chained from the object labeled "test". Filling them into my existing bicep files isn't easy.

[oliverlabs]

Are there any updates on this feature?

[stephaniezyen]

@oliverlabs This project has been put on hold for the moment unfortunately. We will update this issue when we pick it back up!

[aslan-im]

any updates?

[allux2]

Any updates? This would be a great feature to have!

[ElYusubov]

Hi @stephaniezyen ,
is there any update on this exciting feature, any plans to release it?

[alex-frankel]

Hi Folks - no updates. It's something we are eager to put more energy into, but do not have the ability to prioritize it at the moment.