PSArm is an experimental PowerShell module that provides a domain-specific language (DSL) embedded in PowerShell for Azure Resource Manager (ARM) templates, allowing you to use PowerShell to build ARM templates.
We're using this project both to better understand how PowerShell could boost ARM authoring, but also as test case for improving DSL support in PowerShell more generally. We hope that work here can help us build an inventory of PowerShell DSL patterns, and from those determine what could be implemented at PowerShell or tooling layers to make DSL creation and maintenance easier and more "featureful".
Because this is currently an experimental project, it is not, at present, planned for official support or maintenance, and may make breaking changes as development continues. If a functionality is missing or seems to not work correctly, please open an issue!
This project is different from Project Bicep, which is a standalone DSL for building ARM templates. PSArm is a PowerShell-embedded DSL exposed through a PowerShell module, however it uses Bicep's underlying schema backend to power its ARM resource generation and completions.
The primary goal of PSArm is to use the strengths of PowerShell to enhance the ARM authoring experience. In particular, high level goals are:
- Integrate with PowerShell's completion infrastructure to provide discoverability for ARM wherever possible
- Use PowerShell's reflective object awareness to intelligently create ARM structures based on context
- Leverage PowerShell's dynamic scope to make keywords context-dependent
- Reuse PowerShell's expressive pipeline-emitting semantics
to enable powerful generative ARM template specification,
especially with concepts like piping and
foreach
/ForEach-Object
- Take advantage of PowerShell's pithy, whitespace-aware syntax to offer a clean syntax for ARM that displays only the needed information, with as little boilerplate as possible
Full, tested examples are available in the examples directory.
A simple example for creating a network interface, which can also be found here:
# network-interface.psarm.ps1
# Specify the ARM template purely within PowerShell
Arm {
param(
# ValidateSet is turned into "allowedValues"
[ValidateSet('WestUS2', 'CentralUS')]
[ArmParameter[string]]
$rgLocation,
[ArmParameter[string]]
$namePrefix = 'my',
[ArmVariable]
$vnetNamespace = 'myVnet/'
)
# Use existing PowerShell concepts to make ARM easier
$PSDefaultParameterValues['Resource:Location'] = $rgLocation
# Resources types, rather than being <Namespace>/<Type> have this broken into -Namespace <Namespace> -Type <Type>
# Completions are available for Namespace and ApiVersion, and once these are specified, also for Type
Resource (Concat $vnetNamespace $namePrefix '-subnet') -Namespace Microsoft.Network -ApiVersion 2019-11-01 -Type virtualNetworks/subnets {
Properties {
# Each resource defines its properties as commands within its own body
AddressPrefix 10.0.0.0/24
}
}
# Piping, looping and commands like ForEach-Object all work
'-pip1','-pip2' | ForEach-Object {
Resource (Concat $namePrefix $_) -ApiVersion 2019-11-01 -Namespace Microsoft.Network -Type publicIpAddresses {
Properties {
PublicIPAllocationMethod Dynamic
}
}
}
Resource (Concat $namePrefix '-nic') -ApiVersion 2019-11-01 -Namespace Microsoft.Network -Type networkInterfaces {
Properties {
# IpConfigurations is an array property, but PSArm knows this
# All occurences of array properties will be collected into an array when the template is published
IpConfigurations {
Name 'myConfig'
properties {
PrivateIPAllocationMethod Dynamic
# ARM expressions can be expressed in PowerShell
# The subnet ID here is: [resourceId('Microsoft.Network/virtualNetworks/subnets', concat(variables('vnetNamespace'), variables('namePrefix'), '-subnet'))]
Subnet {
id (ResourceId 'Microsoft.Network/virtualNetworks/subnets' (Concat $vnetNamespace $namePrefix '-subnet'))
}
}
}
}
}
Output 'nicResourceId' -Type 'string' -Value (ResourceId 'Microsoft.Network/networkInterfaces' (Concat $namePrefix '-nic'))
}
Run this with the following command:
# Run the template and publish it to a JSON file. By default this is ./template.json
Publish-PSArmTemplate -Path ./network-interface.psarm.ps1 -Parameters @{ rgLocation = 'WestUS2' }
# Deploy the template to a resource group using the Az.Resources command
New-AzResourceGroupDeployment -ResourceGroupName MyResourceGroup -TemplateFile ./template.json
This will create the following template:
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"metadata": {
// PSArm, like Bicep, inserts this metadata so it's known how many deployments
// are done through PSArm (i.e. how useful is PSArm to Azure customers?).
// It can be stripped out harmlessly if it's unwanted.
"_generator": {
"name": "psarm",
"version": "0.1.0.0",
"psarm-psversion": "7.2.0-preview.4",
"templateHash": "6758140738045718234"
}
},
"resources": [
{
"name": "network-interface",
"type": "Microsoft.Resources/deployments",
"apiVersion": "2019-10-01",
"properties": {
"mode": "Incremental",
"expressionEvaluationOptions": {
"scope": "inner"
},
"template": {
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"variables": {
"vnetNamespace": "myVnet/"
},
"resources": [
{
"name": "[concat(variables('vnetNamespace'), 'my', '-subnet')]",
"apiVersion": "2019-11-01",
"type": "Microsoft.Network/virtualNetworks/subnets",
"properties": {
"addressPrefix": "10.0.0.0/24"
}
},
{
"name": "[concat('my', '-pip1')]",
"apiVersion": "2019-11-01",
"type": "Microsoft.Network/publicIpAddresses",
"location": "WestUS2",
"properties": {
"publicIPAllocationMethod": "Dynamic"
}
},
{
"name": "[concat('my', '-pip2')]",
"apiVersion": "2019-11-01",
"type": "Microsoft.Network/publicIpAddresses",
"location": "WestUS2",
"properties": {
"publicIPAllocationMethod": "Dynamic"
}
},
{
"name": "[concat('my', '-nic')]",
"apiVersion": "2019-11-01",
"type": "Microsoft.Network/networkInterfaces",
"location": "WestUS2",
"properties": {
"ipConfigurations": [
{
"name": "myConfig",
"properties": {
"privateIPAllocationMethod": "Dynamic",
"subnet": {
"id": "[resourceId('Microsoft.Network/virtualNetworks/subnets', concat(variables('vnetNamespace'), 'my', '-subnet'))]"
}
}
}
]
}
}
],
"outputs": {
"nicResourceId": {
"type": "string",
"value": "[resourceId('Microsoft.Network/networkInterfaces', concat('my', '-nic'))]"
}
}
}
}
}
]
}
For more in-depth examples, see the examples directory.
The Publish-PSArmTemplate
command is the key cmdlet for executing PSArm templates.
It performs the following tasks:
- Collects PSArm template files from the
-Path
parameter, supporting either file paths or directory paths (which it will recursively search for files ending with.psarm.ps1
). - Passes through any parameters specified with the
-Parameters
parameter to scripts executed (both the psarm.ps1 scripts and the Arm templates within) - Executes the PSArm template scripts in discovery order and collects them into a nested ARM template
- Uses either
Get-AzAccessToken
oraz account get-access-token
to get an Azure access token and uses the Azure template hash API to add a hash to the generated JSON template's metadata. This can be disabled with-NoHashTemplate
or a custom Azure token provided with-AzureToken
. - Writes the final nested JSON template file out to the
-OutFile
path or./template.json
by default.- If the file already exists this will fail unless
-Force
is used. -PassThru
can be specified to also get the full template object from the command-NoWriteFile
can be specified to prevent the file being written
- If the file already exists this will fail unless
-Verbose
will give a good account of whatPublish-PSArmTemplate
is doing
Publish-PSArmTemplate
will write a JSON file to disk only,
and is not intended to deploy the resulting ARM template.
Deployment functionality is already provided and maintained
in Azure PowerShell commands and the az
CLI.
Having to learn and write a new DSL takes time and effort, especially a with complex hierarchical one like PSArm. So PSArm comes with two commands to help:
ConvertFrom-ArmTemplate
, which takes in ARM JSON and converts to a PSArm in-memory objectConvertTo-PSArm
, which takes a PSArm object and writes it out as PSArm PowerShell script
A typical invocation looks like this:
ConvertFrom-ArmTemplate -Uri 'https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/101-vm-simple-windows/azuredeploy.json' |
ConvertTo-PSArm -OutFile ./windows-vm.psarm.ps1 -Force
These conversion cmdlets aren't perfect, and of course they can't replicate things like loops within PowerShell, but they should help to make using PSArm much easier. If you hit a bug or an issue with the conversion cmdlets, definitely open an issue.
PSArm offers contextual completions on keywords and parameters:
PSArm is a hierarchical, context-sensitive domain-specific language embedded within PowerShell.
The DSL tries to be as unmagical as possible;
pretty much all the functionality is ordinary functions, with next to no AST rewriting
(some has to be done with the param
block, but that's it).
That means you can write an ordinary PowerShell script and embed one or more PSArm blocks inside of it,
or you can call out from your ARM block into another script.
You can use the PSArm keywords in any script and they will work.
ARM template definitions are written as scriptblock bodies to the Arm
keyword,
within which PSArm offers its DSL functionality, complete with contextual completions.
The Arm
keyword then constructs an object representation of an ARM template,
which is output when the script is executed.
So when Publish-PSArmTemplate
is run on these scripts,
it simply executes them and collects all the ARM objects they emit.
Publish-PSArmTemplate
only looks for scripts that end with the .psarm.ps1
extension
so that it can support being given directory paths.
This means you can mix ordinary scripts and PSArm scripts in the same directory
without Publish-PSArmTemplate
accidentally executing those ordinary scripts.
Publish-PSArmTemplate
aggregates all the templates it collects into a nested template
and writes that out as an ARM JSON file, ready for deployment.
PSArm scripts are ordinary PowerShell scripts,
so when they are run (by Publish-PSArmTemplate
for example) they are simply invoked like any other script.
That means you can freely add a param
block to your PSArm scripts to parameterize them,
and then provide those parameters to Publish-PSArmTemplate
through its -Parameters
parameter.
Note that the -Parameters
parameter accepts a hashtable, but will also accept a PSObject,
meaning you can do the following:
$parameters = Get-Content ./parameters.json | ConvertFrom-Json
Publish-PSArmTemplate ... -Parameters $parameters
Using ordinary PowerShell variables to create ARM scripts will work in many scenarios, but sometimes you do need an actual ARM variable or parameter, for example:
- For secure parameters, like of type
securestring
, that you don't want to bake directly into an ARM template, - With functions that must be evaluated at deployment time, such as
[resourceGroup()]
or[deployment()]
, - Or in cases where an expression needs to be evaluated only once, like
[uniqueString()]
or[utcNow()]
In these cases, you can create ARM parameters and variables by adding a param
block to the body of the Arm
keyword itself.
ARM parameters and variables are specified by type;
[ArmParameter[<type>]]
and [ArmVariable]
respectively.
Those parameters and variables then use their PowerShell variable name in their template.
They also support PowerShell features like default values and the ValidateSet
attribute.
When Publish-PSArmTemplate
instantiates an ARM template, it will try to use any of the values from the -Parameters
parameter
to also instantiate parameters to the Arm
block (in addition to the .psarm.ps1
script).
Any parameters it doesn't have a value for will be left in the template and published as part of it,
to be provided at deployment.
As a quick example, the following PSArm script:
# storageAccount.psarm.ps1
param(
[Parameter(Mandatory)]
[string]
$StorageAccountName,
[Parameter()]
[ValidateSet('WestUS2', 'CentralUS')]
[string]
$Location = 'WestUS2'
)
Arm {
param(
[ValidateSet('Hot', 'Cool', 'Archive')]
[ArmParameter[string]]
$accessTier = 'Hot',
[ArmParameter[int]]
$httpsOnly,
[ArmVariable]
$deploymentTime = (utcNow)
)
Resource $StorageAccountName -Namespace Microsoft.Storage -Type storageAccounts -ApiVersion 2019-06-01 -Kind StorageV2 -Location $Location {
ArmSku Standard_LRS
properties {
accessTier $accessTier
supportsHTTPSTrafficOnly $httpsOnly
allowBlobPublicAccess 1
allowSharedKeyAccess 1
}
}
Output 'deploymentTime' -Type string -Value $deploymentTime
}
Published like this:
Publish-PSArmTemplate -TemplatePath ./storageAccount.psarm.ps1 -Parameters @{
StorageAccountName = 'MyStorageAccount'
allowPublicAccess = 1
}
Will generate an ARM template like this (the full output will be embedded in a nested template):
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"httpsOnly": {
"type": "int"
}
},
"variables": {
"deploymentTime": "[utcNow()]"
},
"resources": [
{
"name": "MyStorageAccount",
"apiVersion": "2019-06-01",
"type": "Microsoft.Storage/storageAccounts",
"kind": "StorageV2",
"location": "WestUS2",
"sku": {
"name": "Standard_LRS"
},
"properties": {
"accessTier": "Hot",
"supportsHttpsTrafficOnly": "[parameters('httpsOnly')]",
"allowBlobPublicAccess": 1,
"allowSharedKeyAccess": 1
}
}
],
"outputs": {
"deploymentTime": {
"type": "string",
"value": "[variables('deploymentTime')]"
}
}
}
You can see:
-
$StorageAccountName
and$Location
work like ordinary PowerShell variables and were simply subtituted by value. If$StorageAccountName
weren't provided you would see the standard mandatory parameter prompt, and$Location
defaults to the value given. -
$accessTier
has a known value at publish time, so it's directly substitued into the template -
$httpsOnly
has no given value, so becomes a parameter of the published template -
$deploymentTime
is a variable, so always has its value embedded as a variable in the template (it's assumed that if you've created an ARM variable, you deliberately want it in the template)In general the advice is:
- Keep things as ordinary script parameters whenever you can
- Prefer ARM variables to ARM parameters for things that must be evaluated at deployment time
- Try not to make ARM variables and ARM parameters depend on each other
- Use ARM parameters for secure inputs
High-level ARM template properties like resources
and outputs
are available in PSArm through keywords
such like Resource
and Output
respectively.
These keywords instantiate one resource or output instance at a time and can be in any order.
Most of the complexity in ARM templates lies within the resources themselves.
For any resource there may be a series of parameters, properties and nested resources.
In PSArm, simple parameters are parameters on the Resource
keyword,
while parameters with object structure are keywords under the Resource
keyword.
Underneath each resource, properties on that resource are available as PowerShell functions,
which either take a value or a scriptblock body depending on the type accepted by the keyword.
For example, for a resource of type Microsoft.Network/networkInterfaces
,
the properties
keyword will be available, and within that an ipConfigurations
keyword
that specifically configures the ipConfigurations
property.
Whereas in Microsoft.Network/publicIpAddresses
, ipConfigurations
is meaningless,
but publicIPAllocationMethod
allows you to configure the IP allocation method.
The ARM template language has a template expression language embedded in JSON string values that it evaluates at deployment time, allowing parameterization and deduplication of templates.
In PSArm, ordinary PowerShell variables can be used, obviating the need for many ARM variable expressions, but there are still a number of cases where an ARM expression may be required:
- A builtin ARM template function that must be evaluated at deployment time, like
resourceGroup()
orutcNow()
- A variable is needed to be evaluated only once, such as using
uniqueString()
to provide a hash value reused everywhere in a template - The template is to be constructed with PowerShell, but parameterized for later deployment without PowerShell or without the PSArm module
In these cases, it's still desirable to be able to write ARM expressions into a template, but writing these as strings in PSArm would be a cumbersome experience. Instead PSArm provides ARM builtin expression functions as PowerShell functions. These functions allow you to use PowerShell syntax to express function application and member access:
(resourceGroup)
becomes[resourceGroup()]
concat "a" (resourceId 'Microsoft.Storage/storageAccounts')
becomes[concat('a', resourceId('Microsoft.Storage/storageAccounts'))]
(resourceGroup).location
becomes[resourceGroup().location]
PSArm embeds metadata in generated templates to mark them as being generated by PSArm, using the same method as Azure Bicep. When those ARM templates are deployed, this metadata is picked up for telemetry by the ARM deployment service. Importantly, no personal or identifiable information is made available by this. This helps us to count the number of ARM deployments PSArm powers, so we can understand how useful PSArm is to Azure customers and make investments in its development.
The metadata in the template looks like this:
{
"metadata": {
"_generator": {
"name": "psarm",
"version": "0.1.0.0",
"psarm-psversion": "7.1.3",
"templateHash": "<hash>"
}
}
}
You can strip out this metadata from the template without affecting any functionality.
PSArm comes with a build script that tries to keep things simple and minimal. To build it, run:
./build.ps1
This will output the built module to out/PSArm
, which can be imported with Import-Module ./out/PSArm
.
Keep in mind that this is a binary module, so you'll likely want to start a new process before importing it
so that you can easily rebuild and reimport as you make changes.
To run the tests after your build, run:
./build.ps1 -Test
PSArm is currently built with Invoke-Build,
so more advanced build tasks are available through the Invoke-Build
command.
Template schema support in PSArm comes from the bicep-types-az project,
which also powers Bicep.
While an early prototype of PSArm did create and ship its own schemas,
the Azure.Bicep.Types.Az
NuGet package is now the best way to deal with ARM schemas in .NET.
- High-level DSL keywords are implemented as cmdlets that implement logic by hand
- Beyond the high level keywords, all other template building functionality is implemented by wrapping primitive commands:
RawEntry
/New-PSArmEntry
, which describes a JSON key/value pairRawCall
/New-PSArmFunctionCall
, which describes an ARM function call (like[concat('prefix', 'suffix')]
)- All ARM functions and resource keywords are autogenerated functions defined in script that wrap these primitives
- Lower-level keywords within resources are described by Bicep schema types and are converted to script on demand:
- When completions are asked for or an ARM template command is processed with resources, the required schema objects are loaded
- A script writer visits these schemas and converts them to a series of simple PowerShell functions, with inner keywords represented recursively as inner functions
- These inner functions mainly declare their parameters and delegate back to cmdlets that turn these parameters into a named JSON element
- When each resource is invoked, the functions are converted to scriptblocks
and the resource bodies are invoked using the
ScriptBlock.InvokeWithContext()
method, allowing the DSL functions to be defined within the body scriptblock without polluting any higher scopes
- Each keyword invokes its scriptblock body in user scope and collects the output, sifting through it based on object type and reconstructing an object hierarchy from it, like a complex builder pattern
- These objects agglomerate together as they come up through the keywords,
with the
Arm
keyword capturing them all under one big object - The
Arm
keyword also looks at the AST of the scriptblock its given to build a list of ARM parameters and variables, and remember any constraints applied to them like types or enums. It also rewrites the scriptblock to remove any of the constraints on parameter values so that it can run the scriptblock - ARM expression functions like
concat
andresourceId
are also written as functions and instantiateArmFunctionCall
instances to render properly in templates.- These objects also extend
DynamicObject
so that member access and indexing turns such an expression into the corresponding ARM expressions
- These objects also extend
- The DSL also comes with completion logic:
- There's an argument completer for the
Resource
keyword to complete the-Type
,-ApiVersion
and-Namespace
parameters to list resources for which schemas are available - Most completions come from a from-scratch completer written to understand the hierarchical keyword context to provide keywords within each schema that work for particular contexts, in particular keywords that work with each resource.
- There's an argument completer for the
PSArm is open-source and contributions are welcome. The project is still under heavy development, so code contributions, especially to the core library, may not get immediate attention.
Some of the most valuable contributions you can provide are issues, scenarios and PSArm examples (or desired template examples).
PSArm is licensed under the MIT license.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.