Skip to content

Add ARM template validation with GitHub Actions and PSRule #1606

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: dev
Choose a base branch
from

Conversation

Copilot
Copy link
Contributor

@Copilot Copilot AI commented May 19, 2025

This PR adds comprehensive automated validation for ARM templates as requested in the issue. The implementation includes:

🔍 New GitHub Actions Workflow

  • Created a new .github/workflows/arm-template-validation.yml that will trigger on PRs affecting templates (currently disabled for Phase 1)
  • The workflow validates ARM templates using multiple tools:
    • PSRule.Rules.Azure for best practices and security validation
    • ARM-TTK for common deployment issues (pinned to stable version 20250401)
    • Azure CLI validation for deployment verification without actual deployments

🛠️ New PowerShell Validation Script

  • Added src/scripts/Test-ArmTemplate.ps1 for local template validation
  • The script uses the same validation tools as the GitHub Actions workflow
  • It automatically installs required modules if they're missing
  • Provides detailed validation errors to help developers fix issues quickly
  • Supports validation levels (Strict/Lenient) for experimental features

📝 Documentation Updates

  • Added a new section to docs-wiki/Build-and-test.md about ARM template validation
  • Updated the scripts README with documentation for the new Test-ArmTemplate script
  • Included examples for running validation locally
  • Added "What's Being Validated" section explaining specific checks performed

🚀 Phased Rollout Plan

Due to existing validation errors in our templates, we're implementing a phased rollout:

Phase 1 (This PR):

  • ✅ Validation tools available for local use only
  • ✅ CI/CD workflow disabled (manual dispatch only)
  • ✅ ARM-TTK downloads to release/.tools/ (gitignored)
  • ✅ Documentation updated with phased approach

Phase 2 (Issue #1696):

  • Fix ~25 validation errors found in existing templates
  • Address API version updates, hardcoded locations, trailing whitespace

Phase 3 (Future):

  • Enable CI/CD workflow for all PRs
  • Monitor and iterate based on team feedback

Testing

To test validation locally:

# Validate all templates
./src/scripts/Test-ArmTemplate.ps1

# Validate specific template
./src/scripts/Test-ArmTemplate.ps1 -TemplatePath "release/finops-hub/azuredeploy.json"

# Use lenient mode for experimental features
./src/scripts/Test-ArmTemplate.ps1 -ValidationLevel Lenient

This implementation gives developers the ability to validate templates locally while ensuring we don't block current development with existing validation errors.

Fixes #2.


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI and others added 2 commits May 19, 2025 17:26
Co-authored-by: MSBrett <24294904+MSBrett@users.noreply.github.com>
Co-authored-by: MSBrett <24294904+MSBrett@users.noreply.github.com>
Co-authored-by: MSBrett <24294904+MSBrett@users.noreply.github.com>
@Copilot Copilot AI changed the title [WIP] ARM template deployment tests Add ARM template validation with GitHub Actions and PSRule May 19, 2025
@Copilot Copilot AI requested a review from MSBrett May 19, 2025 17:30
Copilot finished work on behalf of MSBrett May 19, 2025 17:30
…n permissions


Test and adjust as needed.

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replace azure/setup-azure-cli@v1 with the latest supported version. As of now, the latest version is azure/setup-azure-cli@v3. Update the workflow as follows:

  • name: Setup Azure CLI
    uses: azure/setup-azure-cli@v3

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's the corrected part:

jobs:
validate_templates:
name: Validate ARM Templates
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v3

  - name: Setup Az PowerShell module
    shell: pwsh
    run: |
      Install-Module -Name Az -Force -AllowClobber -Scope CurrentUser
      Install-Module -Name PSRule.Rules.Azure -Force -Scope CurrentUser

  - name: Setup Azure CLI
    uses: azure/setup-azure-cli@v3  # Updated version

  - name: Setup Bicep
    uses: anthony-c-martin/setup-bicep@v0.5

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've updated the Azure CLI setup action from v1 to v3 as requested. The change can be found in commit 3356e4b.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Azure CLI setup action has already been updated from v1 to v3 as requested. This change was implemented in commit 3356e4b.

Co-authored-by: MSBrett <24294904+MSBrett@users.noreply.github.com>
Copilot finished work on behalf of MSBrett May 19, 2025 18:05
Copilot finished work on behalf of MSBrett May 19, 2025 18:07
@flanakin flanakin added this to the 2025-06 - June milestone May 21, 2025
Phase 1 implementation:
- Move ARM-TTK download location from .temp to release/.tools
- Disable CI/CD validation (manual dispatch only)
- Update documentation with phased rollout plan
- Create issue #1696 for Phase 2 template fixes
- Add validation level support (Strict/Lenient modes)

This allows developers to use validation locally while we fix existing template errors before enabling CI/CD validation.
@MSBrett MSBrett marked this pull request as ready for review June 15, 2025 07:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs: Review 👀 PR that is ready to be reviewed Skill: DevOps GitHub setup and automation Skill: Documentation Documentation updates Tool: FinOps guide Implementing FinOps guide
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ARM template deployment tests
6 participants