RFC: A Maester-native test model for Maester 3.0

[merill]

RFC: A Maester-native test model for Maester 3.0

Status: We have decided on the overall direction, but the design and migration experience are still open for community input.

We are planning a significant architectural change for Maester 3.0: built-in Maester checks will move from Pester wrappers to a native Maester test model.

This is not a criticism of Pester. Pester gave Maester an enormous head start, including a familiar PowerShell DSL, test discovery, assertions, filtering, tags, and result handling. It allowed us to bootstrap Maester quickly and build the community and test library we have today.

However, Maester is evolving from a collection of Pester tests into a security assessment engine. Some of our future plans require Maester to understand and control the complete lifecycle of each check rather than treating it only as a Pester test.

The problem with the current model

Today, the information required to define a built-in check can be spread across several places:

1. A .ps1 file containing the check implementation.
2. A companion .md file containing its description and remediation guidance.
3. A separate .Tests.ps1 Pester wrapper containing the test ID, title, assertion, and tags.
4. The central maester-config.json file containing metadata such as severity and title.

This makes checks harder to create, review, and maintain:

• Metadata can drift between the implementation, Pester wrapper, Markdown, and configuration.
• Adding or changing a check often requires editing several files.
• The large central JSON file is a frequent source of merge conflicts.
• Test prerequisites are implemented repeatedly across individual checks.
• Tags and other metadata do not have one strongly defined schema.
• The Maester engine cannot easily reason about prerequisites, dependencies, or which checks can safely run in parallel.

The current custom test format and configuration model show how Pester metadata and maester-config.json are currently used.

The proposed Maester 3.0 model

A native Maester check would consist of two files:

Test.<ID>.ps1
Test.<ID>.md

For example:

Test.MT.1063.ps1
Test.MT.1063.md

The PowerShell file would contain the test implementation and its metadata. The Markdown file would continue to contain the human-readable explanation, impact, and remediation guidance.

Test metadata would be declared directly on the test function using a Maester attribute. A new class, with validated properties, would act as the schema for that metadata.

The schema could describe information such as:

• Test ID, title, severity, and tags
• Required services and connections
• Minimum licences
• Applicable tenant types, such as workforce or external tenants
• Supported commercial, government, and sovereign clouds
• Whether a check can run in parallel
• Other execution requirements or dependencies

The exact schema is not final and is one of the areas where we want community feedback.

Inspiration from the Zero Trust Assessment

This approach is similar to the model used by Microsoft’s Zero Trust Assessment. Here is a shortened excerpt from one of its [test attributes(https://github.com/microsoft/zerotrustassessment/blob/psnext/src/powershell/tests/Test-Assessment.21770.ps1)

:

function Test-Assessment-21770 {
    [ZtTest(
        MinimumLicense = ('P1'),
        Pillar = 'Identity',
        RiskLevel = 'Medium',
        TenantType = ('Workforce','External'),
        TestId = 21770
    )]
    [CmdletBinding()]
    param(
        $Database
    )

    # Test implementation
}

The Zero Trust Assessment defines these properties through an attribute class and discovers metadata from the function definition.

The Maester schema will be designed for Maester’s requirements, but the underlying idea is similar: keep the metadata with the check and give the engine a structured way to understand it.

What this unlocks

Moving these requirements into structured metadata would allow the Maester engine to handle common behaviour centrally.

For example, instead of every check implementing its own connection and licence checks, a test could declare its requirements and allow Maester to:

• Determine whether the required service is connected.
• Check whether the tenant has the necessary licence.
• Decide whether the check applies to a workforce or external tenant.
• Determine whether it is supported in the current cloud.
• Produce a consistent skip reason when a requirement is not met.

This removes duplicated boilerplate from individual checks and makes prerequisite handling more consistent.

Owning the execution model would also give Maester more control over scheduling. Independent checks could run in parallel, while checks that share constrained state, connections, or dependencies could remain serialized.

It also gives us a foundation for better validation, discovery, filtering, diagnostics, progress reporting, and future execution environments.

Proposed implementation plan

Our current plan is to:

• Introduce a new Maester test attribute class whose properties define the test metadata schema.
• Move built-in checks to the Test.<ID>.ps1 and Test.<ID>.md convention.
• Replace separate built-in .Tests.ps1 wrappers with metadata on the check function.
• Move built-in test implementation functions from public commands to private functions.
• Move common prerequisite and skip logic into the Maester core engine.
• Add build or migration tooling to convert legacy definitions into the new format.
• Migrate all built-in checks to the native Maester model.
• Continue to recognize and run existing Pester-based custom tests through a compatibility path.

Backward compatibility

We know many organizations have invested in custom Pester tests, and we do not want Maester 3.0 to require an immediate rewrite of those checks.

Our plan is for Maester 3.0 to support both:

• The new native Maester test format.
• Existing Pester-based custom tests using Describe, It, tags, and *.Tests.ps1.

Legacy Pester tests should continue to execute and appear in Maester reports. Some future metadata-driven capabilities may require migration to the native format, but existing custom checks should not stop working simply because Maester 3.0 is installed.

We also plan to provide migration or conversion tooling so test authors can adopt the new format without having to translate everything manually.

One important breaking-change question

Moving built-in test functions to private functions means they would no longer be treated as public Maester commands.

Most users run these checks through Invoke-Maester, but we need to understand whether anyone directly invokes individual built-in functions such as Test-Mt... from scripts, integrations, or automation.

If that is part of your workflow, please let us know and share an example. This will help us determine whether we need an additional compatibility layer or a supported public API for invoking a single check.

We want your feedback

In particular, we would like to hear from:

1. Custom test authors: What patterns do your existing Pester tests rely on?
2. Automation maintainers: Do you invoke individual Test-Mt... functions directly?
3. Contributors: What currently makes adding or reviewing a Maester check difficult?
4. Test authors: Which fields should be part of the new metadata schema?
5. Government and sovereign cloud users: What environment-specific requirements need to be represented?
6. Everyone: What would you need to see in the compatibility and migration plan before adopting Maester 3.0?

The goal is to make each check easier to understand, contribute, validate, and execute while giving Maester room to grow as a security assessment platform.

Please share concerns, edge cases, examples of custom tests, and ideas below. We would especially like to uncover compatibility issues before the Maester 3.0 format is finalized.

[KelvinTegelaar]

Personally I love this model; it allows more flexibility in scripts, allows easier integration of tests that has complex needs, but also allows quicker development in genreal.

As we integrated the zero trust tests into CIPP, and by proxy some Maester tests, we have a large stack of already converted tests to this new format, with the caveat that they test against our CIPP DB structure instead of Maester, however - They are very useful to see the testing framework proposed in production.

For example, our CIS framework can be found here:

https://github.com/CyberDrain/CIPP/tree/main/backend/Modules/CIPPTests/Public/Tests/CIS/Identity

Using the first example in those tests showcases how easy it is to create a test:

https://github.com/CyberDrain/CIPP/blob/main/backend/Modules/CIPPTests/Public/Tests/CIS/Identity/Invoke-CippTestCIS_1_1_1.md and https://github.com/CyberDrain/CIPP/blob/main/backend/Modules/CIPPTests/Public/Tests/CIS/Identity/Invoke-CippTestCIS_1_1_1.ps1 combine into a markdown result that is easy to present.

Now these are "Cipp-ified" in general, however if this RFC is approved, we're 100% okay with contributing these back to the repository in their non-cipp version, and the logic remains mostly the same for any conversion that needs to happen, or we contirbute them back as-is in an "example" format so people can actively contribute to convert them.

[diecknet]

I think it would be great, if at the same time a new way would be introduced to configure the tests. For example to define the Break Glass accounts or accepted deviations from security recommendations like "no public m365 groups".

[lnfernux]

@diecknet agreed. Not quite sure what the best option for configuration/customisation would be, but maybe separate filepath/files per test is the best way.

In one way, to disable a single test could be done in the metadata of that test, but you would end up with a lot of drift in the actual test files and could cause issues when updating from source.

Similarly adding more information such as BTG users to a test, or targetting specific groups/users if the test lends itself to that could be handled in a similar way (in file) but would also cause drift and issues.

Maybe a specific folder path and format of files where you define TestName.Configuration.MD locally and you can disable or add specific information such as groups there. There file is not required, but recommended in some cases. Could also be a single json file with blocks per test. For the tests that need this info to work (BTG being a great example), we can add example configuration bundled in.

I'm not sure what other customisation would be required, but as long as we can make configuration of the tests not impact the testfiles themselves, and just be a local addition then should be good.