Writing native tests

New to writing tests?

Start with Your first custom test, a short walk-through for checking one rule of your own. This page is the full reference.

From Maester 3.0 a test is two files: Test.<ID>.ps1, which holds one PowerShell function with a [MaesterTest(...)] attribute, and Test.<ID>.md, which holds the description and remediation shown in the report. Every test that ships with Maester is written this way, and your own tests use the same format.

The engine reads the attribute without running your code, checks that the services the test needs are connected, runs the function, catches any error, and writes the result row. Your function only has to read the tenant, decide, and return $true or $false.

Tests written for Maester 2.x with Pester (*.Tests.ps1) still run. See Pester-format tests, which also shows how to convert them with Convert-MtTest.

Quick start

Prefer to click through the choices? The test builder explains every attribute and writes the code for you.

# 1. Create custom/Test.CONTOSO.1001.ps1 and custom/Test.CONTOSO.1001.md
New-MtTest -Id CONTOSO.1001 -Title 'Guest invitations are restricted' -Service Graph -Severity High `
    -Product entra -Feature external-identities -Area external-access

# 2. Edit the two files, then check them. No tenant connection is needed for this step.
Get-MtTest -Path ./custom

# 3. Run the test on its own while you write it
Connect-Maester
Invoke-MtTest -Path ./custom/Test.CONTOSO.1001.ps1

# 4. Run it with everything else
Invoke-Maester -Path .

New-MtTest writes to ./custom unless you pass -Path. It supports -WhatIf and refuses to overwrite existing files unless you pass -Force. Only -Id and -Title are required; -Product, -Feature, -Area, -ResourceType, -UserImpact, -ImplementationCost and -Mitre fill in the attributes of the same name.

Files and layout

Put your tests in a folder of your own, usually custom/ inside the folder you run Maester from:

maester-tests/
  maester-config.json        (optional: your run configuration)
  custom/
    Test.CONTOSO.1001.ps1
    Test.CONTOSO.1001.md
    suite.json               (optional: name, tags and source for every test in the folder)
    maester-config.json      (optional: an overlay on the root config)

Invoke-Maester -Path ./maester-tests finds every Test.*.ps1 file under the path, in any subfolder. If you leave out -Path, Maester scans the current folder only when it looks like a Maester folder (it has a maester-config*.json file, a custom folder, a 2.x suite folder, or a *.Tests.ps1 or Test.*.ps1 file). Otherwise only the built-in tests run.

Maester matches the custom folder name without regard to case, so a Custom folder created by Maester 2.x keeps working, and New-MtTest and Install-MaesterTests write into it when it exists.

Rules for a test file:

• The file name is Test.<ID>.ps1, where <ID> is the Id in the attribute. The .md file has the same name with the .md extension and sits in the same folder. A custom test without its .md file is reported as Error with reason InvalidMetadata.
• The file contains exactly one function with a [MaesterTest(...)] attribute. It may contain other functions (helpers, or a family's instance source).
• At the top level the file contains only function definitions: no statements, no script-level param() block. Discovery never executes the file, and this rule keeps it that way.
• The test function has [CmdletBinding()].

Test IDs

An ID is letters and digits in segments separated by . or - (and _ after the first segment). It starts with a letter, contains at least one digit or separator, and is at most 64 characters. CONTOSO.1001, CONTOSO-ID-07 and CT0001 are all valid. IDs are compared case-insensitively.

Use a prefix of your own. These prefixes belong to the tests that ship with Maester: MT., CISA., CIS., EIDSCA., ORCA., AD-, AZDO. and MT1060.. A custom test with one of these prefixes still runs, with a warning. A custom test that uses the exact ID of a built-in test is not run at all, and the built-in runs instead. To change how a built-in test behaves, set its parameters or severity in the run configuration, or copy it under your own ID and disable the built-in.

Two native tests with the same ID (or the same function name) both get an Error row with reason DuplicateId, and neither runs. When a native test and a Pester-format test share an ID, the native test runs and the Pester test does not; it is listed under Selection.Superseded in the result with MatchedBy NativeTest.

A complete example

custom/Test.CONTOSO.1002.ps1:

function Test-ContosoDisabledCaPolicyReason {
    [MaesterTest(
        Id           = 'CONTOSO.1002',
        Title        = 'Disabled Conditional Access policies state why they are disabled.',
        Product      = 'entra',
        Feature      = 'conditional-access',
        Area         = 'access-control',
        ResourceType = 'ConditionalAccessPolicy',
        Severity     = 'Medium',
        Service      = 'Graph',
        License      = 'AAD_PREMIUM',
        Tag          = 'Contoso'
    )]
    [CmdletBinding()]
    [OutputType([bool])]
    param(
        # Text that a disabled policy's name must contain.
        [string] $ReasonMarker = 'Disabled:'
    )

    $policies = Get-MtConditionalAccessPolicy
    $withoutReason = @($policies | Where-Object { $_.state -eq 'disabled' -and $_.displayName -notlike "*$ReasonMarker*" })

    if ($withoutReason.Count -eq 0) {
        Add-MtTestResultDetail -Result 'Well done. Every disabled policy states why it is disabled.'
        return $true
    }

    $result = "These disabled policies do not state why they are disabled:`n`n%TestResult%"
    Add-MtTestResultDetail -Result $result -GraphObjects $withoutReason -GraphObjectType ConditionalAccess
    return $false
}

custom/Test.CONTOSO.1002.md:

Disabled Conditional Access policies must say why they are disabled, in the form `Disabled: <reason>`.

#### Remediation action

1. In the Microsoft Entra admin center, open **Protection** > **Conditional Access** > **Policies**.
2. For each policy listed below, add the reason to its name, or delete the policy.

#### Related links

- [Conditional Access policies](https://entra.microsoft.com/#view/Microsoft_AAD_ConditionalAccess/ConditionalAccessBlade/~/Policies)

<!--- Results --->
%TestResult%

There is no connection check, no licence check and no try/catch in the function. The engine does all three.

The [MaesterTest] attribute

The attribute is read from the file's syntax tree, so its arguments must be constants:

• Named arguments only: Id = 'CONTOSO.1001', never a positional value.
• Values are quoted strings, $true, $false, or a list in parentheses such as ('T1078.004', 'T1110').
• A single string is accepted for a list property: Service = 'Graph'.
• A bool property can be written as a bare flag: LongRunning means LongRunning = $true.
• No variables, expressions, @() arrays or hashtables.

A value that breaks these rules, an unknown property, or a value outside the allowed list makes the test an Error row with reason InvalidMetadata, naming the file and line. The function is never called. If the message says a property or value "may come from a newer version of Maester", the test was written for a later release than the one you run.

The attribute answers six questions about a test. Only Id and Title are required in a custom test; a test that ships with Maester must also answer the others marked "Built-in tests". The attribute rubric says how to choose each value.

Question	Properties
Which test is this?	Id, Title
What is it about?	Product, RelatedProduct, Feature, Area, ResourceType
What do I fix first?	Severity, UserImpact, ImplementationCost
What does it map to?	Mitre, Control
Can it run here?	Service, License, Permission, TenantType, Cloud, Platform
Is it current?	Lifecycle, ReplacedBy

Property	Type	Allowed values	Required	When omitted	What it does
Id	string	ID rules above	Always		Stable identifier. The file is Test.<Id>.ps1. Select the test with -TestId.
Title	string	One line	Always		The title shown in the report. The result row's Name is <Id>: <Title>.
Product	string	A product key from the taxonomy (below), or an alias of one. A custom test may use any one line of up to 40 characters	Built-in tests	Empty	The product where the fix is made. Stored and written to the result as the key. The console progress and the run summary group tests by it; tests without one are shown under Other.
RelatedProduct	string[]	Product keys or aliases; a custom test may use other names	No	None	Other products the test also checks or depends on. Must not repeat Product. -Product matches them too.
Feature	string	A feature key of the test's product. A custom test may use any one line of up to 60 characters	Built-in tests	Empty	The part of the product the test is about, for example conditional-access.
Area	string	An area key. A custom test may use any one line of up to 60 characters	Built-in tests	Empty	The security theme, whatever the product, for example authentication.
ResourceType	string	A resource type key such as Tenant or User. A custom test may use another word of letters and digits	Built-in tests	Empty	What a failing result points at.
Severity	string	Critical, High, Medium, Low, Info	Built-in tests	Empty	Default severity. A TestSettings row in the config overrides it.
UserImpact	string	Low, Medium, High	Built-in tests, unless Severity is Info	Empty	How much the recommended fix disrupts end users.
ImplementationCost	string	Low, Medium, High	Built-in tests, unless Severity is Info	Empty	How much administrator work the recommended fix takes.
Mitre	string[]	MITRE ATT&CK technique IDs (T1110, T1110.003), or the single value None	Built-in tests	None	The techniques the checked control prevents or detects. None cannot be combined with an ID.
Control	string[]	<framework-key>:<requirement-id>, for example cis-m365:5.1.5.1 or nist-800-53-r5:IA-2(1)	No	None	The requirements of frameworks the test implements or supports. The first one that a framework file knows becomes the Benchmark of the result. See Mapping a test to a framework.
Service	string[]	A name from the service registry (below), or None	Built-in tests	No requirement	Services that must all be connected.
License	string[]	Service plan names, not product names; & joins plans	No	No requirement	The tenant needs any one element. See License.
Permission	string[]	One permission per element, no commas	No	None	The permissions the test needs, for example Policy.Read.All. Recorded on the result row; not enforced in 3.0.
TenantType	string[]	Workforce, External	No	Workforce	Tenant types the test applies to. Recorded; enforced only when you turn it on.
Cloud	string[]	Commercial, GCC, GCCHigh, DoD, China, Bleu, Delos, GovSG	No	All clouds	Clouds the test applies to. Recorded; enforced only when you turn it on.
Platform	string[]	Windows, Linux, MacOS	No	All platforms	Operating systems the test can run on.
Lifecycle	string	Preview, Stable, Deprecated	No	Stable	A Preview test is not run unless -IncludePreview, any -Tag, or its exact ID in -TestId. A Deprecated test still runs; the result row says so.
ReplacedBy	string	A test ID	No	Empty	The test that took the place of a Deprecated test.
LongRunning	bool		No	$false	Not run unless -IncludeLongRunning, -Tag LongRunning or -Tag CAWhatIf, or its exact ID in -TestId.
Exclusive	bool		No	$false	The test must not run at the same time as another test. Recorded for a future parallel scheduler; no effect in 3.0.
InstanceSource	string	Name of a function in the same file	No	One result	Makes the test a family with one result per instance.
Tag	string[]	Non-empty, no commas (spaces allowed)	No	None	Labels of your own for -Tag and -ExcludeTag. See Tags.
Author	string[]	GitHub handles	Built-in tests	None	Initial authors, credited on maester.dev.
Contributor	string[]	GitHub handles	No	None	Further contributors, in order of first contribution.
HelpUrl	string	An https:// URL	No	The suite's HelpUrlTemplate with {Id} replaced; none for a custom test without one	The "learn more" link on the result row.

Category and Preview were removed in Maester 3.0. A test that still has one is an Error row with reason InvalidMetadata and a message that names the replacement: Product and Feature instead of Category, and Lifecycle = 'Preview' instead of Preview. The names OptionalService and TimeoutSeconds are reserved for later versions and are rejected today.

Product, feature, area and resource type

The keys come from one registry, powershell/assets/MaesterTaxonomy.psd1: the products with their features, the areas and the resource types. Product, feature and area keys are lowercase kebab-case (entra, conditional-access, privileged-access); resource types are PascalCase (Tenant, ConditionalAccessPolicy). The display names live in the registry only.

• A key is matched without regard to case and stored in the registry's spelling. A product can also be named by an alias: Product = 'Entra ID' loads and is stored as entra.
• A test that ships with Maester must use keys from the registry, and a Feature that belongs to its Product. Anything else is InvalidMetadata.
• A custom test may use any value. A value that matches the registry is stored as the key; any other value is kept as you wrote it.

The result row's Block, which 2.x readers and the XML test-case names use, is derived: the display name of Product when the registry knows it, else Product as written, else Feature, else the Name of the folder's suite.json, else Custom.

Tags

Tag holds labels that you find useful for your own tests. -Tag and -ExcludeTag match them, together with the Tags of the folder's suite.json. The result row's Tag lists the Tag values of the test and nothing else.

Tags are not how Maester classifies a test. Product, feature, area, suite and severity are attributes, and Invoke-Maester selects by them with -Product, -Feature, -Area, -Suite and -Severity. The tests that ship with Maester carry no tags for those things. The tags they had in Maester 2.x, such as CA or CIS, still select the same tests; see Selection.

The Markdown file

Test.<ID>.md has no front matter. It holds:

1. The description: what the test checks and why it matters.
2. A #### Remediation action section with the steps to fix a failing result, ideally with a direct link to the admin portal page.
3. Optionally #### Related links.
4. Exactly one <!--- Results ---> line followed by %TestResult%.

Everything above <!--- Results ---> becomes the description in the report. The text after it is the result template: %TestResult% is replaced with the -Result you pass to Add-MtTestResultDetail. The .md is also shown for rows the engine produced without running the test (not selected, skipped because a service is not connected), so the reader always sees what the test is about.

What the function returns

The function does	Result	Reason code
returns $true	Passed
returns $false	Failed
calls Add-MtTestResultDetail -Investigate	Investigate, whatever it returns
calls Add-MtTestResultDetail -SkippedBecause <code>	Skipped. The call ends the test.	TestSkipped, or NotApplicable for -SkippedBecause NotApplicable
calls Add-MtTestResultDetail -SkippedBecause Error, or throws any error	Error	TestError
returns $null without skipping or -Investigate	Skipped	NoResult
returns something that is not a boolean, or more than one value	Error	InvalidReturn
runs longer than the configured timeout	Error	Timeout

When several apply, they are evaluated in this order: -SkippedBecause Error, -Investigate, any other skip, an uncaught error, the returned value.

Watch for stray output. A command whose output you do not capture ($list.Add($x) on an ArrayList, for example) adds a second value to the function's output and turns the result into InvalidReturn. Assign such calls to $null or pipe them to Out-Null.

Add-MtTestResultDetail

Add-MtTestResultDetail works as in 2.x. The parameters you use in a native test:

• -Result: the Markdown result. It replaces %TestResult% in the .md file.
• -GraphObjects and -GraphObjectType: a list of objects rendered as a table with links to the admin portal, inserted where %TestResult% appears in -Result. See Formatting test results.
• -Investigate: the result needs a person to look at it.
• -SkippedBecause: skip the test for a reason only the test can see, such as NotApplicable, NotSupportedAppPermission or Custom with -SkippedCustomReason. Do not use the NotConnected* and NotLicensed* codes for conditions the attribute can declare; use Service and License.
• -Description: replaces the description from the .md file. Leave it out to use the .md.

-TestTitle and -Severity are ignored in a single-result native test: the title and severity come from the attribute and the config. In a family, -TestTitle sets the instance's title when the instance object has no Title. -Severity is used only when neither the config nor the attribute sets one.

Errors and skips

Do not wrap the body in try/catch only to report -SkippedBecause Error. The engine runs every test inside its own try, so any terminating error ends the test and becomes an Error row with the exception message as ReasonDetail and the error record in ErrorRecord. Statement-terminating errors (a failed command, a method call on $null) also end the test, so it cannot carry on with empty data and return $true by accident.

You may still use try/catch for a condition you handle, for example a 403 from Graph that you report as -SkippedBecause NotAuthorized. A -SkippedBecause call made inside your own try block still skips the test; the engine recognises the skip when your catch passes it to Add-MtTestResultDetail -SkippedBecause Error -SkippedError $_.

Warnings and non-terminating errors written by the test are shown on the console and stored on the result row in Diagnostics.

Declare what the test needs

Do not check connections or licences in the test. Declare them, and the engine decides before the test runs. A test whose requirement is not met gets a Skipped row with a reason code and its .md description, so it still appears in the report. See Applicability for the order of the checks and every reason code.

Service

Service lists the services that must all be connected. Use None for a test that needs no connection. The names come from the service registry:

Service	Also accepted	Connected by
Graph		Connect-Maester
Azure		Connect-Maester -Service Azure
ExchangeOnline	Exchange	Connect-Maester -Service ExchangeOnline
SecurityCompliance	EOP	Connect-Maester -Service SecurityCompliance
SharePointOnline	SharePoint	Connect-Maester -Service SharePointOnline
Teams		Connect-Maester -Service Teams
Dataverse		Connect-Maester -Service Dataverse; counts as connected when Graph is
AzureDevOps		Connect-ADOPS
GitHub		Connect-Maester -Service GitHub
ActiveDirectory	AD	Connect-Maester -Service ActiveDirectory

A test that is not connected to one of its services is Skipped with reason ServiceNotConnected. ActiveDirectory is opt-in: its tests are NotRun with reason OptInServiceNotConnected until you connect it. A custom test that names a service the registry does not know is Skipped with reason ServiceNotRegistered.

License

License lists Microsoft service plan names: the servicePlanName values of the tenant's subscribedSkus. The tenant needs any one element; inside an element, & means all of the joined plans.

Use the service plan, not the product. Microsoft lists both in Product names and service plan identifiers for licensing: take the name from the Service plans included column, such as AAD_PREMIUM_P2. The Product name and String ID columns, such as Microsoft 365 E5 and SPE_E5, are products. A product is a bundle that Microsoft re-packages over time, and the same feature ships in many of them, so a test that named a product would be skipped in tenants that have the feature through another one. The test builder has a licence explorer that finds the service plan from a product name, a friendly name or a GUID.

License = 'AAD_PREMIUM_P2'                        # Entra ID P2
License = ('AAD_PREMIUM', 'INTUNE_A')             # Entra ID P1 (or better) or Intune
License = 'AAD_PREMIUM_P2&INTUNE_A'               # Entra ID P2 and Intune

Maester maps these tokens to the service plan and SKU IDs it has always used to detect each product, so government, education and suite variants match too:

Token	Tenant has
AAD_PREMIUM	Entra ID P1 or better (P1, P2 or ID Governance)
AAD_PREMIUM_P2	Entra ID P2
Entra_Identity_Governance	Microsoft Entra ID Governance
AAD_WRKLDID_P1, AAD_WRKLDID_P2	Workload ID P1 or better, Workload ID P2
ATP_ENTERPRISE	Defender for Office 365 Plan 1 or Plan 2
THREAT_INTELLIGENCE	Defender for Office 365 Plan 2
EOP_ENTERPRISE	Exchange Online Protection
EXCHANGE_S_STANDARD	Any Exchange Online mailbox licence
EXCHANGE_DLP	Exchange Online DLP (Exchange Online Plan 2, the DLP plan, or Microsoft 365 Business Premium)
M365_ADVANCED_AUDITING	Microsoft 365 Advanced Auditing
DEFENDER_XDR	Any Defender XDR workload (Endpoint P2, Office 365 P2, Identity or Cloud Apps)
LOCKBOX_ENTERPRISE	Customer Lockbox
INTUNE_A	Microsoft Intune

EXCHANGE_DLP and DEFENDER_XDR are Maester's own tokens for a group of plans; every other token in the table is the service plan's own name.

Any other token is compared literally with the tenant's service plan names, so a custom test can use any plan name.

For the tests shipped with Maester this is checked. The unit test "Names a service plan in License, not a product" in powershell/tests/general/NativeTestAttributes.Tests.ps1 fails when a License value is neither a name from Microsoft's Service plans included column nor a token of powershell/assets/MaesterLicenseTable.psd1. The list of Microsoft's names is website/src/data/license-service-plans.json, which a scheduled workflow refreshes from the Microsoft page every day. Only enabled subscriptions count. When the licences cannot be read (no Graph connection, or no permission), the licence check is not applied and the test runs.

TenantType and Cloud

TenantType (Workforce or External) and Cloud describe where a test is meaningful. Maester detects both and records them in the result's TenantContext, but by default does not skip tests on them. A run turns enforcement on with Environment.Enforce in the run configuration. Declare them now if you know them; a test without TenantType is treated as Workforce.

Platform

Declare Platform = 'Windows' when the test calls a Windows-only command or module. On other operating systems it is Skipped with reason PlatformMismatch. Most tests run everywhere and declare nothing.

Parameters

A test's param() block lists the values a user can tune per tenant: thresholds, switches, objects to exclude. Values come from the run configuration or from Invoke-MtTest -Parameter; a test always has to run with none set, using its defaults.

param(
    # Longest validity period, in days, that a certificate may be issued for.
    [ValidateRange(1, 3650)]
    [int] $MaximumValidityDays = 365,

    # Members of these groups are exempt from this check.
    [MaesterParameter(Kind = 'Entra.Group')]
    [string[]] $ExcludedGroups,

    # How strictly to evaluate.
    [ValidateSet('Strict', 'Lenient')]
    [string] $Mode = 'Strict'
)

Rules:

• Types are int, bool, switch, string or string[].
• Defaults are constants. No parameter is mandatory.
• Every parameter has a description: a # comment on the line directly above it, or .PARAMETER help. Only the comment line nearest the parameter is read, so keep the comment to one line or use .PARAMETER. Get-MtTest shows the description.
• Give thresholds a ValidateRange. A ValidateSet makes the parameter a list of choices.
• Instance and every name that starts with Mt belong to the engine and cannot be set.

Set values in the config:

{
  "TestSettings": [
    { "Id": "CONTOSO.1002", "Parameters": { "ReasonMarker": "Off:" } },
    { "Id": "MT.1198", "Parameters": { "MaximumValidityDays": 180 } }
  ]
}

The engine checks each value against the param() block before the test runs. A name the test does not have, a fraction for an int, a number for a string, a string for a bool or switch, or a value outside the ValidateRange or ValidateSet gives that test an Error row with reason InvalidConfiguration; the run carries on. A string[] parameter accepts one string or a list. The effective values, and where each came from (Default, Config or Parameter), are written to the result row's Parameters.

Parameter kinds

[MaesterParameter(Kind = '...')] says what a string parameter holds, so a user interface can offer a picker and the engine can check the value's shape:

Kind	Value
Entra.User	User object ID
Entra.Group	Group object ID
Entra.ServicePrincipal	Service principal object ID
Entra.Application	Application object ID
Entra.ConditionalAccessPolicy	Conditional Access policy ID
Entra.NamedLocation	Named location ID
Entra.Domain	Domain name
Entra.DirectoryRole	Directory role definition ID
Azure.Subscription	Subscription ID
Azure.ResourceGroup	Resource group ID (/subscriptions/<id>/resourceGroups/<name>)
Azure.Resource	Azure resource ID

A value that does not match the kind's pattern (for example a malformed GUID) is InvalidConfiguration. In the config a value can be the ID itself or an object { "Id": "...", "DisplayName": "..." }; the test always receives the ID.

Families: one test, several results

Some checks produce one result per item: per drift folder, per recommendation, per user. Declare InstanceSource with the name of a function in the same file. The engine calls that function when the test runs (never during discovery) and runs the test once per object it returns, passing the object as $Instance.

function Get-ContosoAdminUnit {
    [CmdletBinding()]
    param()
    foreach ($unit in Invoke-MtGraphRequest -RelativeUri 'directory/administrativeUnits') {
        [pscustomobject]@{
            Id    = $unit.id                     # required: the instance suffix
            Title = "Administrative unit '$($unit.displayName)' is restricted"   # optional
            Data  = $unit                        # optional: anything the test needs
        }
    }
}

function Test-ContosoAdminUnitRestricted {
    [MaesterTest(
        Id = 'CONTOSO.2001',
        Title = 'Administrative units are restricted.',
        Severity = 'Medium',
        Service = 'Graph',
        InstanceSource = 'Get-ContosoAdminUnit'
    )]
    [CmdletBinding()]
    [OutputType([bool])]
    param(
        # The administrative unit to check, supplied by the engine.
        $Instance
    )

    $restricted = [bool]$Instance.Data.isMemberManagementRestricted
    Add-MtTestResultDetail -Result "Member management restricted: $restricted"
    return $restricted
}

The instance object has:

Property	Required	Meaning
Id	Yes	The suffix. The row's ID is <parent ID>.<Id>, for example CONTOSO.2001.4f6c.... It must match ^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$ and be unique within the family.
Title	No	The instance's title. Defaults to -TestTitle from the helper, then the attribute's title.
Severity	No	The instance's severity, used when the config sets none.
Tag	No	Extra tags written to the instance's row.
Data	No	Anything the test needs.

Each instance row carries ParentId (the family's ID) and InstanceId. When the family does not produce instance rows, the result has one row on the parent ID instead:

What happened	Row on the parent ID
The family was not selected, was disabled, or a gate failed	The usual NotRun or Skipped reason
The instance source returned nothing	Skipped with NoInstances
The instance source threw, did not finish within the family's timeout, or ended with exit	Error with InstanceSourceFailed
An instance Id is missing, invalid or repeated	Error with InvalidInstanceId

The instance source runs under the same timeout as the test (the family's TimeoutSeconds, or the run's Execution timeout). The source gets that long to return, and then each instance gets that long to run.

Selection and configuration understand families:

• -TestId CONTOSO.2001 or -TestId 'CONTOSO.2001.*' selects the whole family.
• -TestId CONTOSO.2001.abc runs only that instance; the others get NotRun rows with reason NotSelected.
• -ExcludeTestId and "Enabled": false on an instance ID drop that instance after the source has run.
• A TestSettings row on an instance ID wins over a row on the parent ID. Its Parameters and TimeoutSeconds apply to that instance only; a parameter it does not name keeps the value of the family's row.
• IDs under a family's parent are never reported as unknown, even if that instance does not exist in this run.

Settings shared by many tests

Read tenant-wide settings from GlobalSettings in the config with Get-MtSetting:

$accounts = Get-MtSetting -Name EmergencyAccessAccounts
$siteUrl  = Get-MtSetting -Name 'Contoso.SiteUrl' -Default 'https://contoso.sharepoint.com'

When the config does not set the value, Get-MtSetting returns Maester's default for the settings it knows, else -Default. Use a Namespace.Key name for your own settings. Prefer a test parameter when the value belongs to one test.

What a custom test can call

Each custom test file is loaded into its own private module. It can call every command Maester exports, including Invoke-MtGraphRequest, Add-MtTestResultDetail, Get-MtSetting, Get-MtTenantContext, Test-MtConnection, Get-MtLicenseInformation, Get-MtUser, Get-MtConditionalAccessPolicy and Invoke-MtTest. It cannot call Maester's private functions, and it cannot replace a Maester command.

Functions defined in the same file are visible to the test. Functions in another test file are not. To share code between files, put it in a helper .ps1 (one that is not named Test.*.ps1) and dot-source it from inside the test function: . "$PSScriptRoot/Contoso.Helpers.ps1". $PSScriptRoot is the test file's folder.

To reuse a built-in check, run it through the engine instead of calling its function:

$row = Invoke-MtTest -Id MT.1005
$row.Result   # Passed, Failed, Skipped, ...

The built-in check functions are no longer exported in 3.0, and they no longer contain their own connection checks, so calling one directly is not supported.

A suite manifest for your folder

A suite.json in a test folder sets what is true for every test in it (and its subfolders, unless a nearer suite.json exists):

{
  "Id": "Contoso",
  "Name": "Contoso security baseline",
  "Source": "Custom",
  "Tags": ["Contoso"],
  "HelpUrlTemplate": "https://wiki.contoso.com/maester/{Id}",
  "RequiresMaester": "3.0"
}

Key	Effect
Id	Written to each result row as Suite, which -Suite selects by. Defaults to Custom.
Name	The Block of the result rows of tests that have neither Product nor Feature.
Source	Written to each result row as Source. Defaults to Custom.
Tags	Matched by -Tag and -ExcludeTag for every test in the folder. Not written to the result rows.
HelpUrlTemplate	The HelpUrl of tests that do not set one; {Id} is replaced.
RequiresMaester	Tests in the folder need at least this Maester version. On an older version each one is an Error row with reason RequiresNewerMaester.
NoResultReason	The skip text shown when a test returns $null without calling Add-MtTestResultDetail. Defaults to "The test returned no result."
Frameworks	The frameworks whose catalogue is in the folder. See Benchmarks and framework files.

A single file can state the same as RequiresMaester with #Requires -Modules @{ ModuleName = 'Maester'; ModuleVersion = '3.1' }.

Benchmarks and framework files

A test says which requirement of a framework it implements in its own attribute, with Control:

[MaesterTest(
    Id      = 'CONTOSO.BL.1.1',
    Title   = 'Admins sign in with phishing-resistant MFA',
    Control = 'contoso-baseline:1.1'
)]

A Control value is <framework-key>:<requirement-id>. This is the only way a test is mapped to a framework. It is the same for the CIS and CISA tests that ship with Maester (Control = 'cis-m365:5.1.5.1', Control = 'cisa-scuba:MS.AAD.3.1') and for a test of your own.

The frameworks that ship with Maester, with the key to use in Control:

Key	Framework	Framework file	Declared in
cis-m365	CIS Microsoft 365 Foundations Benchmark	tests/cis/frameworks/cis-m365.json	tests/cis/suite.json
cis-github	CIS GitHub Benchmark	tests/cis/frameworks/cis-github.json	tests/cis/suite.json
cisa-scuba	CISA SCuBA Secure Configuration Baselines for Microsoft 365	tests/cisa/frameworks/cisa-scuba.json	tests/cisa/suite.json

Open a file to see the requirement IDs you can reference and the version Maester implements. NIST SP 800-53 has no framework file; its control IDs are in powershell/assets/MaesterNistControls.json and are written nist-800-53-r5:<control>. The test builder looks all of them up.

A framework file is the catalogue of one framework: its name, its version and every requirement, with the facts about each. A suite that holds a framework file declares it in its suite.json:

{
  "Id": "Contoso",
  "Name": "Contoso security baseline",
  "Frameworks": [
    { "Key": "contoso-baseline", "Version": "2.1" }
  ]
}

Key	Content
Key	The key of the framework: lowercase letters, digits, dots and hyphens. It names the framework file, it is the first part of a Control value, and it is written to the result.
Version	The version of the framework. When omitted, the Version of the framework file is used.

The file is frameworks/<Key>.json, beside the suite.json:

{
  "Framework": "contoso-baseline",
  "Name": "Contoso Security Baseline",
  "Version": "2.1",
  "Url": "https://wiki.contoso.com/baseline",
  "Requirements": [
    { "Id": "1.1", "Level": "L1", "Related": ["nist-800-53-r5:IA-2(1)"] },
    { "Id": "1.2", "Level": "L2" }
  ]
}

• Framework is the Key from the suite.json. Name is the name of the framework and Url is where it is published. A Version here must be the one the suite.json declares.
• A requirement has an Id, written as a string. It is the second part of a Control value, and it is matched exactly, including upper and lower case.
• Related lists controls of other frameworks that the requirement relates to, written <framework-key>:<control-id> with a lowercase key, like a Control value.
• Any other key is a fact about the requirement, such as Level, Profile or Criticality. Keep to identifiers and facts. Do not copy the text of a benchmark into the file: it is often licensed, and the test's own title already says what is checked.
• The file does not list tests. A Tests key on a requirement and an IdPrefix key in the suite.json are ignored.

What the result gets

Maester looks up each Control value of a test, in the order they are written. A value resolves when its key is a framework that has a file and its ID is a requirement in that file.

• Benchmark is built from the first value that resolves: the Framework, its Version, the Requirement and the requirement's own facts. When no value resolves, Benchmark is null.
• Control holds the values the test wrote, followed by the Related references of every requirement that resolved, each once.

Get-MtTest shows both. See the result schema.

A framework is found by its key, whatever suite declares it. A custom test can name a framework that ships with Maester (cis-m365, cis-github, cisa-scuba) or one that a suite in your own test folder declares. Two suites must not declare the same key: Maester warns, names both suite.json files and uses the first, and a framework that ships with Maester comes before one of yours.

A Control value whose key no framework file has, such as nist-800-53-r5:IA-2(1) or iso-27001-2022:A.5.17, is written to the result as it is. It needs no file.

A problem with a framework never stops a run. Maester writes a warning that names the file and what is wrong:

• The file is missing or is not valid JSON, its Framework or Version differs from the suite.json, a requirement has no Id or an Id that is listed twice, or a Related reference is not written <framework-key>:<control-id>. No test gets a Benchmark or Related controls from that framework.
• A Control names a framework that has a file and a requirement that is not in it, usually a typing error. That value gives no Benchmark; the other tests keep theirs.

Mapping a test to a framework

The test	What to write	What the result gets
Implements a requirement of a benchmark that has a framework file (CIS.M365.5.1.5.1, CONTOSO.BL.1.1)	Control = 'cis-m365:5.1.5.1'. If the requirement is new, add it to the framework file.	Benchmark, and the reference in Control
Exists for its own reasons and also supports such a requirement (a Maester test that checks what a CISA policy asks for)	Control = 'cisa-scuba:MS.AAD.3.1'	The same
Supports a control of a framework that has no file	Control = 'nist-800-53-r5:IA-2(1)'	The reference in Control

A test can list several values. Put the requirement the test is written for first: the first value that resolves becomes the Benchmark. Use the keys and requirement IDs of the framework files that ship with Maester so your results line up with the built-in ones. For a framework that has no file, choose a lowercase key such as nist-800-53-r5 or iso-27001-2022 and keep to it.

Why the framework file still exists

A key:id string names a requirement. It cannot hold what is true of the requirement whatever test implements it. The framework file holds that:

• the name and the version of the framework, in one place, so a new version of a benchmark is one edit;
• the facts about each requirement: the CIS level and profiles, the CISA criticality (SHALL or SHOULD), and where the policy stands upstream (current, moved or removed);
• the Related controls, such as the NIST SP 800-53 controls CISA maps each policy to, so that every test of the requirement carries them without repeating them;
• the full list of requirements. A requirement that no test names is a gap you can see, and a Control that names a requirement the list does not have is a typing error that Maester reports.

For the tests that ship with Maester the module build fails on such an error, and the unit tests check that every CIS and CISA test names a requirement of its own suite's framework.

A Benchmark or a Control says that a test supports a requirement. It does not say that a tenant complies with the framework.

Running and inspecting tests

Get-MtTest

Get-MtTest lists tests and their metadata without connecting to a tenant and without running any test code.

Get-MtTest -Product entra -Feature conditional-access |
    Format-Table Id, Title, Severity                             # built-in tests
Get-MtTest MT.1198 | Select-Object -ExpandProperty Parameters    # a test's parameters
Get-MtTest -Path ./custom | Where-Object { -not $_.IsValid } |
    Select-Object Id, File, Errors                               # validate your own tests

Without -Path it returns the tests that ship with Maester. With -Path (a file or a folder) it reads and validates those files; IsValid is $false and Errors lists every problem with its line number. -Id accepts exact IDs and * wildcards. -Product, -Feature, -Area, -Severity and -Suite select by attribute, as they do on Invoke-Maester, and -Tag matches what a run with that tag would select.

Invoke-MtTest

Invoke-MtTest runs tests through the engine, with the same gates, parameter checks and result rows as Invoke-Maester, and returns the rows. It writes no report.

Invoke-MtTest -Path ./custom/Test.CONTOSO.1002.ps1
Invoke-MtTest -Path ./custom/Test.CONTOSO.1002.ps1 -Parameter @{ ReasonMarker = 'Off:' }
Invoke-MtTest -Path ./custom                       # every native test in the folder
Invoke-MtTest -Id MT.1005, 'CISA.MS.AAD.3.*'       # built-in tests
Invoke-MtTest -Path ./custom -Id CONTOSO.2001.abc  # one instance of a family

• -Id takes exact IDs or * wildcards. Without -Path it selects built-in tests; with -Path it selects among the tests in that file or folder. A family instance ID runs only that instance.
• -Parameter is allowed only when exactly one test is selected. Its values win over the config.
• -Config takes a path, an object or a list of them, as for Invoke-Maester. Without it, a test that calls Invoke-MtTest during an Invoke-Maester run uses the run's configuration; otherwise Maester reads the config files found from -Path (or the current folder).
• Tests you name run even if they are preview or long-running tests, and Selection.DefaultAction is ignored. A test disabled in the config is still NotRun.

Dot-sourcing a test file at the prompt defines the function but gives it no engine: no service checks, no error handling and no .md. Use Invoke-MtTest while you write a test.

Console output

Invoke-Maester -Verbosity Normal prints one line per finished native test with its result, ID, title and duration, plus the first line of the reason for Failed and Error. -Verbosity Detailed also prints Running <ID> when a test starts. The default, None, prints only the summary.

Next steps

• Formatting test results
• Advanced guide: Invoke-MtGraphRequest, caching and other helpers
• Applicability and reason codes
• Run configuration
• Pester-format tests

For inspiration, see the community's custom Maester tests by Clayton Tyger (written for 2.x in Pester format; they still run, and Convert-MtTest can convert them).