TelemetryCollectionManager

CLI tool to convert YAML custom collection rule definitions into Defender for Endpoint (MDE) rule JSON, validated against the bundled schema model/mde-model.json. An included JSON Schema (rule-schema.json) enables VS Code IntelliSense.

Features

• Validate all YAML files to work with the Defender model.
• Create and update custom collection rules using these YAML files in the Defender portal.
• List or export all custom collection rules in the Defender portal.
• Convert exported rules to YAML files, in case you've built several in the portal already and want to transition to maintaining them in YAML.
• Delete individual or all custom collection rules in the Defender portal
• VS Code schema assistance via $schema header

Authoring YAML Rules

Place one or more YAML files in the rules/ directory. Each file describes a single rule. Start with the schema header for editor assistance:

# yaml-language-server: $schema=../rule-schema.json
name: Example Connection Success Rule
description: Detects non test.exe process making a successful connection
enabled: true
platform: Windows
scope: Organization
table: DeviceNetworkEvents
actionType: ConnectionSuccess
filters:
  operator: And
  expressions:
    - filter: NotEquals
      source: InitiatingProcessFileName
      values: ["test.exe"]
    - filter: Equals
      source: LocalPort
      values: [443]

Additional Examples

See the following in Examples/:

File	Purpose
minimal.yaml	Smallest valid rule (single predicate)
numeric-columns.yaml	Demonstrates numeric validation and multiple numeric values
deep-nesting.yaml	Arbitrarily deep nested logical groups (And/Or)
operators-showcase.yaml	Shows every supported operator
single-value.yaml	Demonstrates using a single scalar for values instead of an array

Structure Reference

Top-level fields:

• name (string) – Rule display name
• description (string) - Optional, free-form description
• enabled (bool)
• platform (string) – Must exist in schema (e.g. Windows)
• scope (string) – Typically Organization or use Asset group tags
• table (string) – Must match a schema table
• actionType (string) – Must be valid for the table
• filters (object) – Root filter group

Filter group:

filters:
  operator: And|Or
  expressions:
    - ...predicate or nested group...

Predicate expression fields:

• filter – One of: EndsWith, Contains, Equals, StartsWith, NotContains, DoesNotEndWith, DoesNotStartWith, NotEquals
• source – Column name from schema for the chosen table/actionType
• values – Array of one or more scalars (string or number). Numeric columns may use unquoted numbers.
  • You may also provide a single scalar (string or number) instead of an array; it will be wrapped automatically. See single-value.yaml.

Nested group (inline via group:):

- group:
    operator: Or
    expressions:
      - filter: Equals
        source: InitiatingProcessFileName
        values: ["example.exe"]
      - group:
          operator: And
          expressions:
            - filter: DoesNotStartWith
              source: InitiatingProcessCommandLine
              values: ["powershell"]

Validation Rules

• Platform, table, actionType, and column names must exist in the loaded schema.
• Operators must be from the supported list.
• Numeric columns reject non-numeric values (including malformed numeric strings).
• A nested expression (group:) must not specify any predicate fields (filter, source, or values). Doing so is an error.
• An empty nested group yields a validation warning (not an error) so you can iteratively build complex logic.
• The root or any group must have a valid logical operator (And or Or).

Tips

• Group logically related predicates together; use nested groups to express precedence.
• Prefer narrower predicates early (helps readability; tool currently doesn't optimize order).
• Keep filenames descriptive (e.g., network-block-temp.yaml). Filenames are not used in output JSON but aid maintenance.

Common Errors & Fixes

Error	Cause	Fix
unknown table	Table misspelled or not in schema	Use a valid table from model/mde-model.json
unknown actionType	Wrong action for table	Confirm actionType belongs to the table in the schema file
unknown column	Column not present for that actionType	Check columns under the actionType in schema
unsupported filter operator	Typo or removed operator (e.g. old In/NotIn)	Use one from supported list
value 'x' for numeric column ... is not a number	Non-numeric literal provided	Remove quotes or correct value

Also, in some cases between exporting and importing there may be minor differences in casing of some fields. This is because the API does not fully complies with its own model. These need to be manually fixed for now. The YAML schema and local validation should help with this.

Deleting rules over the API might sometimes throw a 500 error. If this happens, try again, it may have something to do with the backend not being able to keep up with the rate of rule deletions. I've seen this happen very rarely, so it's not a big deal.

Output Semantics

Generated JSON assigns new ruleId, timestamps, and leaves createdBy / lastModifiedBy blank for MDE to populate later. The tool does not currently deduplicate or merge rules.

Usage

Acquiring a token

You will need an access token to call the Defender API. Sadly at the time of writing, the only way to authenticate right now is to use a client bearer token. This means you will need to get it from a user session in the portal. The API currently does not support service principals. :(

If AZURE_TOKEN (or the legacy ACCESS_TOKEN) is not set in your environment or .env, the tool automatically reuses the currently logged-in Azure CLI session by running the equivalent of az account get-access-token --resource https://securitycenter.microsoft.com/mtp. Log in via Azure CLI with an account that has Defender for Endpoint permissions and the token will be fetched on demand.

When a device login is not allowed, create a new tab and open the F12 developer tools. Navigate to the API Explorer in the Defender portal. (https://security.microsoft.com/interoperability/api-explorer)

In the Network tab, filter for getToken?resource=MATP&serviceType= and get the Token from the Response Body.

File placement and naming

• Organize rules under rules/<TableName>/. For example:
  • rules/DeviceFileEvents/drivers_added.yaml
  • rules/DeviceNetworkEvent/ports_suspicous.yaml
• The file name does not have to match name, but keeping them aligned helps tracking.
• Include the $schema directive at the top of your YAML for IDE validation and autocompletion:

# yaml-language-server: $schema=../rule-schema.json

Adjust the relative path depending on the folder depth of your rule file.

Validation and suggestions

The tool can validate rules without posting them to Defender for Endpoint.

./TelemetryCollectionManager --validate-only

If validation errors occur, they are printed and the process exits non-zero without writing output. You can capture diagnostics (errors and warnings) to a file:

./TelemetryCollectionManager --validate-only --diagnostics diagnostics.json

Post rules to Defender for Endpoint

The tool can post the validated rules directly to the Defender API. It will post all rules in the rules/ directory or the specified folder. This includes rules that are set to disabled.

Example:

export AZURE_TOKEN="<paste bearer token here if you dont have an azure cli session logged in>"
./TelemetryCollectionManager --post-rules

Optional flags you may find useful:

• --api-base https://wdatpprd-weu.securitycenter.windows.com to target a specific cloud/region (default shown).
• --rules YourFolder to point to a different rules directory (defaults to rules).

Expected output will show per‑rule POST results. On success you will see HTTP 200/201 responses per rule.

Delete rules from Defender for Endpoint

You can delete specific rules by ID, or delete all rules in the tenant. Be careful: deletions are irreversible.

1. List current rules and capture their IDs:

./TelemetryCollectionManager --list-rules

2. Delete specific rules by ID (comma‑separated):

./TelemetryCollectionManager --delete-rules "11111111-1111-1111-1111-111111111111,22222222-2222-2222-2222-222222222222"

3. Delete all rules in the tenant:

./TelemetryCollectionManager --delete-rules=all

Notes:

• If an ID cannot be deleted, the tool prints an error for that ID and continues with the rest.
• When no valid IDs are provided, the tool gives a helpful hint and prints how many rules currently exist.

Output

Produces an array of rule objects similar to model/rule-example but with fresh IDs and timestamps. Metadata fields createdBy and lastModifiedBy are intentionally empty (populated later by the Defender API).