mtf-facts is an offline Go CLI that parses Terraform module trees and reports deterministic facts: rule results, candidate findings with evidence, interface and schema diffs.
It serves two uses:
- Module authoring:
lint, a provider coverageledgercheck, andschema-difffor provider upgrades. - Module review: a facts bundle that lets a model reviewer skip the rules a parser can decide, and
fixture-check, the allowlist a verification ladder scratch tree passes before Terraform runs on it.
It reads files only. It makes no network call and runs no Terraform or other program.
With Homebrew, on macOS or Linux:
brew install modulestf/tap/mtf-factsOr with Go:
go install github.com/modulestf/facts/cmd/mtf-facts@latestPin a release with @v0.4.0 in place of @latest, or download a release binary and check it as in Verify a release.
Or build from a checkout:
go build -trimpath -o bin/mtf-facts ./cmd/mtf-factsCheck a whole module tree:
mtf-facts lint path/to/moduleCheck only the areas a change touched, against the tree the change started from:
mtf-facts lint path/to/module --base path/to/base-treeOr give the change itself, a GitHub pull request files list or a unified diff, when you have no base tree:
mtf-facts lint path/to/module --diff files.jsonWrite the full report as JSON:
mtf-facts lint path/to/module --base path/to/base-tree --json > report.jsonOr the AI report a model reviewer reads, which leaves out what a reviewer never decides on (see AI report):
mtf-facts lint path/to/module --base path/to/base-tree --json --ai > report.jsonlint exits with one of these codes:
| Code | Meaning |
|---|---|
| 0 | No findings, and every rule was decided in every area. |
| 1 | At least one finding. |
| 2 | An error: bad input, a limit, the timeout, or a failed integrity check. |
| 3 | No findings, but at least one rule was not_evaluated in some area. |
Candidates are unconfirmed: the author confirms or rejects each one, and they never change the exit code.
mtf-facts --help lists the commands, and mtf-facts <command> --help gives the flags and exit codes of one. Help goes to stdout and exits 0. Long options take two dashes (--json); the single-dash form (-json) is not accepted. Every command exits 2 on an error. Ignore stdout on exit 2: a failed write to stdout can leave partial output.
These commands check pull request #780 of terraform-aws-modules/terraform-aws-lambda. To check a change to your own module, put your repository and branch in place of the pull request.
Get the rule registry and the terraform-aws-modules module profile from modulestf/modulestf, at the commit this repository's CI pins:
git clone https://github.com/modulestf/modulestf.git skills
git -C skills checkout 3bd9cf8bb175f2db92f82409544b8fda9efa90bd
REGISTRY=skills/.agents/skills/terraform-module-reviewer/references/rule-facts.md
PROFILE=skills/.agents/skills/terraform-module-maintainer/profiles/terraform-aws-modulesCheck out the head of the pull request and the commit it branched from, side by side:
git clone https://github.com/terraform-aws-modules/terraform-aws-lambda.git lambda
git -C lambda fetch origin pull/780/head
HEAD_SHA=$(git -C lambda rev-parse FETCH_HEAD)
BASE_SHA=$(git -C lambda merge-base origin/master FETCH_HEAD)
git -C lambda worktree add ../head "$HEAD_SHA"
git -C lambda worktree add ../base "$BASE_SHA"Write the provider schema, so the coverage.* rules run. This step runs Terraform and downloads the providers; mtf-facts does neither:
(cd head && terraform init -backend=false >/dev/null && terraform providers schema -json) > schema.jsonLint the areas the pull request touched:
mtf-facts lint head --base base \
--registry "$REGISTRY" --profile-dir "$PROFILE" --schema-files schema.jsonWith v0.4.0-rc.1 it prints this and exits 1 (the not evaluated lines are cut):
examples/fixtures: MEDIUM examples.versions-missing: Example "examples/fixtures" has no versions.tf, so its Terraform and provider constraints are not in the file the profile and its sibling examples use.
fix: Add `versions.tf` to `examples/fixtures`, copied from a sibling example, or rename the file that holds its `terraform` block to `versions.tf`.
examples/function-url/main.tf:45: unconfirmed structure.version-floor-understated: examples/function-url:required_providers.external, confirm or reject
examples/function-url/main.tf:45: unconfirmed structure.version-floor-understated: examples/function-url:required_providers.local, confirm or reject
examples/function-url/main.tf:45: unconfirmed structure.version-floor-understated: examples/function-url:required_providers.null, confirm or reject
scope: change from base; 4 areas checked, 0 calling a changed module, 18 left out: examples/alias, examples/async, examples/build-package, examples/code-signing, examples/complete, examples/container-image, examples/deploy, examples/event-source-mapping, examples/multiple-regions, examples/runtimes, examples/simple, examples/simple-cicd, examples/triggers, examples/with-efs, examp... (--full-tree checks every area)
1 findings, 3 candidates; manifest: 1 finding, 1 candidate, 76 clean, 2 not_evaluated, 68 not_applicable
Fix each finding, and confirm or reject each candidate. The two rules left not_evaluated need the declared minimum provider version, which a raw schema file does not record.
For the JSON report a hosted review writes for the same two commits, add --json and the commits:
mtf-facts lint head --base base --json \
--registry "$REGISTRY" --profile-dir "$PROFILE" --schema-files schema.json \
--head-sha "$HEAD_SHA" --merge-base-sha "$BASE_SHA" > report.json
jq '.findings[].summary, .candidates[].subject' report.jsonPass the rule registry and the module profile so the report records their hashes and the profile rules run, and the provider schema files so the coverage.* rules run:
mtf-facts lint path/to/module --base path/to/base-tree --json \
--registry rule-facts.md --profile-dir path/to/profile \
--schema-files schema.json > report.jsonWithout --profile-dir, the rules that need the profile are not_evaluated with profile_unavailable. Without a schema, a coverage.* rule is not_evaluated(schema_unavailable) in an area where the change adds or edits a resource or data block. See lint for each flag.
Add the commits the two trees came from, so the report records them as a hosted review does:
mtf-facts lint path/to/head --base path/to/base --json \
--registry rule-facts.md --profile-dir path/to/profile \
--head-sha <sha> --merge-base-sha <sha> > report.jsonEach of --registry, --head-sha and --merge-base-sha is recorded as null when its flag is absent, and a host that checks a report against its pins must treat null as invalid.
List the provider fields of each type the module declares, then check the ledger you wrote against that inventory:
mtf-facts ledger path/to/module --schema-files schema.json
mtf-facts ledger path/to/module --schema-files schema.json --check coverage-ledger.jsonWhen the ledger is incomplete, the check exits 1 and lists each problem in problems. See ledger.
Compare the provider schema of the current and the target version, for the types the module tree uses:
mtf-facts schema-diff --from current.json --to target.json --types-from path/to/moduleEach schema file is a terraform providers schema -json output. See schema-diff.
A hosted review runs the commands in this order:
lint --json, without--ai, on the head tree with--base,--registry,--profile-dir,--schema-handoff,--head-shaand--merge-base-sha.bundleon that report, with--head-sha,--merge-base-sha,--binary-sha256and the schema and profile flags the lint run had. It printsjudgmentand writes the facts bundle and the injected rule text, or printsfulland writes no file. Steps 3 to 6 run only afterjudgment.- The model reviewer reads the bundle and the injected text and writes its reply.
merge checkon the bundle and the reply. Exit 1 means the reply needs the repair turn.merge final --registryon the bundle, the reply and, after a repair turn, the repair reply.merge countson the bundle and the same replies, read only aftermerge finalsucceeds.
Host contract:
- Exit code 2 means stdout is not a report. Ignore stdout on exit 2: a failed write to stdout can leave partial output.
- A timeout, or any error before output, writes nothing to stdout.
- Drain both stdout and stderr while the process runs, and keep your own wall-clock kill.
--timeoutdoes not replace it. - Pin one version and its SHA-256 per platform, and check the attestation. See Update a consumer pin.
mtf-facts lint <dir> [--base <dir> | --diff <file>] [--full-tree] [--json [--ai]] [--timeout <duration>]
[--registry <rule-facts.md>] [--profile-dir <dir>]
[--schema-files <file>]... [--schema-handoff <file>]
[--head-sha <sha>] [--merge-base-sha <sha>]| Flag | Meaning |
|---|---|
--base <dir> |
The tree the change started from. Only the areas the change touched are checked. |
--diff <file> |
The change as a file, to rebuild the base tree from, when there is no base tree. Not with --base. See Change as a diff. |
--full-tree |
Check every area, not only those the change touched. The change still decides which lines are judged. |
--json |
Write the full report as JSON. |
--ai |
With --json, write the AI report on one line. See AI report. The text report is the same with or without it. |
--timeout <duration> |
Stop a run that takes longer. Default 60s. |
--registry <rule-facts.md> |
Record the registry hash. |
--profile-dir <dir> |
The module profile directory. |
--schema-files <file> |
A terraform providers schema -json output. Repeatable. |
--schema-handoff <file> |
The schema handoff the schema job writes, gzipped or not. |
--head-sha <sha> |
Record the commit the head tree came from. |
--merge-base-sha <sha> |
Record the commit the base tree came from. |
The --json report holds the envelope, one manifest entry per rule and area (see Areas for a cross-area rule), findings in the reviewer's findings schema, and candidates. A structure.argument-order candidate carries predates_change: true when every item out of order in the head block, by name, is also out of order in the base block of the same address, so the same subject: the defect is older than the change. A change that puts a new item out of order does not predate it. No other rule sets it today; it is for a rule whose check already reads the base block. It also holds:
-
scope: how the change was given (none,baseordiff),full_tree, and three disjoint lists of areas.areas_checkedhave their own manifest entries.areas_callingare the other areas that call a module listed incallers.areas_left_outare the rest, so no area is dropped without a trace.module_areaslists every discovered area of kindmodule, at head or at base, sorted: the set that decides which area owns a file (see Module ownership). -
callers, with a change only: each root or submodule area the change touches whoseversions.tforterraformblock, orvariables.tfor avariableblock, changed, with its version floors at base (nullfor a new module) and at head, and every area that calls it by a local path: the call's file and line, whether that area is checked, the floors it declares, and the ids of thestructure.version-floor-understatedcandidates filed in it. A floor is anaddress(required_versionorrequired_providers.<name>), the providersourceand theconstraintas written. Callers underwrappers/are not listed: no area reads that directory. -
module_callers: always written,[]without a change. See Module ownership. -
in_repo_downgrade: always written, sorted, often[]. See In-repo downgrade. -
On a candidate,
downgrade: "in-repo"andseverities: ["MEDIUM"]when the in-repo downgrade covers it; absent otherwise. -
A manifest entry for a rule the profile's options gate is
not_applicablewithreason_codeprofile_optionand a reason text. See Profile options.
The text report prints a scope: line, and with a change one line per changed module with the floors the change moves, and one line per caller with its own floor at those addresses. The JSON schemas are in schema/.
--json --ai writes the parts of the report a reviewer decides on, as one line of JSON. The exit code is the same as without --ai. Compared with the full report:
manifestkeeps only itsfinding,candidateandnot_evaluatedentries, in the same order and shape.manifest_summaryis added.statuscounts every manifest entry of the full report by status.omitted_by_familycounts thecleanandnot_applicableentries left out, per rule family: the rule id up to its first dot, such asstructure.areaslists every area with a manifest entry, sorted. A rule with no entry in a listed area wascleanornot_applicablethere, or has no entry there in the full report either.envelope.inputs.headandenvelope.inputs.basekeeptree_sha256and givefile_countin place of thefileslist. Every other envelope key is unchanged,schema_inputswith its handoff hash, commits and tree hashes included.index,interface_diffandschema_coverageare left out.findings,candidates,scope,callers,module_callersandin_repo_downgradeare unchanged.
The report keeps schema_version 2 and the keys envelope, manifest, findings and candidates with their types, and manifest_summary is present only in the AI form. bundle and merge final --report need the index and the file lists, so they refuse a AI report and exit 2.
--diff reads a GitHub pull request files list, the JSON array gh api repos/<owner>/<repo>/pulls/<number>/files --paginate --slurp | jq add writes, or a unified diff as git diff writes it, with or without diff --git lines. The format is detected from the first byte. Paths must be relative to <dir>. The base tree is the head tree with each changed file rebuilt by reversing its hunks: a removed file comes back from its removed lines, an added file is left out, and a renamed file moves back to its old path. Every context and added line must match the head file, line ending included, or the run exits 2, and so does a path that is not clean and relative, a path listed twice, or a status the tool does not know.
A file the diff does not list is unchanged, even when it is too large to read or cannot be read: its base entry is the head's own. With --base, a file too large to read is compared by a SHA-256 of its bytes, read in a stream and never kept, up to 64 MiB a file and 256 MiB a tree; past those, or unreadable, it counts as changed. A file the diff gives no patch for, as GitHub does for a binary or very large file, has no known base content. It is in inputs.base with kind no_patch, and the configuration rules of its area are not_evaluated with no_base and a reason that says the diff has no patch for it. A GitHub rename with no patch and no added or removed line is a pure rename when the head file is text, as git judges it (no NUL byte in the first 8000 bytes), and a binary one is no_patch. A modified entry with no patch and no added or removed line is, for a text file, a change of its exec bit only: its base content is the head's. A removed .tf file with no patch and no removed line was empty, or binary and so no configuration: its base content is empty, unless a .gitattributes file of either tree names binary or -diff, which may hide the diff of a text file, and then it is no_patch. A unified diff's old mode and deleted file mode lines set the base exec bit, and a changed exec bit alone makes a path changed. The envelope records the diff in inputs.diff: its format, github-files or unified, and its SHA-256. A report from --diff has a base tree, so bundle accepts it.
Two schema files with one base name exit 2. The schema handoff and the schema files are read offline under size caps; a missing, oversized or malformed one exits 2. The handoff is read only when --head-sha, --merge-base-sha and the .tf tree hashes of the head and --base trees equal the ones it records; otherwise every lookup in it is schema_unavailable.
The envelope's schema_inputs records the handoff's hash, commits and tree hashes, and the hash of each schema file, named by its base name. With schema files and no handoff, it records the hash of each schema file and the handoff keys are null. handoff_sha256 hashes the file's bytes as read, so a gzipped and a plain copy of the same handoff hash differently: a host that checks it must hash the form it passed.
The coverage.* rules read the bound handoff when one is given, and never fall back to the schema files. With schema files and no handoff, as in local lint, they resolve each block's provider source from required_providers and read the files as the latest schema; a provider no file has stays schema_unavailable. A schema file records no provider version, so its schema facts carry the version schema-file, and coverage.argument-above-min-version, which needs the declared minimum, stays not_evaluated(schema_unavailable).
--profile-dir reads the directory's PROFILE.md profile-constants blocks and the templates they name, without following a link and with size caps. The output envelope records its profile_sha256, and profile_tree_sha256: the SHA-256 of sorted <sha256> <path> lines over every file of the directory, null when an entry is a link, special or unreadable, or when a name is not UTF-8 or holds a control character or DEL, which could forge a line of the hash input. profile-snapshot prints the same value as tree_sha256. A path that is missing, not a directory or itself a link exits 2; a link in a parent component of the path is resolved. A profile that cannot be read or parsed is no error: the rules that need it are not_evaluated with profile_unavailable, as they are without --profile-dir.
--timeout must be greater than zero. A run that takes longer writes mtf-facts: timeout to stderr and exits with code 2.
The exit codes are in Quick start.
mtf-facts affected lists the examples a change can affect, most likely first, so a host can plan them in that order within its time budget.
mtf-facts affected <dir> [--base <dir> | --diff <file>] [--json] [--timeout <duration>]--base and --diff read the change as lint does (see Change as a diff). With --json it writes one JSON document in the shape of affected.schema.json:
{"schema_version": 1, "inert": false, "fallback": null,
"examples": [{"dir": "examples/c", "reason": "touched"}, {"dir": "examples/a", "reason": "caller"}, {"dir": "examples/b", "reason": "all"}],
"changed": ["examples/c/main.tf", "modules/log/main.tf"]}examplesholds every example: each first-level directory underexamples/with a.tffile in it. First those with a changed path under them (touched), then those whose local module sources reach a changed module directory, transitively and the root module included, as withsource = "../../"(caller), then the rest (all), each group in path order. Registry and git sources are not callers of the local tree; a local source written with backslashes, as on Windows, is read with slashes. A changed file Terraform loads by its directory (.tf,.tf.json,.tfvars,.tftest.hcl,.terraform.lock.hcl, or an entry that is not a regular file) is a change of that directory. Any other changed file, such as a script, a template or a README, is a change of every module whose read sites reach it, and of none when no read site does.inertis true only when every changed path is a documentation or metadata path (Markdown and other docs files,LICENSE,.gitignore, anything underdocs/or.github/that is not configuration), its content changed but not its exec bit, and no HCL read site in the whole tree reaches it. Read sites are the first argument offile,templatefile,filesetand the other file functions, the file calls inside every template atemplatefilecall reaches, the file attributes oflocal_file,archive_fileand the provisioners, and any other string that is a path frompath.module,path.rootorpath.cwd, or a plain literal resolved against the repository root and each example. A read site with a dynamic path, a shell command (local-execcommandandinterpreter,externalprogram), a template that does not parse, and a path that leaves the tree, such as${path.module}/../x, reach every file. A docs file a read site reaches is not inert, so a README a module reads withfile()makes its callerscaller.- A change of the exec bit alone is a change. The exec bit comes from the file mode with
--base, fromold modeanddeleted file modelines in a unified diff, and for a GitHub files list entry that is modified with no patch and no added or removed line, from the head file with its bit flipped when the file is text. Such an entry for a binary file, like any file the diff gives no patch for, is never inert. The tree hashes alintreport records do not include the exec bit, so they stay as they were. fallbacksays why the examples could not be ranked, and then every example has reasonallandinertis false:unparsedwhen a.tffile of the head tree does not parse or a.tf.jsonor override file is present,dynamic-sourcewhen a module call has no literal source,no-change-inputwhen neither--basenor--diffis given.changedlists the changed paths, both paths of a rename.
It exits 0 with the list, or 2 on bad input, a limit, an example directory whose name is outside [A-Za-z0-9._-], or the timeout, with nothing on stdout. Without --json it prints one <reason> <dir> line per example and a summary line.
mtf-facts depth decides how deep a review of a change must go: light, focused or deep. It reads the head tree and the merge base tree, the pull request files list, and the behaviour --json and affected --json reports of the same change, and never runs Terraform. Every rule fails closed: anything it cannot prove makes the change deep.
mtf-facts depth --head <dir> --base <dir> --files <files.json> [--classify <behaviour.json>] [--affected <affected.json>] [--examples <dir,dir>] [--timeout <duration>]--files is GitHub's pulls/<n>/files list (filename, status, previous_filename, additions, deletions). --examples names example directories the host knows of besides those of the affected report; each is selected unless proven out of scope. It prints one JSON object:
{"format": 1, "depth": "focused", "reasons": ["focused-within-limits"],
"checks_out_of_scope": [], "examples_selected": ["examples/complete"], "examples_out_of_scope": [],
"touched_types": ["aws_rds_cluster"], "walk": {"complete": true, "unwalked": 0},
"limits": {"tf_files": 1, "changed_lines": 2, "changed_blocks": 1, "selected_examples": 1},
"schema_providers": ["base:.:aws=hashicorp/aws:>=5.0", "head:.:aws=hashicorp/aws:>=5.0"],
"schema_types": [".:resource:aws_rds_cluster:hashicorp/aws"], "schema_locks": []}- The changed paths are every path whose entry differs between the trees and every path the files list names; a list that disagrees with the trees is
deep-files-list-mismatch. Terraform-read files are.tf,.tf.json,.tfvars,.tfvars.json, any.hcl(.tftest.hcland.terraform.lock.hclincluded),.tftest.json,.tfmock.json,.terraformrc,terraform.rcand*.tfrc, matched case-insensitively. Both trees are parsed, HCL and JSON forms; a Terraform-read file that does not parse, is not a regular file, or is an override file isdeep-parse-failure, andwalkcounts it. lightneeds all of: every changed path is Markdown (*.md),CHANGELOG*, under adocsdirectory at any level,LICENSE,.gitignoreor.editorconfig; none is Terraform-read; no file sink string of the head (below) resolves to a changed path or a directory holding one, and every one resolves; the head calls only Terraform's built-in functions (a provider-defined function blocks light); the classifier report is valid, of this change, andinert; at most 50 changed files, elsedeep-too-many-files. Out of scope at light: checks A, B and D and the Check G ladder (G-ladder); every example is listed inexamples_selected, for validate.focusedneeds a Terraform change with no deep class, at most 6 changed Terraform-read files, 200 changed lines (additionsplusdeletionsof every listed file), 4 changed resource, data and module blocks, and 12 selected examples, and every classifier trigger on the allow-list: an expression or literal change in a resource or data argument, a local, an output value or a module call argument; an added output; an unknown propagation candidate; an example.tfvarschange. Selected examples are those the affected report lists astouchedorcaller, those it does not list, those holding a changed path, and those whose local module calls reach a changed directory or the closure; the rest, reasonallin a report with no fallback, are out of scope. B is out of scope with no touched type; D is always in scope at focused, and with no example selected and none changed the reasons addfocused-no-example.deepis everything else, with every check and example in scope.- Changed values: every changed argument of a resource or data block, local, variable, output, module call or variables file assignment, and every argument whose file sink string resolves to a changed path. Each is followed to every consumer, through outputs to the calling module and through module call arguments into the callee's variables. A changed argument
Xof a managed resourceRaffects references toR.Xand toRas a whole (R,R[*],R[k], aforexpression over it); references to other attributes are not affected (a changeddynamic "X"block ofRis the argumentX), and every reference is whenRis added or itscount,for_eachorproviderchanges, or whenXdefinesR's identity: its whole name isname,name_prefix,bucket,bucket_prefix,identifier,identifier_prefix,cluster_identifier,cluster_name,function_name,family, or ends in_nameor_identifier. That rule is a heuristic for the computed attributes (id,arnand the like) a replacement changes; the plan of every selected example still shows a replacement. A changed variables file assignment is a changed value of that variable in its directory; one in a directory with no module, or naming a variable no module there declares, isdeep-unresolved-consumer. A data source is a query: a changed argument affects every reference to it. A reference insidedepends_oncarries ordering, not a value. A consumer the parser does not model (a variable validation, a precondition or postcondition,check,import,removed, an unknown block, a registry or git module, a module call whose source does not resolve) isdeep-unresolved-consumer. - File sinks: the path argument of the file functions (
file*,templatefile,fileset); the provisioner arguments that read files (the file provisioner'ssource, remote-exec'sscriptandscripts, local-exec'sworking_dirandinterpreter); and any argument whose whole name is one ofpath,paths,file,files,filename,dir,directory,source,source_dir,source_file,chart,config,config_path,config_paths,template,template_dir,program,script,scripts,content_dir,output_path, or ends in_path,_paths,_file,_files,_filename,_dir,_dirs,_directory,_template,_scriptor_scripts(such assource_pathoriam_role_path), at any nesting, in any block. A module call's arguments are sinks only for a registry or git module, sincefile()resolves against the working directory; a local module's sinks trace the value where it lands. Every string a sink can read must be a literal, orpath.moduleorpath.rootjoined with literals; anything else isdeep-unresolved-file-dependency. References are traced to their sources: a local to its expression; a variable to its default and every in-repo assignment (callers' arguments and the variables files of its directory; module calls under awrappersdirectory add none); a module output to its value in a local callee;each.keyandeach.value, a dynamic block's iterator and aforexpression's variables to the collection they iterate, attribute by attribute through objects,merge,flatten,concatand the other collection functions that pass elements on;tryandcoalesceto their arguments,lookupwith a literal key to the map's value at that key and its default, an index or a splat to the collection's elements,chompandtrimspaceto their argument trimmed. An encoding, hash, number, bool or time (jsonencode,md5,lengthand the like) is no path and passes. A variable with no default and no in-repo assignment is an external input and passes, as does a null, a number or a bool. An attribute of a managed resource or data source passes, except one oflocal_file,local_sensitive_file,archive_file,template_file,template_dirorexternal, which is traced to the argument of the same name. A string built in the repository (a template with any part that is not a path, or any other function) does not resolve. A local-execcommandis not resolved, but its literal text is searched for the name of every changed path and of every directory holding one, case-folded; a match isdeep-unresolved-file-dependency. A relative path resolves against every configuration directory, segment by segment as the operating system does: each symbolic link of the head is replaced by its target before a following..applies, and a link that is absolute, leads out of the tree or loops does not resolve. Paths are compared with the changed paths case-folded. A source of several expressions (a JSON list of strings) and an object with a computed key do not resolve. Tracing stops after 2,000,000 steps or at the deadline, and the change is thendeep-unwalked. A template atemplatefilecall reads is scanned for file function calls too. Strings outside sinks are not resolved. Tracing into registry and git modules is a later improvement: a changed value that reaches one isdeep-unresolved-consumer. - Paths: a changed path that is not printable ASCII, has an empty,
.or..segment, or a backslash; a changed link or special file; and two paths of either tree equal when case-folded aredeep-path-unsafe. A change under.githubor.circleci, toaction.ymloraction.yaml,.pre-commit-config.yaml,.pre-commit-hooks.yaml,.releaserc*,release.config.*or.gitlab-ci.ymlat any path isdeep-ci-config. touched_typesare the types of resource and data blocks with changed lines and those the closure reaches.limitsgives the counts the focused limits compare.schema_providers,schema_typesandschema_locksare the schema pass's tokens, in the grammar of its Inputs section:<rev>:<dir>:<local>=<namespace>/<name>:<constraints>per revision and configuration directory fromrequired_providers(<rev>:<dir>:!constraint_unparsablewhen it cannot be read or gives a local name two sources,<rev>:<dir>:<local>=!registry_hostfor another registry);<dir>:<resource|data>:<type>:<namespace>/<name>for every head resource and data block, bound by itsproviderargument, else its type's prefix, through that local name's source, elsehashicorp/<local>, none for the builtinterraformprovider;<dir>:<namespace>/<name>=<version>from each head.terraform.lock.hcl. A token that fails its pattern is left out. They are written at every depth.
Reasons are fixed tokens, written in this order. A host that raises a report to deep writes one of the host tokens after them, or deep-schema-<reason> for a schema pass failure; mtf-facts never writes those.
| Token | When |
|---|---|
light-docs-only |
the change is light |
focused-within-limits |
the change is focused |
focused-no-example |
focused, with no example selected and none changed: Check D stays in scope to report the missing example |
deep-terraform-block |
a terraform block changed, or a changed value reaches one |
deep-lock-file |
a .terraform.lock.hcl changed |
deep-cli-config |
a CLI configuration file changed |
deep-ci-config |
a CI, hook or release configuration file changed |
deep-provider-config |
a provider block, alias, provider or providers argument changed or is reached |
deep-depends-on |
a depends_on changed |
deep-new-type |
a resource or data type new to its directory |
deep-block-removed-or-renamed |
a resource, data, module, output or variable block removed or renamed |
deep-variable-contract |
a variable added, or anything but its description changed |
deep-output-sensitive |
an output's sensitive or ephemeral changed |
deep-lifecycle-moved-removed-import |
a lifecycle, moved, removed or import block, a precondition or postcondition, changed or reached |
deep-count-for-each |
a count or for_each changed or reached |
deep-provisioner |
a provisioner or connection changed or reached |
deep-policy |
an argument whose name contains policy, an object key or block named Statement or Principal (any case), an IAM resource or data source (a type with _iam_), a resource type ending in _policy or _permission, or a *_policy_document data source, changed or reached; a changed block counts only when such a name inside it, what it holds, or the for_each or iterator of a dynamic block around it differs between the revisions, and the keys of a variable's type are no such names |
deep-module-source-version |
a module call's source or version changed, or a module call added |
deep-dynamic-structure |
a dynamic block changed or reached |
deep-wrappers |
a file under a wrappers directory changed, or a changed value reaches an argument there other than a pass-through output |
deep-limits |
over a focused limit, or a files list entry with no line counts |
deep-classifier-category |
a classifier trigger outside the allow-list, or capped triggers |
deep-classifier-unavailable |
no classifier report |
deep-classifier-invalid |
a classifier report that does not decode as behaviour --json (an unknown key, a wrapper object, another schema_version) |
deep-classifier-mismatch |
a classifier report with no entry for a changed path: of another change, or with paths relative to another directory than the head tree's root |
deep-affected-fallback |
no affected report, or one with a fallback |
deep-affected-invalid |
an affected report that does not decode as affected --json (such as {}) or lists an unknown reason |
deep-parse-failure |
a Terraform-read file does not parse |
deep-unwalked |
a file, block or expression the parser cannot walk, or a limit of the walk |
deep-unresolved-consumer |
a consumer the parser does not model |
deep-unresolved-file-dependency |
a file sink string does not resolve, or a local-exec command names a changed path |
deep-path-unsafe |
an unsafe path, a changed link, or a case-folded collision |
deep-too-many-examples |
more than 12 selected examples |
deep-too-many-files |
a light change of more than 50 files |
deep-test-or-mock |
a test or mock file changed |
deep-files-list-mismatch |
the files list and the trees disagree |
deep-non-terraform-change |
not light, and no Terraform-read file changed |
deep-owner-raise |
host only: the owner asked for depth: deep |
deep-validate-failed |
host only: validate failed at light |
Paths of the files list, the classifier report and the affected report are relative to the head tree's root, the trees --head and --base name. A .git file or directory (a linked worktree's, a submodule's) is not part of either tree. For each reason a bad input gave, a line mtf-facts: depth: <why> goes to stderr, such as the paths a files list mismatch is about.
It exits 0 with the report, a tree over a read limit and the timeout included (deep, deep-unwalked), or 2 on a missing tree, a files list that does not read or has a line count over 10,000,000, or an unsafe --examples entry, with nothing on stdout.
mtf-facts behaviour lists every changed tree entry and decomposes the change into triggers: one per AST-level change, each with one category and the verification levels it needs. It never runs Terraform.
mtf-facts behaviour <dir> (--base <dir> | --diff <file>) [--schema <file>] --json [--timeout <duration>]The change is read as lint reads it, and the report is the same for a base tree and for every diff form and context size. It is one JSON document in the shape of behaviour.schema.json:
{"schema_version": 1, "inert": false,
"entries": [{"path": "main.tf", "change_kind": "modified", "inert": false, "trigger_ids": ["T-001", "T-002"]}],
"triggers": [{"id": "T-001", "file": "main.tf", "line": 10, "category": "wrapper-change", "change_kind": "modified",
"address": "aws_rds_cluster.this", "attribute": "vpc_security_group_ids",
"base_expr": "compact(concat(aws_security_group.this[*].id, var.vpc_security_group_ids))",
"head_expr": "concat(aws_security_group.this[*].id, var.vpc_security_group_ids)",
"referenced_inputs": [{"name": "vpc_security_group_ids", "type": "list(string)"}],
"contextual_refs": ["aws_security_group.this"], "contextual_ref_details": [...],
"pure": true, "levels": ["L1"], "reason": null}, ...],
"capped": 0}- A documentation or metadata path (as for
affected) is inert when the read-site scan of both trees is complete and no read site reaches it. A.tffile is inert when both revisions of its directory decompose and no trigger lands in it: comments, layout, a block moved unchanged between files of one directory, or a variable or outputdescription; not when a read site can read the file itself. A mode-only change, a lock file, a script and every other path are triggers. - Each module directory is compared block by block: resources, data sources, module calls, variables, outputs, each local, providers, the
terraformblock'srequired_versionand each required provider,movedblocks, and any other block by its source. A directory with a.tffile that does not parse, is not a regular file, or is an override file gets oneunclassified-changetrigger per changed.tffile. - Categories and levels:
wrapper-change(one revision wraps the other in function calls, or calls another function on the same arguments),default-change,count-for_each-conditional-change,type-validation-changeandlocals-outputs-changeselectL1;literal-argument-change,resource-added-or-removed,lifecycle-change,dynamic-block-change,unclassified-changeandnon-hcl-dependencyselectL4;provider-version-floor-changeselectscompat; acountorfor_eachchange of a resource, data source or module selectsL1andL2(terraform testwith mock providers), and a dynamic block whosefor_eachalone changed (same labels, iterator and content) selectsL1on itsfor_eachand is markedfor_each_only; a change inside nested dynamic blocks that lies in one inner dynamic block alone, with the blocks around it the same but for that one, is placed at the innermost such block, itsattributethe path of group keys such asdynamic.origin.dynamic.custom_origin_config, withL4and neverfor_each_only, since an innerfor_eachreads the iterator around it;moved-or-renamed-addressand provisioner or connection changes select none and say why. - A changed resource argument whose expression is not a literal on both sides, or that is removed, also yields an
unknown-propagation-candidatetrigger (L1,L3) when the provider schema given with--schema(aterraform providers schema -jsonfile) marks it Optional and Computed, or when there is no schema, the schema lacks the provider, the type or the attribute. A data source argument is not planned atL3: a changed data source argument whose expression is not a literal on both sides yields anunknown-propagation-candidatetrigger withL1only, since an unknown argument defers the read to apply whatever the schema says. A deferral bydepends_onor by a pending change of a dependency is not modelled, so no claim that a read is not deferred can verify. - An argument added to, or removed from, a resource or data source block that is present at both revisions reads as
nullat the revision without it, when the schema given with--schema, at each revision's provider source, marks the name as an attribute that is not write-only, reached through nested attribute types only. One schema stands for both revisions, so there is nonullwhen the provider'srequired_providersentry (its source or version constraint) or its local name differs between the revisions in that directory. A revision that writes the name in block syntax, as a nested block or adynamicblock of that name, is not absent and gets nonull. That side's expression is thennulland the trigger carriesbase_absentorhead_absent; the flags change no category and no level. There is nonullwith no schema, a disagreement, a nested block on the path, or a name that is a meta-argument or a block Terraform reads (count,for_each,depends_on,provider,lifecycle,provisioner,connection,dynamic,moved,import,removed,check); arguments of provider, module, output, locals, variable and terraform blocks never get one. - A variable the change adds, whose default is a bare literal (a quoted string with no
${or%{,true,falseornull), gives the triggers that read it adefaultcase with that value, unless a case of that value is already there. pureis set when every function the expressions call is on the built-in pure-function allowlist and neither readsself. A variable'stypeexpression reads nothing: its keywords, such asstring, are no references.contextual_ref_detailsgives each other reference as written, its kind, its type from the schema and whether its block hascountorfor_each.- A trigger that selects
L1lists its requiredcases. The inputs are its variables, its resource, data,count.indexandeach.keyreferences, and for a default or type change the variable itself. Each input has states: a list or set is typical, empty, an empty element, a null element, an unknown element, wholly unknown and null; a map the same with keyk; a string, number or bool typical, zero, unknown and null; every string, number and bool literal of either expression (at most 8 of a kind) is one more state of each input of its type, and one more element of each collection of it; a variable whose default changed is also omitted (unset, so each revision's default applies); a splat reference has one unknown instance, none, a null attribute or one known, and no null attribute when it reads theidof a managed resource (<type>.<name>[*].id), which may be unknown but is never null; any other resource or data reference is unknown or known;count.indexis 0 or 1 andeach.keyaor empty. A list or set input, and a splat reference, also take order states: two distinct elements and a duplicate. The cases are the first that fits 32: every combination of all states; every combination of the other states plus one case per order state with the rest typical; for acountorfor_eachtrigger that selectsL2, or a dynamic block, no cases and a reason; with no order states, every combination of the states; one typical case and one case per other state of each input, order states included; past 32, or with alocal,module,selforeach.valuereference, the trigger has no cases and says why. Each input of a case sets exactly one ofvalue,unknown,nullandomitted, withunknown_elementsfor the unknown elements of a value. - Triggers of one file and line whose base and head expressions are the same, and neither null, are one change seen by several categories, and are merged into one trigger, verified once: the first keeps its
categoryand its place,also_categorieslists the others' categories,levelsis the union inL1,L2,L3,L4,compatorder, andcasesthe union by case id;addressandattributecome from whichever has them. A merge is not made when either trigger selects no level, when both name an address or an attribute and they differ, when the union of the cases would be over 32, when one case id has other inputs in each, or when one trigger could not get the required cases it needs while the other has some. The ids are numbered after the merge, and each entry'strigger_idslists the merged ids. - At most 40 triggers are listed, ordered by file, line and category;
cappedcounts the rest.
It exits 0 with the report, or 2 on bad input, a limit or the timeout, with nothing on stdout.
mtf-facts claims-eval validates the claims a reviewing model wrote, and computes the state of every trigger and every claim from a ladder record.
mtf-facts claims-eval --behaviour <file> --ladder <file> [--claims <file>] [--head-sha <sha>] [--base-sha <sha>] --json- The claims file has the shape of claims.schema.json: each claim names a closed predicate (
expression-output-unknown,expression-output-length-known,expression-output-equals-for-inputs,plan-attribute-unknown,plan-attribute-unchanged-vs-prior,plan-action,input-filtered) and a trigger by id inparams.node, never HCL text. A claim that does not validate is listed indiscarded_claimswith the reason. - The ladder record has the shape of ladder-record.schema.json; unknown fields are refused, and a record with runs must name its
head_shaandbase_sha. A record with runs needs--head-shaand--base-sha, and a record of other commits is refused.cappedechoes the behaviour report's. input-filterednames a case input (a variable or a contextual reference) and a marker (empty-stringornull); with"filtered": falseit states that the input's marker elements reach the output instead of being removed.expression-output-unknown,expression-output-length-knownandinput-filteredtake an optionalparams.cases_where: a non-empty object of the trigger's case inputs to states (wholly_unknown,unknown_element,empty,knownfor any other known non-null value,singleandmultiplefor a known collection of one or more elements, whichknownalso matches,null,omitted); the claim is judged only on the required cases whose inputs are all in those states, and a scope that matches no case leaves it unverifiable.- A claim reads the runs of its trigger at its levels (
L1for expression predicates andinput-filtered,L3orL4for plan predicates) that ran with a valid control. It is verified when some observation is about it and all agree, refuted when one contradicts it, and unverifiable otherwise; an unknown, sensitive or redacted value leaves a value predicate unverifiable. - A trigger is refuted when one of its claims is. A trigger that selects
L1orL3is decided by the tool's own sweep, never by the claims alone. Every selected level needs base and head observations from runs with a valid control and a fixture hash (and none with an invalid control): atL1andL2of every required case the trigger lists (the same cases at both), each observed with exactly the case'sinputs, other cases ignored; atL3the same case ids at both revisions; at most 32 cases at each level. A case whose plan fails in the subject (error) has no output; a failure at one side only is a difference, and a sweep whose cases all fail at both, or whose first case does, is unverifiable. Each case is compared attribute by attribute:known,length,value(the value, or the known parts of a partly unknown one),sensitive,redacted,unknown_inputs,action,unchanged_vs_prior, and when neither side is knownwholly_unknownandunknown_mask. A side not known with nounknown_mask, a partly known side without its known parts, or a case known at both revisions whose value is sensitive or redacted leaves the trigger unverifiable. With no difference the trigger is verified, claims or not. Otherwise it is verified only when verified claims state both sides of every difference exactly, each on that case's observation at its own revision: an unknown side byexpression-output-unknown(L1) orplan-attribute-unknown(L3), a known side byexpression-output-equals-for-inputs(L1) orplan-attribute-unchanged-vs-prior(L3), a length byexpression-output-length-knownorexpression-output-equals-for-inputs, a value byexpression-output-equals-for-inputsat both revisions, an action byplan-actionat both. AtL1awholly_unknownorunknown_maskdifference from wholly unknown at the base to partly known at the head (a definite length, not failed, sensitive or redacted) is explained by a scopedexpression-output-unknownat the base and a scopedexpression-output-length-knownwhoselengthequals the observed one at the head; the reverse, a known length becoming unknown, is never explained; any otherwholly_unknownorunknown_maskdifference, and everysensitive,redacted,unknown_inputsanderrordifference, is never explained. AtL1avalueorlengthdifference where one revision drops an input's marker elements that the other keeps is also explained by a verifiedinput-filteredclaim at each side (filtered at one,"filtered": falseat the other, same input and marker) when the keeping side's value with those markers removed is the other side's value and it holds no more marker elements than the claimed inputs do. AnL2difference (anL2observation is the instance count forcount, or an object of instance key to planned input forfor_each) is listed and never explained. AnL2run, whose mock providers make computed values known, never evidencesexpression-output-unknown,expression-output-length-knownorplan-attribute-unknownand explains no unknown-ness difference. A trigger whose expressions compare an input with an input-free function call, such asvar.cidr == lower("EVIL"), has no cases. Each difference is listed in the trigger'schanges, withexplained, the claims that explained it inexplained_by, and the case inputs. - A trigger that selects levels of more than one family, the sweep (
L1,L2,L3),L4andcompat, which only a merged trigger does, needs each family to decide it, and is then covered; one family never stands in for another. - A trigger the sweep reads that is not inert or refuted carries
complete,outstandingand, whenoutstandingis not empty,evidence_digest.completeis true when every level it selects ran with a valid control, every required case was observed at both revisions, every observation could be compared, and anL4orcompatlevel it also selects decided it.outstandinglists the differences no verified claim explains as<trigger id>|<level>|<case id>|<attribute>, sorted,[]when none; a complete trigger with an empty set is verified. Past 256 unexplained differencescompleteis false,outstandingandevidence_digestare left out, and the reason ismore than 256 unexplained differences: no finding can list them.evidence_digestis the sha256, in hex, of canonical JSON (keys sorted, no whitespace, numbers as written) ofhead_sha,base_sha, the outstanding ids, the base and head observations of every case with an outstanding difference, and the sorted fixture hashes of the trigger's valid runs at those levels: the same ids over other observations give another digest. A finding that explains the differences carries the ids inexplainsand the digest inevidence_digest, so a host can tell it judged this evidence. - A trigger that selects
L4is covered when no claim of it is unverifiable and its address and changed attribute (adynamicblock by the blocks it generates; never alifecycleargument) are in a planned example: in the example's own directory, or under a module call the record'smodule_callsties to the trigger's directory. An inert entry's triggers are inert; anything else is unverifiable. Refuted outranks covered.
The output has the shape of claims-eval.schema.json. It exits 0 with the report, or 2 on bad input.
mtf-facts bundle turns a lint --json report into the facts bundle a model reviewer reads, and the injected rule text for the rules the report left not_evaluated.
mtf-facts bundle --report lint.json --skills path/to/terraform-module-reviewer \
--out bundle.txt --injection injection.md \
--head-sha <sha> --merge-base-sha <sha> --binary-sha256 <sha256> \
[--schema-handoff <file>] [--schema-files <file>]... [--profile-dir <dir>]--skills is the reviewer skill directory at its pinned commit. From it, the bundle reads the rule registry, references/rule-facts.md, and the rule text under references/engine/. --out and --injection must be different files. --head-sha and --merge-base-sha take 40 or 64 lowercase hex characters, and --binary-sha256 the pinned SHA-256 of the binary that wrote the report. Give --schema-handoff, --schema-files and --profile-dir exactly as the lint run had them.
These envelope fields of the report must equal their pins: the schema version, both commits, the binary SHA-256, the SHA-256 of the registry, the profile SHA-256 and the schema inputs. The report must also come from a run with --base, name only tool and candidate rules of the registry, give every not_evaluated entry a reason code, give each candidate a well-formed id that no other candidate has, and carry schema coverage only when it read a schema handoff. Any mismatch exits 2.
It prints the mode to stdout and exits 0:
judgment: it wrote--outand--injection. Both files must not exist yet; when the second write fails, the first file is removed.full: the tool findings and candidates alone are over the 48 KiB cap, or the injected rule text is over its 24 KiB budget. It writes no file and gives the reason on stderr.
--out is not plain JSON. It is a framed region:
FACTS-BUNDLE-BYTES <n>
FACTS-BUNDLE BEGIN
<json-string>
FACTS-BUNDLE END
<json-string> is one line holding the bundle as a JSON string with every byte outside printable ASCII escaped. The bundle inside that string has the shape of facts-bundle.schema.json, and <n> is the byte length of the string line. The 48 KiB cap counts that line. Over the cap, sections are shed lowest priority first (file_index, schema_coverage, interface_diff, area_rules), never tool_findings or candidates.
The bundle carries schema_version: 1, module_areas (the report's scope.module_areas) and in_repo_downgrade together with area_rules, as one ownership unit: they are shed together, last, as area-rules. A report without scope.module_areas, from an older release, or with a module or downgrade area longer than 1024 bytes sheds the unit from the start. A bundle without schema_version is a legacy bundle.
mtf-facts merge reads the bundle file mtf-facts bundle writes. It checks a model reviewer's reply to that bundle and merges the two into the final findings. The reply is untrusted: it is read with exact keys, no duplicate key, no trailing data and a 1 MiB cap.
mtf-facts merge check bundle.txt reply.json
mtf-facts merge final --registry rule-facts.md [--report lint.json] [--templates templates.json] \
bundle.txt reply.json [repair.json]
mtf-facts merge counts bundle.txt reply.json [repair.json]The reply is one JSON object with the arrays candidate_decisions and findings and, optionally, a counts object, in the shape of model-output.schema.json. counts holds the three counts a reviewer's summary line carries: instruction-like-text-from-others, conversation-claims-unread and review.quoted-claim-unverifiable. When present, it has exactly those three keys, each a natural number written as decimal digits, 0 included and at most 9007199254740991; anything else makes the reply invalid.
merge check prints {"result":"valid"} and exits 0, or {"result":"repair","reasons":[...]} and exits 1 when the reply needs the repair turn. Its reasons are fixed text and quote nothing from the reply.
merge final prints the merged findings, in the shape of merged-findings.schema.json, and exits 0. It merges the first reply when that one is valid, and otherwise the repair reply, which must then be given: an empty file when the repair turn returned nothing. A repair reply that is still invalid keeps only its valid entries; a candidate it leaves undecided becomes review.check-not-run. --report takes a lint --json report, whose index derives the subject a lead finding dedups by. --templates takes the per-rule host text for confirmed entries the model gave no text for. A tool finding keeps the predates_change its bundle entry carries, which the host copies from the confirmed candidate; an llm or host finding never has it. A lead finding of compat.expression-behaviour-changed or a claim.* rule may carry explains, up to 256 difference ids as claims-eval lists them in outstanding, and evidence_digest; on any other rule, or malformed, they make the finding invalid. Lead findings that dedup into one and carry the same digest explain the union of their ids.
A lead finding on a rule that is not an llm rule is kept only where the bundle's area_rules lists that rule not_evaluated for the finding's area, or under . for a cross-area rule. In a bundle with schema_version 1 the area is the one facts.AreaOf gives with the bundle's module_areas, after the two-segment examples/<n> and modules/<m> directory of a path rule, which is its own area when the bundle lists it. A legacy bundle, without schema_version, maps paths as before module areas and has no downgrade. A schema_version other than 1, or one without all of area_rules, module_areas and in_repo_downgrade, or with an area that is not a clean relative path, exits 2.
A model severity off its rule's list is clamped to the nearest permitted one, as before, with one exception: exactly MEDIUM on compat.variable-removed, compat.variable-now-required, compat.type-narrowed, compat.default-changed or compat.generated-policy-narrowed, in an area of the bundle's in_repo_downgrade, is kept. merge final writes the number of clamped severities to stderr when there are any.
merge counts takes the replies as merge final does, picks the same reply, and prints its counts as {"counts":{...}}, or {"counts":null} when that reply has none, and exits 0. A reply without counts has none: no count is ever read as 0 for its absence. A repair reply that is still invalid keeps its counts when they are well formed. merge counts does not load the registry, the report or the templates, so it can succeed where merge final fails: a host reads the counts only after merge final succeeds.
mtf-facts profile-check --profile-dir <dir> [--gates] [--downgrade-ids]First reads the directory with the walker profile-snapshot uses and exits 2 on anything it refuses (a link, a special file, a bad name, another device or a bound), so a profile cannot pass the check and then fail the snapshot. Hosts run it on the snapshot directory. It then reads the profile as lint --profile-dir does and checks it, options included. It prints one JSON object: valid, problems, and for a valid profile options and relaxing. --gates adds gates, the gate table, and gated_rules, the rules this profile gates; --downgrade-ids adds downgrade_ids. Exit 0 for a valid profile, 1 for an invalid one, 2 when the directory is missing, a link or cannot be read.
mtf-facts profile-snapshot --profile-dir <dir>Copies the profile directory into a new 0700 temporary directory and prints {"dir", "tree_sha256", "profile_sha256"} for the host. Each directory is opened relative to its parent and each file with O_NOFOLLOW|O_NONBLOCK, checked with fstat as the regular file the listing saw and on the source's st_dev, and copied from that descriptor. A link, a special file, another device, a name that is not UTF-8 or holds a control character or DEL, more than 256 entries, more than 8 MiB or a path more than 8 levels deep exits 2, and so does a copy that does not read back the same. The host reads options and gates only from the copy and removes it when done. profile_sha256 is null when the copy is not a valid profile.
mtf-facts ledger lists the provider fields of each resource and data source type the module in the given directory declares, nested fields included, from one or more terraform providers schema -json files:
mtf-facts ledger path/to/module --schema-files schema.json [--schema-files more.json]... \
[--check coverage-ledger.json] [--registry rule-facts.md]It reads only that directory, not modules/* or examples/*. At least one --schema-files is required. The run exits 2, with no inventory, when:
- a provider the module uses is in no schema file;
- a type is not in its provider's schema;
- the directory has no
.tffile or no resource or data block; - a block is an ephemeral resource;
- a file is one the parser cannot model.
The JSON output has an envelope, the types and their fields, and problems. The envelope holds the tool, registry_sha256 (null without --registry), and the hash of each schema file and of the ledger, each named by its base name.
--check reads the author's coverage ledger, in the shape of ledger.schema.json, and lists each problem it finds:
- a leaf field with no disposition or more than one;
- a field not in the schema;
- a type the module does not declare, or one listed twice;
- an input or output the module lacks;
- an input on a computed-only field;
- a disposition other than
intentional_omissionornot_applicableon a block or nested attribute that has fields.
Without --check, problems is null.
| Code | Meaning |
|---|---|
| 0 | The inventory, and with --check a complete ledger. |
| 1 | With --check: at least one problem, listed in problems. |
| 2 | An error: bad input or a limit. |
mtf-facts schema-diff compares two terraform providers schema -json files, the current and the target provider version, for the resource and data source types a module tree uses:
mtf-facts schema-diff --from current.json --to target.json --types-from path/to/moduleAll three flags are required. It writes the report as JSON, in the shape of schema-diff.schema.json, and exits 0, with changes or none. Each schema file is recorded by base name and SHA-256 only. A provider the tree uses that either file lacks, bad input or a limit exits 2.
mtf-facts fixture-check checks the scratch tree a verification ladder level will run against a default-deny allowlist, before any Terraform command runs on it, and hashes the tree:
mtf-facts fixture-check scratch/head --json
mtf-facts fixture-check scratch/l2 --level L2 --pins pins.json --schema schema.json --json
mtf-facts fixture-check scratch/l3 --level L3 --resource aws_rds_cluster --pins pins.json --jsonThe output, in the shape of fixture-check.schema.json, is {"schema_version":1,"ok":...,"tree_sha256":"...","violations":[{"file","line","rule","detail"}]}. line is 0 for a whole file and file is . for the whole tree. detail names the refused function, block type or argument when that name is a plain identifier; it is left out otherwise, so no text from the tree is echoed. At most 200 violations are listed, then one limit.violations entry.
tree_sha256 is the SHA-256 of one name NUL type NUL content-sha256 LF line per directory entry, sorted by name. The type is file, file+x (any exec bit set), dir, symlink or special, so a umask does not change the hash. It identifies a tree that passes; on a violation it covers what was read.
--expect takes the manifest the trusted template wrote, in the shape of fixture-manifest.schema.json:
{"schema_version": 1, "files": {
"main.tf": {"blocks": [{"type": "resource", "labels": ["terraform_data", "ref_0"]}]},
"head.tf": {"blocks": [{"type": "output", "labels": ["head"]}],
"single_expressions": [{"block": "output", "labels": ["head"], "attribute": "value"}]}}}With it, each file's top-level blocks must be exactly the listed set, by type and labels in any order, and no other .tf file may exist (manifest.block-set). The manifest binds .tf files only; at L2 the test file is checked by its grammar and never by the manifest. A file has at most one single expression, so each splice (base and head) goes in its own file. Its attribute must exist, its block must be the last block of the file, the attribute the last argument or block of its block, and no comment may start at or after the attribute (manifest.expression). A spliced expression that closes its block to add another, ends its attribute to add an argument, or opens a comment to swallow template text fails one of the two.
--pins takes {"providers":[{"source","version"}]}, optionally with a sha256 per entry: the provider versions the host installed, each version exact and each source once.
The grammar, by level (--level, default L1):
- Files: only names matching
^[A-Za-z0-9][A-Za-z0-9_.-]*\.tf$directly in the directory, nooverride.tfor*_override.tf. Any subdirectory (.terraformincluded), symbolic or hard link, special, hidden,.tf.json,.tfvarsor lock file is a violation. At most 64 entries, 256 KiB a file and 1 MiB in all. - Top-level blocks:
terraform,variable,locals,output,providerandresource.data,module,moved,import,removed,check,ephemeral,actionand every other block are refused. terraform: onlyrequired_providers, each entry{ source, version }as literal strings, each key once, equal to a pin, with the local name equal to the last part of the source and a provider of the level.backend,cloud,provider_metaandrequired_versionare refused.variable:type,default,description,sensitive,nullableandvalidation.output:value,description,sensitive,depends_onandprecondition.resource:terraform_dataat every level. L3 adds exactly one resource of the--resourcetype, which must be a curated type:aws_rds_clusteroraws_db_instance. Adding a curated type takes an audit of every argument in the pinned provider schema, nested ones included, for any that reads a local file or other local source; the audit is recorded in the type's test. A type with such an argument is not curated. No resource argument, nested ones included, may be namedfilename,source,file,path,content_base64or end in_file,_filenameor_path.provisionerandconnectionblocks are refused, also asdynamiclabels;lifecycletakescreate_before_destroy,prevent_destroy,ignore_changes,replace_triggered_by,preconditionandpostcondition. Theproviderargument names a provider of the level.provider:terraformwith no argument, at every level. At L3,aws, which must be set:regionas a literal matching^[a-z]{2}(-[a-z]+)+-[0-9]$,access_keyandsecret_keyas nonempty literal strings,skip_credentials_validation,skip_requesting_account_id,skip_metadata_api_checkandskip_region_validationas literaltrue, and anendpointsblock that has the key of the service under test (rdsfor both curated types) and whose every entry ishttp://127.0.0.1:1orhttp://[::1]:1. Every other argument or block,aliasincluded, is refused.- Expressions: every function call is on a pinned pure-function list (no
file*,templatefile,templatestring,abspath,pathexpand,dirname,basename,timestamp,plantimestamp,uuid,uuidv5,bcrypt,rsadecrypt,yamldecode,ephemeralasnullorprovider::function). Every reference starts withvar,local,each,count,self,terraform_dataor the type under test, and aforexpression, templatefordirective ordynamiciterator may not bind one of those names orpath,terraform,data,module,ephemeral,providerorresource. A heredoc may only be a plain string.
L2 runs module logic under terraform test with every provider mocked. Its .tf files follow the grammar above, with these changes:
- Files: exactly one
*.tftest.hclfile besides the.tffiles (test.file-count)..tftest.json,*.tfmock.hcland atests/directory are refused like any other file or directory. - No
providerblock, in a.tffile or the test file: every provider is amock_provider. A mock still starts the pinned provider plugin: terraform calls its GetProviderSchema, GetResourceIdentitySchemas and ValidateResourceConfig on the tree's configuration, never Configure or plan. So the runner must start L2 under the same provider sandbox profile as L3, and the file-argument denylist stays. Resources and data sources of any type are allowed. The provider of each one (itsproviderargument, or the prefix of its type) must be declared inrequired_providersas a pin and have amock_provider(test.provider-not-mocked). So must every declared provider. The built-in provider needs no mock, and itsterraform_remote_statedata source is refused. dataandmovedblocks are allowed.movedtakesfromandto, each a static resource address, never a module or data address. References may also start withdataor any resource type name (^[a-z][a-z0-9]*_[a-z0-9_]+$).
The test file holds only these blocks:
mock_provider "<name>", with no argument (nosourceof mock files, noalias), holdingmock_resourceandmock_datablocks with onlydefaults, andoverride_resourceandoverride_datablocks.override_resourceandoverride_datawithtarget, a static address of a resource or data source in the tree, andvalues.- Each mock and override block,
mock_providerincluded, may also setoverride_during = plan, so a plan-only run sees the mock values.apply, the default, is refused. The result then carries"override_during_plan": true. - Every data source a run reads must have fixed values. A mock generates a random value, on each run, for every computed attribute no mock value covers. So a
mock_datablock of its type in its provider's mock, or anoverride_datatargeting the whole block (no instance key) at the top level or in the mock, must give a non-null literal to every computed attribute of the type (test.data-not-mocked). That includes optional and computed ones, nested attributes, and every nested block that holds a computed attribute (with at least one element for a list, set or map). An override inside onerundoes not count. - When the test file sets
override_during = plan, every resource of a mocked provider needs the same cover, from amock_resourceor a whole-blockoverride_resource(test.resource-not-mocked). Without it, computed resource values stay unknown at plan and need nothing. - Which attributes are computed comes from
--schema: theterraform providers schema -jsonfile the acquisition phase wrote for the pinned providers. A type it lacks is not covered. Without--schema, a tree with a data source of a mocked provider, or a mocked resource underoverride_during = plan, fails withtest.schema-required.--schemais refused at L1 and L3. variables.run, which must setcommand = plan(test.command-not-plan, also when the command is left out, since the default is apply), withassert(condition,error_message),variables,override_resource,override_data,plan_optionswith onlymode = normal(test.plan-options-denied), andexpect_failuresnamingvar,output, data or resource addresses.module,override_module,providers,state_keyand every other argument or block are refused.
Mock values are not provider behaviour, and a mock can make computed attributes known at plan. So an L2 run never evidences expression-output-unknown, expression-output-length-known or plan-attribute-unknown, with or without override_during_plan. It evidences module logic only: keys, counts, addresses and known values.
Mock defaults, override values and variables must be literals: strings without interpolation, numbers, bools, null, and lists and objects of them with bare or quoted keys (test.value-not-literal). Asserts use the pure functions and references from var, local, output, run, data or a resource type. A test file provider, test or other block is refused.
The check reads syntax only and evaluates nothing. It does not bound what a passing tree can make Terraform allocate, such as a large count or range: the host runs plan under its own memory and time limits.
| Code | Meaning |
|---|---|
| 0 | The tree passes. |
| 1 | At least one violation. |
| 2 | An error: bad flags, a bad pins file or a directory that cannot be read. |
mtf-facts rules
mtf-facts version
mtf-facts completion bashrules lists the implemented rules, one per line with its tier (tool or candidate), sorted by id. version prints the binary version, as does mtf-facts --version. completion prints a shell completion script for bash, zsh, fish or powershell to stdout.
All 42 tool and candidate rules of the reviewer's rule registry are implemented. CI checks that the mtf-facts rules listing equals the registry's tool and candidate rules, tier for tier, against a registry pinned by commit and SHA-256.
Areas are the root module, each modules/<name> and each examples/<name>, parsed one directory at a time. A change under .github/ or to a root profile file such as .editorconfig counts as the root. Candidate subjects are scoped to their area, <area>:<address>, such as .:var.name.
A tree outside that layout, such as a monorepo with infra/modules/network called from infra/live/prod, gets one more kind of area, module: each other directory that directly holds a .tf or .tf.json file, at head or at base. A file belongs to the nearest such directory above it, whatever its extension, so a change to infra/modules/network/main.tf puts that module in scope, not the root. Directories with a tests, wrappers or examples segment, a dot segment or a top-level modules/ are not discovered, and their files stay the root's as before, so a module under such a directory is read as part of the root. A module area takes only the compat.* and coverage.* rules; the convention, profile and layout rules (structure.*, profile.*, scope.*, examples.*) are not_applicable there, scope.tooling included, so a scanner config file added under a module is not judged. A local source that resolves to a module area counts as a call in callers and scope.areas_calling. A directory that held configuration at base and is a link at head stays an area and is not_evaluated for every rule that applies; a link that never held configuration is not an area, and links are never followed. A tree with more than 512 discovered areas is a limit: exit 2, area_limit_exceeded, no report. Reports of a tree with no discovered area are unchanged.
With a base tree, the root is in scope only when the change touches it. A cross-area rule has one entry under . for its pass, whether or not the root is in scope, and one more under each other area that holds one of its findings or candidates: the deepest area directory that holds the item's file. The . entry is not_evaluated when the pass is, and otherwise has the status of the items in the root area, clean when there are none. A structure.version-floor-understated candidate points at the caller's own floor, or at the call when the caller declares none, so it is filed under the caller's area; it is in the change when the caller's floor, the callee's floor or the call is a changed line. A cross-area pass reads every area of the module, in scope or not, and keeps only the findings and candidates of the areas in the change. When any area it reads cannot be evaluated, untouched areas included, that entry is not_evaluated with one reason code (parse_error, unsupported_file, symlink or limit_exceeded) and no findings or candidates.
Files Terraform ignores (a leading ., a trailing ~, #...#) are skipped. A directory gets not_evaluated for its configuration rules, never clean, when it has a parse error, a .tf.json file, an override file, a .tf file that is a link or cannot be read, or a file over the token, nesting or unary operator limits. Path rules still run: a link where a path rule looks for a file counts as present and is never followed. An area directory that is a link or cannot be read gets not_evaluated for every rule.
facts.AreaOf(file, module_areas) is the one ownership function, shared by the engine and the merge: the top-level modules/<m> or examples/<n>; a module area whose own path the file is; the root for a file under a tests, wrappers, examples or dot segment; the nearest module area above the file; the root. It refuses a path that is empty, absolute, holds a backslash or a drive letter, is not clean or has a .. segment.
module_callers has one entry per head area of kind root, submodule or module (never example) with a file the change adds, changes or removes, sorted by module: {module, complete, callers: [{area, file, line, file_changed}]}, callers sorted by area, file and line. A module the change removes gets no entry. Callers come from every head .tf file, those under tests/, wrappers/ and examples/ included; .git and .terraform are never read. A literal source starting ./ or ../ is joined to the calling file's directory and must name exactly a discovered area path: ., a submodule or a module. Registry addresses, git::, hg::, s3::, gcs::, http(s)://, github.com/ and bitbucket.org/ sources are ignored. file_changed is true when the change adds the caller's file, changes its bytes or, with --diff, lists it. More than 4096 caller records exit 2.
complete is one value for the whole tree, repeated on every entry. It is false when a local source escapes the tree or names no area (no ancestor is tried), a source is empty, not a literal, absolute, a drive letter or of no known form, a .tf or .tf.json file cannot be parsed or read, any entry is a link, special or unreadable, the tree holds a file that may call a module but that the engine does not parse (a .tftest.hcl or .tftest.json test file, a terragrunt.hcl or any other .hcl file except .terraform.lock.hcl, or an OpenTofu .tofu or .tofu.json file), or a change path is not a clean relative path. A changed or unchanged such file anywhere in the head tree is enough, so a Terragrunt caller beside the .tf ones blocks the downgrade.
A profile may hold a profile-options constants block: format: 1, consumers (public or in-repo), partitions (portable or fixed), examples (required or none) and release (registry or none), every key required. Without it the options are the defaults public, portable, required, registry; any other value relaxes the review. An unknown format, key or value, or another unknown block, makes the profile invalid (profile_unavailable). A key written as <key>: with nothing after the colon has the empty value: an empty list, or a single value that is not set; <key>: with a trailing space stays invalid. Only these keys may be empty: required-files example and example-dirs (and only with examples: none, which needs both empty), profile-owned-files verbatim and workflow-dir, tooling files and dirs, and banner line. template-destinations may hold no key at all. An empty value judges nothing: no verbatim file and no workflow directory for scope.profile-file, no banner for scope.readme-banner, no scanner file or directory for scope.tooling, and no template for profile.template-drift.
The gate table is held in the binary; profile-check --gates prints it. A listed value makes only the listed rules not_applicable with reason code profile_option:
| Option | Rules |
|---|---|
partitions: fixed |
structure.arn-partition-hardcoded |
examples: none |
examples.argument-undemonstrated, examples.disabled-call-missing, examples.feature-undemonstrated, examples.submodule-missing, examples.versions-missing |
release: none |
profile.changelog-handwritten, profile.commit-type-mismatch, profile.migration-notes-misplaced, profile.owner-identifier, profile.provider-meta-missing, profile.release-config-changed, profile.template-drift, profile.title-misspelling, profile.title-uninformative, profile.unrelated-change-bundled |
profile.required-file-missing is never gated; a profile with examples: none has no example entries for it to require. consumers: in-repo gates nothing: it only feeds the downgrade below.
in_repo_downgrade lists the modules of module_callers that have at least one caller and every caller's file_changed set, when the profile sets consumers: in-repo, the change is base or diff, the run is not --full-tree, and complete is true. The candidates of compat.variable-removed, compat.variable-now-required, compat.type-narrowed, compat.default-changed and compat.generated-policy-narrowed in those areas carry downgrade: "in-repo" and severities: ["MEDIUM"].
Depth limitation: depth does not read module areas. Examples nested under a module area, such as infra/modules/network/examples/basic, are out of its scope.
No area reads tests/ or a directory more than one level under modules/<name>/, though both hold boundary directories for the scope rules. So these rules are not_evaluated(unsupported_file) in these cases:
scope.module-callandscope.providerat the root, when the change adds or edits a.tfor.tf.jsonfile, or any link, special or unreadable entry, undertests/, or whentestsitself is such an entry.scope.module-callandscope.provideratmodules/<name>, when the change adds or edits a.tfor.tf.jsonfile, or any link, special or unreadable entry, in a nested directory there, or adds a link, special or unreadable entry directly inmodules/<name>/that is not a.tfor.tf.jsonfile (one that is keeps its own reason code).scope.new-submoduleatmodules/<name>, on the same entries when they lie in a nested directory the base lacks, or are a link, special or unreadable entry.
The modules/<name> cases exempt a regular .tf file in a nested directory that parses with no issue and holds only variable, output, locals, moved and removed blocks. For a new modules/<name>, a candidate from its own top level comes first, and the nested entries give not_evaluated only when that top level gives none.
The build supports Unix only: the tree reader opens files with O_NOFOLLOW.
Each release publishes one binary per platform, mtf-facts_<tag>_darwin_amd64, mtf-facts_<tag>_darwin_arm64, mtf-facts_<tag>_linux_amd64 and mtf-facts_<tag>_linux_arm64, a SHA256SUMS file, and a build provenance attestation for each binary. The build is reproducible: the same tag and the Go version in .go-version give the same bytes on any host. scripts/build-release.sh fixes the build settings known to change the output: -trimpath, an empty build ID, the toolchain, CGO_ENABLED=0, GOFLAGS=-mod=readonly, GOWORK=off, an empty GOEXPERIMENT, GOFIPS140=off and the architecture levels. Go embeds no build time; the module version comes from the tag.
Releases follow the Conventional Commits in the titles of merged pull requests, which become the commit subjects on master. pr-title-lint checks each title.
- A merge to
masterthat holds afeat,fix,perforrevertcommit or a breaking change startsautomated-release. It bumpsversion.json, adds the new section to CHANGELOG.md and opens and mergeschore(release): vX.Y.Z. Other commit types wait for the next releasable one. tag-releasetags that merged commit with the version.- The tag starts the
releaseworkflow. It builds every binary, attests them and creates the GitHub release with the changelog section as its notes. The build job holds no token that can sign or write. The attestation runs in thereleaseenvironment, so it waits for approval once that environment has reviewers. Only the job that creates the release can write to the repository. A last job asks modulestf/homebrew-tap to update its formula.
The first two steps need the secret RELEASE_TOKEN in the release-automation environment, which only master may use: a fine-grained token of an administrator with contents and pull requests write. A tag or pull request made with the default GITHUB_TOKEN starts no workflow, and only administrators may create v* tags. The last job needs HOMEBREW_TAP_TOKEN, which may run workflows in the tap; without it the tap picks up the release on its daily schedule.
To cut a release by hand, as before, tag a commit on master whose ci run passed and push the tag. The workflow refuses a tag whose commit is not on master. A pre-release tag such as v0.5.0-rc.1 is published as a pre-release.
git tag -a v0.5.0 -m v0.5.0 <commit>
git push origin v0.5.0A pushed tag publishes a public release. Do not reuse or move a tag: delete a bad release and cut the next version.
Check the attestation of a downloaded binary. -R alone accepts an attestation from any workflow on any ref of the repository, so also name the release workflow, the tag and GitHub-hosted runners:
gh attestation verify mtf-facts_v0.1.0_linux_amd64 -R modulestf/facts \
--signer-workflow modulestf/facts/.github/workflows/release.yml \
--source-ref refs/tags/v0.1.0 \
--deny-self-hosted-runnersCheck its SHA-256 against the release:
sha256sum --ignore-missing -c SHA256SUMS # Linux
shasum -a 256 --ignore-missing -c SHA256SUMS # macOSRebuild the release from source and compare every SHA-256 with the published one. This needs git, curl and Go; a different local Go downloads the release's toolchain.
make release-verify TAG=v0.1.0A consumer, such as the verify job of a hosted review workflow, pins one version and one SHA-256 per platform next to the step that downloads mtf-facts. To move the pin:
-
Verify the new release as above: the attestation and the rebuild.
-
Read the SHA-256 of each platform the consumer runs on from that release's
SHA256SUMS. -
Change the version and every SHA-256 in one commit. A version without its matching SHA-256 values fails the download check, as intended.
-
Keep the attestation check in the download step, with the same flags as above and the pinned tag, so the pin rests on both the hash and the provenance:
gh attestation verify "mtf-facts_${VERSION}_linux_amd64" -R modulestf/facts \ --signer-workflow modulestf/facts/.github/workflows/release.yml \ --source-ref "refs/tags/${VERSION}" \ --deny-self-hosted-runners
Apache-2.0. See LICENSE.