Avm.Authoring 0.1.7

Second remediation round from end-to-end testing released 0.1.6 against the
canary repo Azure/terraform-azurerm-avm-ptn-example-repo (F23–F48).

Two numbering sequences collided during review, so the F tags below are
branch-local. Where a reviewer used the same number for a different defect, the
mapping is:

Reviewer ID	Tag used here	Defect
F39	F41	Three unanchored ::error:: annotations per failure, worst-first
F40	F42	format and docs auto-fix the CI working copy and report pass

Both are fixed in this release. The F39 and F40 tags below refer to the
shell-hook guard and the empty-tier status respectively.

Breaking

• avm pr-check now fails on formatting and documentation drift. format
  and docs previously rewrote your files in the CI working copy and reported
  pass, so the fix was thrown away with the runner and the defect merged. Both
  steps now check rather than fix, and report fail with one issue per file. A
  repository with unformatted sources or a stale README.md that passed on
  0.1.6 will fail on 0.1.7. Action required: run avm pre-commit
  locally and commit the result. pre-commit still auto-fixes — only pr-check
  gates. (F42)

• The end-to-end check name has changed. Fanning e2e out across a per-example
  matrix renames the check from End-to-end tests to
  End-to-end tests (<example>), one per example. A branch protection rule that
  requires the old literal name will not error — it will simply never be
  satisfied, and pull requests will wait on a check that no longer reports.
  Action required: in Settings → Branches → your protection rule, remove the
  required check End-to-end tests and add each End-to-end tests (<example>)
  entry that now appears. Do this in the same change that picks up 0.1.7.

• avm no longer writes result objects to the success stream. Anything that
  captured a command's output as an object needs --passthru to get it back.
  (F23)

• avm test e2e --example <name> treats an unknown name, or one carrying a
  .e2eignore marker, as a hard error rather than a silent skip. This is
  deliberate — it is what stops a typo in a CI matrix from quietly passing — but
  a matrix generated by hand rather than by --list may now fail.

• The round-robin subscription fan-out reads TEST_SUBSCRIPTION_IDS. Repos
  configured with only ARM_SUBSCRIPTION_ID still work through the preserved
  fallback path, but every parallel matrix leg then deploys into the same
  subscription and may hit quota where a sequential run did not.

• avm pr-check now runs the unit test tier, and its test step is renamed
  to validate. The step called test never ran a test: it routes to
  terraform validate / bicep build, reports FilesProcessed and carries no
  run counts, so a module whose tests were broken — or entirely absent — still
  produced an all-green gauntlet in a few seconds. The step is now named for
  what it does, and a real unit test step runs after it.

  Two consequences. pr-check gets slower by the cost of your unit tier
  (~20s on a typical module). And it can newly fail: if your unit tests were
  never actually running locally, this is the run that tells you. Both are the
  point — pr-check documents itself as running every check that runs in CI,
  and that claim was not true while the unit tier was absent. Only the unit
  tier is included, because it is the one tier that is credential-free by
  design; integration and e2e need a live subscription and stay out.
  avm pre-commit is unchanged and still offline. (F38)

• A repo misconfiguration now fails the gauntlet instead of skipping it.
  avm pr-check and avm pre-commit mapped every AvmConfigurationException
  to skipped, which does not flip the overall status. That type is thrown
  both for "this verb does not apply to this ecosystem" and for real
  misconfigurations — an unresolvable tflint or mapotf config bundle, an
  unresolvable managed-files repo id, an invalid .avm context override,
  AVM_OFFLINE=1, an invalid AVM_MIRROR, an unknown or .e2eignored e2e
  example. Any of those inside a gauntlet step rendered as a benign green run.
  Ecosystem gates now throw the new AvmNotSupportedException (which derives
  from AvmConfigurationException, so nothing else changes), and only that
  type still skips. Action required: none, unless one of your repos was
  quietly misconfigured — in which case the gauntlet will now say so. The step
  fails rather than errors, so the chain still runs to the end and reports
  every problem instead of stopping at the first. (F39)

• A test tier with no test files now reports skipped, not pass.
  avm test unit, avm test integration and avm test e2e returned
  Status: pass with FilesProcessed = 0 when the module shipped no
  tests/<tier>/*.tftest.hcl (or no runnable examples/). Once F38 put the
  unit tier inside pr-check, that rendered as unit test -> pass for a module
  with no tests at all — indistinguishable from a real pass, and exactly the
  vacuous-green failure mode the F33 run counts exist to prevent, one level up.
  skipped does not flip the overall status and the CLI still exits 0, so a
  module with no tier is reported rather than broken. Action required: none,
  unless you parse Status from these verbs — an absent tier now yields
  skipped. A tier whose files exist but execute no run blocks still reports
  pass; RunsTotal = 0 is the signal there. (F40)

Upgrade notes

• tflint --init needs an authenticated token outside the reusable workflow.
  TFLint resolves its ruleset plugins through the GitHub releases API, which
  allows 60 unauthenticated requests per hour per source IP. Hosted runners
  share an egress IP, so a busy period surfaces as an intermittent, platform-
  correlated lint failure that looks like a flake rather than a rate limit.
  .github/workflows/terraform-module.yml and this repo's own CI now both set
  GITHUB_TOKEN for the 5,000/hour authenticated budget, and a workflow-contract
  test keeps the two from drifting apart again. If you self-host runners, or call
  tflint outside the reusable workflow, set GITHUB_TOKEN in that environment
  too.

Added

• avm test e2e --example <name> targets a single example. It accepts either
  the folder leaf (example-a) or a repo-relative path (examples/example-a)
  and may be repeated. Omitting it keeps the existing behaviour of running every
  runnable example sequentially. A name that does not exist, or that carries a
  .e2eignore marker, is a hard error listing the valid names rather than a
  silent pass, so a typo in a CI matrix cannot skip a real test. (F26)
• avm test e2e --list emits a compact JSON array of runnable example names and
  nothing else, so a workflow can build a matrix with fromJson() and no
  post-processing. A module with no runnable examples emits []. (F27)
• The Terraform reusable workflow fans end-to-end tests out across a per-example
  matrix with fail-fast: false, restoring the governance round-robin
  subscription fan-out so parallel legs do not collide on quota. The matrix is
  skipped cleanly when a module has no runnable examples.
• Every step and sub-step now carries StartTime, EndTime and a duration, both
  on the result objects and in the rendered output. (F29)
• RUNNER_DEBUG=1 (set when a workflow is re-run with debug logging) turns
  verbose on, and verbose now cascades from the entry point to engines,
  sub-cmdlets and Invoke-AvmProcess. AVM_VERBOSE=1 does the same outside
  GitHub Actions. (F30)
• avm test unit and avm test integration report RunsTotal, RunsPassed and
  RunsFailed alongside FilesProcessed, and render the tally. A file count is a
  poor coverage signal; a run count exposes an empty suite immediately. (F33)

Changed

• All module output routes through a single writer with levels, which is GitHub
  Actions aware (::group::, ::error::, ::warning::, ::debug::) instead of
  mixing Write-Host, Write-Information and Write-Verbose. (F31)
• Subprocess output is quiet by default and replayed in full at the point of
  failure. Verbose or debug streams it live; GitHub Actions wraps it in a
  collapsed ::group:: so the log is present but not noisy. Long-running
  sub-steps emit a periodic elapsed-time heartbeat so a slow deploy is
  distinguishable from a hang. (F28)
• avm renders one summary per invocation instead of one per emitted result
  object, and labels per-item rows with the item's own identity, so
  avm tool list no longer reads as six conflicting statuses for one command.
  (F25)
• avm no longer writes result objects to the success stream by default; pass
  --passthru to get them back for scripting. (F23)
• Both test tiers now invoke terraform test identically and render a progress
  line per test run with its duration, instead of the unit tier running silently
  and the integration tier dumping raw -json NDJSON on the human channel. The
  terraform init sub-step follows the same quiet-by-default rule as everything
  else. (F33)
• Caller-rendered progress is no longer collapsed behind a GitHub Actions
  ::group::. Grouping exists to fold away raw child output, and a sub-step that
  supplies its own line renderer emits no raw output, so the fold hid exactly the
  curated per-run progress it was meant to reveal. (F33)

Fixed

• The release workflow no longer fails at the point of publishing. The build
  task stamps the tag's whole CHANGELOG section into the staged manifest's
  ReleaseNotes, and the PowerShell Gallery rejects any package whose
  ReleaseNotes exceeds 10600 characters — 0.1.7's section is 23987, so the
  first attempt was rejected with 400 (The package is invalid) after the tag
  and the GitHub Release already existed. Get-AvmReleaseNotes.ps1 gained
  -MaxLength, which truncates on a line boundary from the bottom (a section
  leads with ### Breaking, so the tail is the safe end to lose) and appends a
  link to the full notes on the GitHub Release, which has no such limit. Newlines
  are measured at their CRLF worst case, because the packer may rewrite them on
  the way into the nuspec. The build task then re-reads the stamped manifest
  and fails locally if the value is still over the limit, so the cap is verified
  against what actually ships rather than trusted from the generator. (F49)
• -CheckDrift is now genuinely read-only for docs and transform. format
  already checked without writing, but terraform-docs and mapotf have no
  dry-run mode, so those two engines detected drift by writing first and
  comparing hashes afterwards — leaving the caller's working copy rewritten by a
  command whose whole job is to report rather than fix. Both now snapshot the
  managed files before the tool runs and restore them from a finally block, so
  the tree is byte-identical even when the tool throws part-way through. On an
  ephemeral CI runner this was invisible; locally, avm pr-check silently
  rewrote two of the three managed-content areas. Drift detection itself is
  unchanged — the tool still runs and the changed-file list is still computed
  from real hashes. (F44)
• avm pr-check gates formatting and documentation instead of silently fixing
  them. format and docs had no -CheckDrift parameter — only sync and
  transform did — so two of the four managed-content steps could not gate at
  all. Both cmdlets gained the switch and all four steps now use it. Issues
  carry relative, forward-slashed paths, so a drifted file is annotated inline
  in the pull request. avm pre-commit is unchanged and still auto-fixes.
  (F42)
• A shell hook is now only a misconfiguration when it has no PowerShell
  counterpart. The guard rejected the mere presence of setup.sh,
  teardown.sh, pre.sh or post.sh, but upstream AVM governance ships both
  a .ps1 and a .sh side by side — so avm test unit, avm test integration and avm test e2e threw on every governance-compliant module.
  A .sh only matters when there is no .ps1 beside it, because only then
  does the hook silently never run. (F39)
• The test suite result object now carries RunsTotal / RunsPassed /
  RunsFailed even when a tier ships no .tftest.hcl files at all. That case
  previously returned Status='pass' with FilesProcessed=0 and no run-count
  members, so the one shape that most needs to be conspicuous — a module with no
  tests — was the one shape indistinguishable from a real result. The tier also
  now says so on the console. (F38)
• That same empty tier now reports Status='skipped' rather than pass, in the
  suite engine and the e2e engine alike. Run counts made the gap legible on the
  object, but inside a gauntlet the rendered status is what an author reads, and
  pass there is indistinguishable from a real one. Fixed in the engines rather
  than the gauntlet, so avm test unit on its own is honest too. (F40)
• A failing avm command reports a clean one-line failure summary and exits
  non-zero instead of surfacing a raw OperationStopped stack trace pointing at
  module source. The remainder of a calling script no longer stops running, so
  cleanup lines after an avm call execute as written. (F24)
• The failure headline now prefers a diagnostic that carries a file position.
  A failing terraform test records a bare test run '<name>' fail issue ahead of
  the diagnostic naming the file, line and cause, so the console summary, the
  GitHub Actions error annotation and the terminating error message all carried
  the least useful line while the actionable one sat in the body. (F24)
• A failed command now emits exactly one GitHub Actions annotation, anchored on
  the failing file, line and column so it renders inline on the offending line in
  the PR Files-changed view. A single failing test run previously produced three
  ::error:: annotations — the positionless progress line first, the actionable
  diagnostic last — and none carried a position. Annotations are capped at ten
  per step, so a badly broken PR could push the useful ones past the cap and show
  a reviewer nothing but run … -> fail. Narration (the per-run progress line,
  the subprocess FAILED: / TIMEOUT: lines and the gauntlet step error line)
  is now plain text and can no longer crowd out the diagnostic. Paths are
  normalised to forward slashes and rebased on GITHUB_WORKSPACE, because
  GitHub only anchors repo-relative paths; a diagnostic with no position still
  produces an unanchored annotation rather than none. Console output outside
  Actions is unchanged. (F41)
• A gauntlet step that fails or errors now narrates its error message at the
  point of failure, so callers that invoke Invoke-AvmPrCheck or
  Invoke-AvmPreCommit directly (bypassing the dispatcher's renderer) get a
  diagnosis instead of a bare status.
• terraform init now receives the same -test-directory as terraform test.
  Init only scans the default tests/ directory when resolving modules declared
  inside .tftest.hcl run blocks, so with AVM tiers under tests/unit/ and
  tests/integration/ a run { module { source = "./tests/integration/setup" } }
  helper was never installed and the tier failed immediately with
  Module not installed. This affected a cold checkout identically, so CI was
  hit as well as local runs, and it made the standard HashiCorp pattern for
  prerequisite infrastructure unusable. (F34)
• terraform init now always runs before terraform validate and
  terraform test unless -NoInit was passed. The previous gate on the
  existence of .terraform/ only proved init had run at some point, against a
  possibly different set of requirements, so bumping or adding a module
  dependency was not picked up on a warm working directory. CI never saw this
  (every job is a fresh checkout); it only bit locally. -backend=false keeps
  the warm path cheap. (F32)
• avm check policy reported pass without evaluating a single policy, and
  now reports skipped. conftest was invoked with no namespace selector, so
  it evaluated only its default main namespace — and none of the 266 bundled
  APRL/AVMSEC .rego files declares package main. Zero policies ran, exit was
  0, and the step passed in ~320ms, which is about what doing nothing costs. One
  of nine pr-check gates had never been able to fail on any AVM module. The
  engine now counts what conftest actually evaluated, exposes it as Evaluated,
  and reports skipped with an avm.tf.policy-not-evaluated diagnostic when
  nothing could have been checked. skipped does not flip a gauntlet's overall
  status, so no module breaks; under GitHub Actions it surfaces as a single
  ::warning::, which makes the gap visible in every run instead of invisible.
  Deliberately not fixed by adding --all-namespaces: every input accessor in
  the bundles destructures terraform show -json shapes, so that flag alone
  would evaluate all 260 rules against an input none of them can read and count
  every one as a success — slower, more convincing, still no gate. The second
  skip reason exists precisely to catch that state, and both reasons key off the
  parser mode used to build the argv, so they retire themselves when the
  plan-JSON input path lands. Policy enforcement is therefore still a follow-up
  slice — this release stops it claiming otherwise. (F46)

Tests

• Mock-invocation assertions are pinned to exact counts. The original claim
  under this heading was wrong: Should -Invoke -Times 0 is not vacuous on
  Pester 5, which special-cases zero as exact — that behaviour is Pester 4's,
  and the repo pins [5.5.0,) everywhere. The genuinely loose assertions were
  the positives: -Times 1 without -Exactly means "at least once", and 15 of
  them sat in the two gauntlet suites. A double-invoke injected into
  Invoke-AvmPrCheck was caught by exactly one test — the F42 drift gate, the
  only one already using -Exactly; the eight-assertion compose test passed
  with every step invoked twice. All 15 are now -Exactly, and
  Invoke-AvmTestUnit gained the compose assertion it never had. (F43, F45)

• The conftest component-tier stub gained an AVM_STUB_CONFTEST_OUTPUT hatch,
  so the component tier proves check policy can go red end-to-end rather
  than only proving it stays quiet. The previous stub emitted [] and the
  fixture asserted pass on it — the vacuous case encoded as the expectation,
  which is how F46 survived a green suite. The real-binary integration tier had
  encoded the same false expectation against genuine conftest and the
  genuine pinned bundles, and now asserts the honest skipped /
  Evaluated = 0 / avm.tf.policy-not-evaluated triple instead. Pinning the
  evaluated count doubles as a canary: if a future bundle re-packages into
  conftest's default namespace, that assertion fails loudly rather than the gate
  silently going quiet again. (F46)

• Anchored every negative-only assertion with a positive control. A negative
  matcher (Should -Not -Match, -Not -Contain, -Not -Be) passes on an empty
  or $null subject, and Should -Not -Throw passes when the operation was
  never reached — so an It whose only assertion is negative cannot distinguish
  "the bad thing did not happen" from "nothing happened at all". Six genuine
  holes were found by forcing each captured value to empty and seeing which
  tests still passed: the avm doctor --json routing test ('' | ConvertFrom-Json does not throw), the Get-AvmVersion PSVersion cast
  ([version]$null does not throw), the avm test e2e -List discovery surface
  (an empty array would collapse the CI matrix to zero legs), the module-version
  encoding test (a zero-byte manifest satisfied both BOM and CRLF negatives), the
  dispatcher's non-zero-exit test, and a Write-AvmLog verbose test that was
  worse than vacuous — it captured -InformationVariable for output that only
  ever reaches stream 4, so it could never fail in either direction. Each fix is
  proven load-bearing by mutating the source and confirming the new anchor
  catches. (F47)

• Pinned the avm test e2e -List discovery contract under GitHub Actions. The
  reusable workflow feeds that string straight to fromJson() to build the
  per-example e2e matrix, so a single ::group:: marker leaking onto the same
  channel collapses every e2e leg — and the F31 logging overhaul emits those
  markers only when GITHUB_ACTIONS is set, so no existing test ran under the
  condition that enables them. Proven decisive by mutation: an
  Actions-conditional leak is caught by this test and by nothing else in the
  suite (31 passed / 1 failed), where the same leak made unconditional is caught
  by four. (F47)

• Documented the output-capture discard trap and the sweep that finds it
  (Appendix L.10). | Out-String yields nothing when a terminating error unwinds
  the pipeline, which turns any negative matcher on the capture into a vacuous
  pass; redirect to a file instead. The sweep pattern is \d?\*?>&1 — an earlier
  audit that grepped only *>&1 and 6>&1 missed two live 2>&1 sites. Each of
  the four capture sites in the suite is safe for a different reason, so the
  reason is now recorded alongside what would break it. (F47)

• avm check policy now stages the module's *.tf / *.tfvars into a temporary
  directory and runs conftest there, instead of pointing --parser hcl2 . at the
  repository root. conftest walks every file under its working directory, so on a
  real module it hit .gitignore (or .editorconfig, or .github/**/*.yml),
  died in parse configurations before loading a single policy, and exited 1
  with no output. That is on every AVM repository in existence: measured on the
  integration fixture, conftest reported
  parse config: [:1,1-2: Argument or block definition required], path: .gitignore.
  After the fix the same fixture runs to completion and reports
  namespaces seen: main. Naming the Terraform files as arguments — the obvious
  alternative — does not work: conftest 0.68.2 on Windows then resolves --policy
  paths relative to the named input and loses the drive letter.

  The staging directory lives under the AVM cache rather than the system temp
  directory, and that is load-bearing for the same reason. conftest strips the
  drive letter from --policy regardless of the input form, so an absolute bundle
  path only resolves when the working directory is on the same volume. With
  the bundles on C:, staging on C: exits 0 and staging on Q: exits 1 with
  GetFileAttributesEx \Users\…\avm-policy-aprl\…: The system cannot find the path specified. The bundles live under the cache root, so staging there shares a
  volume by construction on every OS. This was latent until the parse abort above
  was fixed — that crash happens first and masked it — and it surfaced on Windows
  CI, where the checkout and AVM_HOME are on D: while the temp directory is on
  C:. Reproduced locally by pointing AVM_HOME at a second volume: the released
  arrangement reports error / avm.tf.policy-run-failed / loading policies,
  and the fixed one reports namespaces seen: main. (F48)

• avm check policy no longer reports a conftest crash as the F46 vacuity skip.
  conftest reuses exit 1 for "a policy failed" and "I aborted before evaluating
  anything", and the engine only treated codes other than 0 and 1 as a
  malfunction — so a crash fell through to the normal path, produced zero records,
  and was reported with the diagnostic "the bundles declare no rules in the
  default 'main' namespace". True about the bundles, false as the reason. A
  non-zero exit with no parseable output is now Status='error' carrying
  conftest's stderr under avm.tf.policy-run-failed. This outlives F46: that
  guard self-cancels once the plan-JSON slice lands, this one does not. A module
  holding no Terraform sources at all is detected before conftest is launched and
  stays a skip, because nothing to check is not a failure. (F48)

• Stopped the test suite ratifying the crash above. The integration and component
  tiers both asserted Evaluated -eq 0 — a value both causes produce, so the
  assertions passed for the crash for as long as it existed. They now pin the
  cause (namespaces seen: main), which only a completed run can report. The
  component stub's default output was [] with a comment claiming that was what
  real conftest emits; it is not — that is the crash shape. The stub now
  enumerates its working directory, emits one main-namespace record per input,
  and reproduces the parse abort if a non-Terraform file reaches it, so the
  component tier witnesses the staging too. (F48)

---

What's Changed

• fix: e2e per-example matrix, logging overhaul, and gauntlet honesty (F23-F48) by @jaredfholgate in #17

Full Changelog: v0.1.6...v0.1.7