audit --ci: verify deployed file content, not just existence

[srid]

apm audit --ci should verify file content, not just existence

Problem

apm audit --ci's baseline deployed-files-present check (ci_checks.py:106-137) only verifies that each file in the lockfile's deployed_files exists on disk — it does not check whether the content matches what apm install would produce. This means:

• A locally edited .claude/rules/workflow.md passes the check as long as the file exists
• An upstream source change in a remote package is invisible if the local copy was never reinstalled

The policy-layer unmanaged-files check (policy_checks.py:606-690, requires --policy) covers the adjacent problem of orphan detection — files on disk that aren't in the lockfile. But neither layer verifies content fidelity.

The only reliable way to detect content drift today is to run apm install and compare. But running it on the live tree briefly destroys the vendored directory (if doing a clean reinstall to catch orphans from packages that changed shape). We hit this in juspay/kolu#468: CI ran apm install on the live .claude/ while a Claude Code session was active in the same worktree. Claude Code's stop hook crashed because .claude/hooks/ didn't exist mid-reinstall.

What we had to build instead

We built a scratch-dir verification (juspay/kolu#469): symlink inputs into a temp dir, run apm install there, diff -r against the live tree. This works but required working around several behaviors:

1. No --root / --target-dir flag

apm install always writes relative to $PWD. To install into a scratch directory, we had to cd into it and stage all inputs (symlinks for apm.yml + local packages, cp for apm.lock.yaml). A --root <dir> flag would make this trivial.

2. -t claude doesn't match auto-detection behavior

When we used -t claude to force the target in the scratch dir, local package integration silently produced no output — rules and commands from the local ./agents package were not deployed. But when we created an empty .claude/ directory in scratch (triggering auto-detection via target_detection.py:82), the same local package correctly deployed 10 rules and 1 command.

This means -t claude and "auto-detect claude from .claude/ existing" are not equivalent code paths.

3. apm install writes generated_at into lockfile on every run

Even when nothing changed, apm install updates apm.lock.yaml's generated_at timestamp. This means the lockfile can't be symlinked for read-only verification — the symlink writes through and mutates the real file. We had to cp it instead.

Suggested improvements

Content verification in audit --ci — the most natural home. Two possible approaches:

• Content hashes in the lockfile — if apm install recorded a content hash per deployed file in apm.lock.yaml (alongside the path in deployed_files), the baseline deployed-files-present check could verify hash(live_file) == lockfile_hash without needing a scratch install. Fast, simple, no disk writes.
• Internal scratch install + diff — audit --ci stages a fresh install in a temp dir and compares content against the live tree. More expensive but catches everything including orphans (complementing the policy-layer unmanaged-files check).

Promote unmanaged-files to baseline — orphan detection is useful even without an org policy. Consider making it a baseline check (possibly with a default warn action) so projects get it without needing --policy.

Secondary fixes:

• Fix -t claude to match auto-detection — the silent no-op on local packages is a bug regardless
• Skip generated_at write when content is unchanged, or add --no-lockfile-update
• Add --root <dir> for scratch-dir verification

Environment

• apm 0.8.11 (via uvx --from git+https://github.com/microsoft/apm)
• Workaround PR: juspay/kolu#469

[sergio-sisternes-epam]

Thanks for this thorough write-up, @srid -- real-world CI scenarios like this are exactly the feedback that shapes APM's roadmap.

Accepting as an enhancement. Here's how I see the items you've raised:

Primary track (this issue) -- content verification in audit --ci:
The content-hash-in-lockfile approach is the most promising direction. Recording a hash per deployed file in apm.lock.yaml would let the baseline deployed-files-present check verify fidelity without a scratch install. We'll keep this issue focused on that feature.

Sub-items worth separate issues:
Your report surfaces several distinct concerns beyond content verification. If you'd like to see them tracked, we'd welcome you filing them as individual issues (referencing this one for context):

1. -t claude doesn't match auto-detection behavior -- local packages silently produce no output with -t claude but work with auto-detection via .claude/ directory. This is a bug.
2. --root flag -- allow apm install to target a directory other than $PWD, enabling scratch-dir verification workflows.
3. --no-lockfile-update / skip generated_at write -- avoid mutating the lockfile when content hasn't changed.
4. Promote unmanaged-files to baseline audit -- orphan detection without requiring --policy.

Your workaround in juspay/kolu#469 is a great reference implementation for the scratch-dir approach. Thanks for sharing it!