[BUG] apm compile produces different Build IDs on macOS vs Linux

[Coolomina]

Describe the bug

apm compile produces a non-deterministic Build ID that differs between macOS (APFS) and Linux (ext4). Running the same command against identical source files on the two platforms yields different AGENTS.md content and therefore a different <!-- Build ID: ... --> hash.

This has two practical consequences:

• CI (In Github, Ubuntu) always reports a dirty/changed AGENTS.md when the file was last compiled on macOS by a developer.
• apm compile cannot be used in pre-commit hooks: a hook that verifies the compiled output is up-to-date (e.g. git diff --exit-code AGENTS.md) will always fail on Linux/CI for files compiled on macOS, making the check unreliable and unusable.

The same non-determinism affects all compile targets (distributed AGENTS.md and CLAUDE.md paths) since the root cause is shared across the compilation pipeline.

Root cause

Two compounding issues:

1. context_optimizer.py — _get_all_files() uses os.walk() without sorting dirs or files. os.walk() returns entries in filesystem-native order, which differs between APFS and ext4. This makes instruction discovery order platform-dependent.

2. template_builder.py, distributed_compiler.py, claude_formatter.py — instructions within each ## Files matching pattern group are written in discovery order without sorting. Different discovery order → different file content → different SHA-256 hash → different Build ID.

To Reproduce

Requires Docker. Files are created natively inside the Linux container (ext4) to avoid virtioFS mount order preservation masking the issue.

Step 1 — Create a minimal test project on macOS:

mkdir -p /tmp/apm-repro/.apm/instructions

cat > /tmp/apm-repro/apm.yml << 'EOF'
name: repro-project
version: 1.0.0
description: Build ID reproduction
author: tester
EOF

for name in alpha beta gamma delta epsilon; do
  cat > /tmp/apm-repro/.apm/instructions/${name}.instructions.md << EOF
---
description: Instruction ${name}
applyTo: "**/*.ts"
---
# ${name} rules
Content from the ${name} instruction.
EOF
done

Step 2 — Compile on macOS:

cd /tmp/apm-repro && apm compile --single-agents && grep "Build ID" AGENTS.md

Expected: <!-- Build ID: 8ded1a99e015 -->

Step 3 — Save this script locally as /tmp/linux-repro.sh:

cat > /tmp/linux-repro.sh << 'EOF'
#!/bin/bash
apt-get update -qq && apt-get install -qq -y git > /dev/null 2>&1
pip install -q --root-user-action=ignore apm-cli==0.8.5

mkdir -p /project/.apm/instructions

cat > /project/apm.yml << 'YAML'
name: repro-project
version: 1.0.0
description: Build ID reproduction
author: tester
YAML

for name in alpha beta gamma delta epsilon; do
  cat > /project/.apm/instructions/${name}.instructions.md << INSTR
---
description: Instruction ${name}
applyTo: "**/*.ts"
---
# ${name} rules
Content from the ${name} instruction.
INSTR
done

cd /project && apm compile --single-agents > /dev/null 2>&1
grep "Build ID" AGENTS.md
EOF

Step 4 — Run the script inside Ubuntu:

docker run --rm -v /tmp/linux-repro.sh:/repro.sh python:3.12-slim bash /repro.sh

Expected: <!-- Build ID: 3f55104cd7fd -->

Step 5 — Observe differing instruction order:

Both platforms compiled the same 5 files, but they appear in a different order. macOS (APFS) picks them up in insertion order, Linux (ext4) in inode/hash order:

macOS AGENTS.md:

## Files matching `**/*.ts`
<!-- Source: .apm/instructions/epsilon.instructions.md -->
<!-- Source: .apm/instructions/alpha.instructions.md -->
<!-- Source: .apm/instructions/delta.instructions.md -->
<!-- Source: .apm/instructions/gamma.instructions.md -->
<!-- Source: .apm/instructions/beta.instructions.md -->

Linux AGENTS.md:

## Files matching `**/*.ts`
<!-- Source: .apm/instructions/alpha.instructions.md -->
<!-- Source: .apm/instructions/beta.instructions.md -->
<!-- Source: .apm/instructions/gamma.instructions.md -->
<!-- Source: .apm/instructions/delta.instructions.md -->
<!-- Source: .apm/instructions/epsilon.instructions.md -->

Same files, different order → different file content → different SHA-256 → different Build ID.

Expected behavior

apm compile should produce byte-for-byte identical AGENTS.md (and identical Build IDs) regardless of the OS or filesystem, given the same input files. This is a prerequisite for the command to be usable in pre-commit hooks.

Environment

• OS: macOS 15.x (APFS) vs Linux Ubuntu 24.04 (ext4)
• Python Version: 3.12
• APM Version: 0.8.5

Additional context

This reliably breaks CI: a developer compiles on macOS, commits AGENTS.md, and the CI pipeline (Ubuntu) recompiles and detects a diff.

It also blocks adoption of apm compile as a pre-commit hook step — any check enforcing that AGENTS.md is up-to-date before committing will always fail on Linux for files compiled on macOS, even when no instruction content has changed.