feat(megatron): add sampled Flux weight-state audit

[gphuang]

Summary

Tracks Issue #220, Task 9 Option 9.

Add an opt-in Flux audit that records deterministic rank-zero parameter samples after selected completed training iterations. This lets early MXFP4, FP8, and BF16 forks compare model drift without extra full optimizer checkpoints or changes to Torch RNG state.

• Sample after the model forward so overlapped distributed-optimizer gathers have refreshed every parameter buffer.
• Record per-parameter shape, dtype, finite counts, sampled moments, and SHA256.
• Publish one strict JSON file per completed iteration using atomic, no-overwrite semantics; identical restarts are idempotent and conflicting output fails closed.
• Skip validation and synthetic warmup, and explicitly record completed-iteration, next-iteration, forward-counter, and microbatch coordinates.

Test plan

• Changed modules compile with py_compile
• git diff --check
• CI formatting and dependency checks
• Targeted audit-marker tests in the pinned v26.5 container
• Real Megatron wrapper smoke verifies post-gather step-5,120 and step-8,192 snapshots

[gphuang]

Verdict: weight-audit review passed

Ready to merge at 21ae58d01b73f6be990f4837a3efa98e275d67b4. The review found no remaining blockers.

• Iteration coordinates now use Megatron's optimizer state, independent of changing or replayed microbatches.
• Impossible terminal selections fail on every rank before model collectives; sampling remains after parameter-refresh hooks.
• Atomic restart handling is type-strict: identical payloads are idempotent, while changed weights or provenance fail closed.
• Checks: pinned Black formatting, py_compile, focused CPU harnesses, clean diff, and all available CI passed.

[gphuang]

Verdict: ready to merge

Full base-to-head review found no blocking findings after the TP validation fix and public-history sanitization.

• Lint, dependency review, Docker build, JAX, Torch, coverage, and docs checks passed.
• TP=1, TP>1 validation participation, broadcast dtype, replay identity, and atomic publication contracts were reviewed.
• No credentials or internal infrastructure identifiers remain in the rewritten branch history.