Fix enhanced CPS OOM with half-sample ExtendedCPS

[baogorek]

Closes #580, closes #581, closes #582, closes #583, closes #584, closes #585, closes #586

Summary

• Fix enhanced CPS OOM by building ExtendedCPS from a 50% sample instead of the full CPS, reducing peak memory from ~30GB to ~18GB (Enhanced CPS OOM: revert to half-sample ExtendedCPS input #580, Enhanced CPS OOM during Modal CI builds #584)
• Free memory earlier in the calibration pipeline — delete stale DataFrames and downcast to float32 (Free memory earlier in calibration pipeline #582)
• Fix CI checkpoint cache invalidation: scope checkpoint paths by commit SHA so new commits force a full rebuild instead of restoring stale H5 files from previous builds (CI checkpoint cache never invalidates on code changes #583, CI checkpoint cache never invalidates on code changes #585)
• Widen poverty rate test threshold to 30% to accommodate sampling variance from the half-sample approach (Poverty rate sanity test threshold too tight #581, Poverty rate test threshold too tight for sampled CPS #586)

Details

OOM fix (#580, #584)

ExtendedCPS was consuming ~30GB during construction, exceeding Modal's 32GB limit. The fix samples 50% of CPS households before extending, cutting peak memory to ~18GB while preserving calibration targets (applied post-extension).

Memory cleanup (#582)

Delete the full loss matrix DataFrame after creating the cleaned copy, free the Microsimulation object after build_loss_matrix returns, and convert the loss matrix to float32 before passing to reweight().

Checkpoint invalidation fix (#583, #585)

is_checkpointed() only checked if the file existed and was non-empty — it never checked whether the code had changed. New commits on main kept restoring stale H5 files from previous builds (broken since Feb 4, ~27 commits affected).

Approach: Include the git commit SHA in the checkpoint path:

• Old layout: /checkpoints/{branch}/{filename}
• New layout: /checkpoints/{branch}/{commit_sha}/{filename}

Within the same build (same commit), preemption resilience works exactly as before. Across different commits, checkpoints are invisible because they live under a different SHA directory. Stale commit directories are cleaned up at the start of each build.

Test threshold (#581, #586)

Widen poverty rate test tolerance from 20% to 30% to accommodate sampling variance from the half-sample approach.

Test plan

• CI passes with full rebuild (no stale checkpoints restored)
• Poverty rate tests pass within 30% threshold
• Peak memory stays under 32GB on Modal

🤖 Generated with Claude Code

[MaxGhenis]

Looks good — OOM fix, memory cleanup, checkpoint invalidation, and test threshold all make sense. Added minor nit fixes (cached get_current_commit, moved inline gc import to top-level).