Add proximal L1 solver path

[MaxGhenis]

Summary

Adds a real proximal L1 selection path to populace-calibrate while preserving compatibility with existing solver configs.

• Adds method="prox" with l1_lambda, optimizing non-negative weight ratios and applying an L1 soft-threshold step so records can be selected to exact zero.
• Keeps the existing Adam/log-weight path as the default and records it honestly as method="adam".
• Preserves method="apg" as a deprecated alias for "adam"; old configs still run, but result options normalize to "adam" so new manifests do not keep the misleading label.
• Rejects incompatible combinations (l1_lambda outside prox, prox with L0/budget pruning, prox with l2_lambda).
• Adds solver-owned errors for prox all-zero outputs and prox conserve+cap infeasibility so failures do not leak generic frame or L0 messages.
• Updates the public calibrate docs/README for the two solver paths.

Validation

• uv run ruff format packages/populace-calibrate/src/populace/calibrate/solve.py packages/populace-calibrate/src/populace/calibrate/__init__.py packages/populace-calibrate/tests/test_solve.py --check
• uv run ruff check packages/populace-calibrate/src/populace/calibrate/solve.py packages/populace-calibrate/src/populace/calibrate/__init__.py packages/populace-calibrate/tests/test_solve.py
• git diff --check
• uv run --project packages/populace-calibrate --group dev python -m pytest packages/populace-calibrate/tests/test_solve.py -q -> 42 passed
• uv run --project packages/populace-calibrate --group dev python -m pytest packages/populace-calibrate -q -> 106 passed, with existing torch sparse warnings

Review notes

This supersedes the earlier old-base version of #184. Two review cycles were run. The first found prox projection wording and missing l1_lambda docs; both were fixed. The second found the apg rejection compatibility break, a prox all-zero error leak, and stale Adam-only public docs; all three were fixed here.

[MaxGhenis]

Review: PolicyEngine/populace PR #184

Recommendation: REQUEST_CHANGES

Findings

1. [CRITICAL] The L1 path does not implement the objective it documents and records.

In packages/populace-calibrate/src/populace/calibrate/solve.py, _optimize_proximal documents the objective as capped weighted-MAPE plus l1_lambda * mean(w_i / w0_i), and result metadata records l1_penalty="mean_initial_weight_ratio_abs". But the update at lines 685-690 normalizes the smooth gradient step by RMS, making the effective step size learning_rate / rms, while the soft-threshold is fixed at learning_rate * l1_lambda. For the documented mean penalty, the threshold should use the same effective step size and the mean divisor, approximately (learning_rate / rms) * l1_lambda / n. The current update is therefore not the proximal operator for the recorded objective; it behaves like an unnormalized heuristic whose lambda scale changes with record count and gradient scale. That is especially risky because this PR is meant to supply a solver/provenance path for l0-paper comparisons.

Relevant lines: packages/populace-calibrate/src/populace/calibrate/solve.py:627, :632, :648, :652, :685, :690, :1289.

Suggested fix: choose one contract and make code, docs, metadata, and tests agree. If the contract is mean(w/w0), use the actual effective step in the prox shrink and divide by n. If the desired behavior is a budget heuristic, rename the metadata away from an objective penalty and stop claiming it minimizes target_loss + lambda * mean(r).

Test Gap

The new tests show that prox can produce exact zeros and that pruning is monotone on one fixture. They do not test the L1 scale/objective contract. Add a small deterministic one-step prox test or a duplicate-record invariance test for the declared mean penalty.

Other Notes

The PR is mergeable but the branch is three commits behind the current #182 base. It should be rebased after the solver issue is fixed. No GitHub checks were reported for this branch during review; local focused test_solve.py passed 37 tests.

[MaxGhenis]

Fixed the review finding and pushed the update to #184.

What changed:

• Rebases the branch onto the current Treat PE-US aggregate outputs as formula-owned #182 base (dcdde76).
• Keeps the documented L1 objective as l1_lambda * mean(w / w0).
• Changes the prox shrink to use the same effective smooth-step size and divide by n, so the implementation matches the recorded mean_initial_weight_ratio_abs provenance.
• Adds a one-step zero-gradient regression test that would fail if the code drifts back to an unnormalized sum-penalty shrink.
• Updates the PR body so the L1 formula and validation examples match the pushed code.

Verification:

• ruff format / ruff check on the touched files.
• pytest packages/populace-calibrate/tests/test_solve.py -q → 38 passed.
• pytest packages/populace-calibrate/tests -q → 99 passed, with existing torch sparse warnings.

[MaxGhenis]

Follow-up after the pushed fix: the L1 proximal update now matches the documented objective, l1_lambda * mean(w / w0), because the shrink threshold uses the same effective smooth step size and divides by record count. The new one-step regression test covers the scale contract that the original review flagged.

Validated locally with pytest packages/populace-calibrate/tests/test_solve.py -q and pytest packages/populace-calibrate/tests -q. I am leaving this as a comment review rather than approving my own pushed fix.

[MaxGhenis]

Follow-up after the second review pass: I pushed 99b4f36 to address the remaining solver-contract issues. method="prox" now rejects l2_lambda > 0 instead of recording an L2 objective it does not optimize, and the L1 scale tests now cover both the zero-gradient / n shrink and a nonzero-gradient case that pins use of the RMS-normalized effective step.

Validated with uv run --package populace-calibrate --group dev python -m pytest packages/populace-calibrate/tests/test_solve.py -q, uv run --package populace-calibrate --group dev python -m pytest packages/populace-calibrate/tests -q, uv run --no-sync ruff check packages/populace-calibrate/src/populace/calibrate/solve.py packages/populace-calibrate/tests/test_solve.py, and git diff --check. Existing torch sparse warnings only.