populace-calibrate: representation operator (targets -> calibrated weights) - #3
Merged
Conversation
…e uncommitted work) Core modules (target/matrix/solve/gates + compat tests) built by the build agent before it was killed mid-test-authoring. Committing to preserve; tests + verification to follow. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… mode Complete the killed agent's shard: behavioral contract suites for the solver (loss reduction + feasible-target hit, CALIBRATED kind, mass free/conserve, hard weight-ratio bound + landmine prevention, multi-period one-weight stacking, L0 record-budget pruning), targets (aggregations, column/callable measure, validation), and the constraint matrix (shape, multi-period rows, skipped-target reporting). Two fixes found while testing: (1) make the max_weight_ratio bound exact on the returned float64 vector (the per-step clamp is float32 and can overshoot the bound by ~1e-7); (2) --import-mode=importlib at the root so the shards can share test-file basenames (each ships a test_compat.py). 23 calibrate tests; 234 passed workspace-wide. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…==target-1 Adversarial review found the eCPS +1-in-numerator loss is minimized at est = target - 1, not target — negligible at $2T magnitudes but fatal for small targets (a count of 5 converged to 4.0). Drop the +1 from the numerator (keep it in the denominator regularizer); this is the loss the docstring always described. Regression test: a count target of 5 now converges to 5.0. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…om guard) Finding 2 (HIGH): the eCPS relative-error loss divides each residual by (target_value + 1). A target value of exactly -1 -- a raw value=-1, or a mean target whose value is exactly 1 below the current mean (compiled RHS is value - current_mean) -- zeroes that denominator. The loss goes NaN, gradients go NaN, every weight goes NaN, and the only surfaced error was the kernel's opaque "Weights must be finite; found N NaN", naming neither the target nor the cause. build_constraint_matrix now validates the compiled target_vector and raises a clear ValueError naming the offending target(s) and the cause when any |target_value + 1| < 1e-8. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Finding 5 (HIGH): build_constraint_matrix built a target's constraint row at a
person-aligned linearization point (via _entity_row's cross-entity collapse) but
called target.offset(frame, w0) with the GROUP weight vector. For a person-entity
mean target on a multi-person frame, offset builds person-length arrays, so
filter_mask * weights broadcast (n_persons,) against (n_households,) and raised
ValueError -- caught and recorded as a SkippedTarget with a raw-numpy reason
("operands could not be broadcast together..."). The target was silently dropped.
Resolve the linearization point once, aligned to the target's own entity
(_linearization_weights), and hand the same vector to both _entity_row and
offset. _entity_row now receives the already-aligned weights instead of
re-deriving them, so the row and its offset share one point.
Adds a multiperson_frame fixture (exercising the previously-unused
persons_per_household machinery) and tests a person sum and a person mean target
across real multi-person households.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ords Finding 4 (HIGH): with L0 pruning, mass='conserve', and a max_weight_ratio cap, _project_to_total filled the post-pruning mass deficit by additive redistribution over ALL records with headroom (cap - weight > 0), INCLUDING the gate-closed (~0-weight) pruned ones -- resurrecting them. A run that pruned hundreds of records under free mass returned zero pruned under conserve+cap. _project_to_total now takes a `pruned` mask (the gate-closed records, by the same prune threshold the result's n_nonzero uses) and redistributes the deficit only over surviving records, holding pruned ones at zero. If the survivors lack the headroom to absorb the freed mass under the cap, it raises a clear ValueError naming pruning + conserve + cap as jointly infeasible, instead of silently un-pruning to balance the books. The no-pruning path is unchanged. Finding 7 (also fixed): max_weight_ratio < 1 with mass='conserve' is infeasible a priori (sum of caps < input total) -- now rejected in argument validation with a named error rather than surfacing later as the kernel's opaque mass-conservation failure. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ad param) Finding 3 (HIGH): target_records was dead -- it only appeared in `if l0_lambda > 0.0 or target_records is not None:`, so the NUMBER never entered the optimization. Pruning was controlled entirely by l0_lambda, and budget 10 vs 350 at the same lambda produced bitwise-identical weights despite the docstring promising "prune toward target_records non-zero weights". Implement honest budget control: when target_records is set, the solver searches l0_lambda by bisection on log10(lambda) -- the realized non-zero count is monotone decreasing in the penalty -- so the achieved count tracks the budget. Each evaluation is a full _optimize reseeded to `seed` (deterministic count-vs-lambda response); the search keeps the best run seen, stops early within ~5% of the budget, and is bounded by `budget_iters` optimizations (new param, default 10). A supplied l0_lambda is the warm start. The result reports the penalty the search settled on as l0_lambda. Achieved vs budget on the n=400 fixture (epochs=250): 50->49, 100->100, 150->156, 200->210, 250->238, 300->306; lambda decreases monotonically with the budget. l0_lambda alone (no target_records) still prunes at a fixed penalty, unchanged. Updates the solve/init docstrings and README to describe budget control. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…in diagnostics Two diagnostic-correctness notes from the review (6 and 8), both making the reported numbers describe the weights actually returned. Finding 6 (MEDIUM): a `mean` row is linearized about the input weights, so `A @ w` is the linearized value, not the achieved ratio. After a large mass move the diagnostic reported a near-zero relative_error and a spurious within_tolerance=True even when the true achieved mean missed the target and its tolerance. Target.achieved_value now computes the true aggregate (sum/count exact, mean as the true ratio sum(measure*filter*w)/sum(filter*w)), and _build_diagnostics uses it for mean targets -- against the user's declared target value, not the linearization's shifted RHS. On a 1.5x-mean run the diagnostic now reports the true ratio (~63k, real -0.8% miss) instead of the linearized ~21k. Finding 8 (cheap): final_loss returned loss_trajectory[-1], a pre-step, pre-projection value that does not describe the returned weights (the closing mass/cap projections move them). CalibrationResult now stores closing_loss -- one eval-mode eCPS relative-error loss on the returned weights -- and final_loss returns it. On a conserve+cap run the two differ by ~0.8%. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Budget control (the target_records fix) replaced the old "auto-select a small default penalty when target_records is given without l0_lambda" behavior, leaving _DEFAULT_L0_LAMBDA defined but unreferenced. Remove it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…search infeasible probe) From the verification review of the review-fixes: - Update the loss formula in 4 doc spots (module/calibrate docstrings, __init__, README) to the implemented ((A@w-b)/(b+1))^2 — the +1 was dropped from the numerator but the docs still showed it. - Harden the L0 budget search: an intermediate bisection probe can over-prune past the conserve+cap feasible floor and trip the infeasibility raise; catch it, steer toward a smaller penalty, and only raise (naming all three causes) if no penalty is feasible. A feasible budget under conserve+cap no longer crashes mid-search. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…sed failure Both x86-CI failures were knife-edge test designs, not solver bugs — but chasing them surfaced one real solver gap: - prune+conserve+cap test: a fixed l0_lambda prunes platform-dependently (tens of survivors on arm64, 3 of 400 on x86, which fell below the cap-feasibility floor of n/ratio = 4 and tripped the infeasibility guard; a cap loose enough to make any count feasible instead lets the conserve rescale feed an all-gates-closed collapse). The run now uses target_records budget control, which adapts the penalty to a survivor count on any platform; the resurrection regression is still caught (under the bug every probe returns ~400 nonzero). - solve: an L0 run whose gates all close returned an all-zero vector and died downstream in the kernel's opaque "Weights cannot be all zero". calibrate now raises a named error (cause: the penalty; remedies: lower l0_lambda or budget control) on the direct path; budget-search probes already steer off zero counts. New contract test. - fit tail-share test: a single predict() round's share noise (~0.0011 at n=5000) matches the ~0.0015 nearest-snap bias under test, so the one-draw comparison was a platform coin flip (passed arm64, failed x86). Both shares now average 5 independent predict() rounds, putting the bias gap near 3 sigma. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The third shard —
populace.calibrate, the representation operator (DESIGN.md "populace-calibrate: representation"). Targets become a sparse linear constraint system on a Frame's weight vector; torch Adam over log-weights minimizes the eCPS relative-error loss to produce CALIBRATED weights.Features (each a behavioral contract test)
mass="free"/"conserve"— total moves to fit, or is held exactly.max_weight_ratio— a hard per-record bound (exact on the returned vector), the landmine guard; a test shows it prevents a rare near-zero-weight donor's weight from detonating to hit a tail target.(target, period)rows stack over one weight vector ("one weight per trajectory").target_recordsbudget (generate-big-then-prune).Grounded in the proven eCPS reweight loss/optimizer. Built largely by an agent (killed mid-test-authoring; work preserved and finished). 23 tests; workspace 234 passed.
🤖 Generated with Claude Code