Two-signal FLSA overtime incidence: usual-hours leg joins the reference-week snapshot (#451 item 4 residual) - #526
Merged
Merged
Conversation
…ce-week snapshot The retired ORG/FLSA port derived fsla_overtime_premium from the ASEC reference week alone (A_HRS1 > 40), annualizing one survey week's premium share over the retrospective income year. Measured on certified Build N, that construction collapses incidence to a single week's draw: 19.9M weighted carrier persons (19.0M tax units) against Treasury's published ">29 million" TY2025 No-Tax-on-Overtime claimant floor, with a 65M-person weighted heap sitting exactly at 40 hours, and the reference week not even inside the income year the premium is attributed to. The estimator now adds the income-year-consistent persistent signal: when usual weekly hours worked last year (HRSWK) exceed the threshold, the worker carries and the usual-hours share is the annualizer; usual hours at or below the threshold retain the retired reference-week snapshot leg unchanged. Exemption screens are untouched, the signal gate's positive_without_overtime invariant now requires both measured signals at or below the threshold, and the stage lineage documents the deliberate deviation from the exact retired port with its adjudication. Measured on the certified Build N frame: 12,208 carrier rows / 26.98M weighted persons / 25.73M weighted tax units / $149.5B premium mass (from 8,748 / 19.86M / 19.03M / $114.8B), with every shipped carrier preserved. Part of the populace#451 item 4 overtime-incidence lane.
…ed claims, hardening Blocking 1: a frame whose ORG surface is already populated skipped the whole stage, so a stale premium from the retired construction could silently survive re-entry; the idempotency skip now covers only the semantically unchanged QRF/union/occupation family while the premium is recomputed from the present columns and refreshed on disagreement (poisoned-QRF test proves the skip, consistent surfaces pass through unchanged). Blocking 2 / major 3 / minor 4: the income-year-consistency claim is now channel-scoped (ASEC rows exact; PUF-support rows carry donor-ASEC hours schedules onto PUF income — the support design's property, shared by the retired reference-week leg; recovered carriers split 3.08M ASEC / 4.04M PUF-support on certified N), the false "mass-preserving in expectation" statement is replaced with the honest bias statement for the snapshot leg, and the module overview no longer claims hot reference weeks stopped annualizing for usual-sub-threshold workers (that retained leg is stated as such). Minor 5: the end-to-end test adds a signal-conflict row (usual 45, reference 60) that fails under swapped caller wiring. Nit 6: non-finite hours/income/ weeks inputs read as absent instead of overflowing the straight-time denominator into silent zeros.
…ent, CI-runnable integration tests The function docstring's residual "preserves aggregate mass in expectation" claim about the retired construction is corrected to the honest mass-proxy statement. The populated-surface refresh no longer coerces a stored NaN to zero before comparison, so a NaN-carrying premium column always takes the recomputed clean values instead of surviving as a nonfinite gate failure. The signal gate's structural checks now read the four input signals through the same non-finite-as-absent transform as the estimator (_finite_nonnegative hoisted to module level), so a fabricated premium riding on an infinite hours signal trips positive_without_overtime instead of slipping past it; the premium column itself stays raw so nonfinite outputs remain counted. Both stage integration tests monkeypatch _flsa_policy instead of skipping when policyengine-us is absent, so the always-run base CI enforces them, and the refresh test adds conflicting-signal rows (usual 45 / reference 60) that would surface a refresh-branch hours swap, plus a NaN-premium row proving convergence.
…refresh regression guard The branch is rebased onto origin/main e56b8ad (three commits replayed cleanly; the audited three-file diff content is unchanged) so the combined post-main state is what CI and the suite certify. The populated-surface refresh test gains a third pass on an otherwise-consistent surface whose only difference is a stored NaN on a recomputed-zero row — the exact scenario a fillna(0) comparison would silently preserve. Mutation-verified: reintroducing the coercion fails the test, the shipped code passes it.
Contributor
Author
|
CI green on the exact head (test + wheels, 3.13/3.14) and |
Contributor
Author
|
Owner gate: my review + sol cross-family confirm — items 1–5 pass (statute-faithful two-signal incidence with the port-deviation adjudication note, measured before/after receipts, base-stage-only scope, cache pin fail-closed, coherent tests); sol's single REQUEST CHANGES was the effect-timing wording, corrected in the body (base-stage change ⇒ lands at Build P's base rebuild, not on base-O releases). Merged. |
MaxGhenis
added a commit
that referenced
this pull request
Jul 25, 2026
…rt-year workers (#539) * Concept-align the ORG QRF income feature: full-year equivalent for part-year workers The donor's employment_income is the annualized reference week (pternwa x 52) while recipients fed actual annual income, so part-year workers matched low-wage full-year donors: 61.5% of weeks<50 earners landed below $16.80/hr with band p10 $2.13 and 32% of band mass below the federal minimum on certified N, against 19.4% below the line in the donor itself. Recipients now feed income x 52/weeks (untouched when weeks are zero or missing), the premium derivation keeps actual annual income, and the stage lineage documents the deliberate deviation. Fixes #529. * Address sol round-1 findings on the stacked diff: full re-imputation on populated surfaces, masked divide, lineage disclosure HIGH: the populated-surface skip could pin stale QRF outputs now that the income feature's semantics changed — the stage re-imputes the whole family on every entry and passes an identical surface through unchanged (seeded QRF + deterministic lottery + pure derives make same-code re-entry byte-stable), superseding the premium-only refresh design from the #526 base; regression test proves stale sub-minimum wages and inverted hourly status converge, and the idempotent pass now also pins the hash-lottery union column. MEDIUM: the full-year-equivalent factor uses a masked divide so zero/missing weeks never touch the division (strict-errstate pinned in the feature test). LOW x2: the structured QRF operation declares the weeks_worked dependency, and the lineage notes disclose both the within-quota union reshuffle under the scaled hash and the re-imputation entry semantics. * Address sol delta findings: strict-boolean convergence guard, composition-binding tests, fast-path identity MEDIUM: a boolean output carrying np.nan cast to a truthy True inside _outputs_match (and nullable pd.NA raised), so a damaged column could survive the pass-through or crash it — booleans now compare through strict 0/1 numerics where NaN, pd.NA, and non-canonical values all take the rebuild path; regression test covers both damage classes and pins the healed dtype. LOW: the populated-surface tests now bind the #529 scaling composition (a feature-sensitive fake QRF emits $12.50/hr for the part-year probe if the full-year-equivalent scaling is reverted), assert the planted all-True union column is re-imposed to quotas and a poisoned occupation carry re-derives from POCCU2, and the idempotent pass asserts the SAME frame object returns (the fast path itself, not merely equal values). * Address sol delta-2 findings: dtype-gated convergence guard, full-surface convergence test MEDIUM: coercible numeric strings ("0"/"1") passed the strict 0/1 membership and could identity-match while preserving a damaged str dtype on the fast path — _outputs_match now rejects object and string dtypes outright for every output before any coercion (regression: an int-then-str is_paid_hourly column rebuilds and heals to bool). Partial finding: the weak not-all-True union assertion is replaced by full-surface convergence — the poisoned populated frame must reproduce a from-scratch clean build column-for-column across all twelve outputs, which binds the recomputed union mask exactly, alongside the retained part-year composition pin. * Address sol delta-3 findings: dtype-family allowlist, nonconstant union plant MEDIUM: HDF-storable temporal dtypes (datetime64 at 0/1 ns) coerced through the numeric comparison and could identity-match while preserving the damaged dtype — the guard now allows only the boolean/integer/float dtype families (the only representations the cached Build N/O stores carry) and rebuilds everything else; datetime regression added. MEDIUM: the planted all-True union column was constant, so _surface_has_signal bypassed the populated-surface comparison the convergence test exists to bind — the plant is now a wrong-but-nonconstant pattern, keeping the full twelve-column comparison on the populated path where a dropped union comparison would fail. * Close sol delta-4 LOW: bind the union comparison specifically Tests-only. A union-only damaged surface (inverted, nonconstant, every other column already matching) must converge back — dropping the union column from the convergence guard would identity-pass it unchanged, which the full-surface test could not catch because it exits on earlier poisoned columns.
This was referenced Jul 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two-signal FLSA overtime incidence: usual-hours leg joins the reference-week snapshot (#451 item 4 residual)
The diagnosis (measured, certified N
c3e378a-20260722T010408Z)The overtime surface was measured-known-short: 19.86M weighted
fsla_overtime_premiumpersons vs Treasury's published ">29 million" TY2025 NTOO claimant units; the OBBBA overtime probe at −$16.86B = 51% of JCX-35-25's FY2026 −$32.806B. Decomposed per the #496 forensics pattern (target-frame checkpoint pairing for pre-solve design weights; direct measurement on the published store):(c) Concept — small, and it makes the gap worse, not better. The 19.86M carrier persons collapse to 19.03M carrier tax units (only 0.82M weighted units hold 2+ carriers), so in Treasury's own unit space the gap is 29/19.03 ≈ 1.52×. Universe edges (all measured or sourced): TY2024 frame vs TY2025 claims; claims may include non-FLSA-required overtime under the TY2025 transition-relief reporting (over-inclusive edge on the Treasury side); caps/phaseout shrink deductions, not claimant counts. Per-unit mean $6,032 vs Treasury's ">$3,100" average deduction — the tips signature (fewer carriers, ~2× richer) on the overtime surface.
(b) Selection — exonerated on both margins. The stage runs release-side at build_us_fiscal_refresh_release.py:8141, after the frozen rmloss100 selection (~:7033), so selection could not drop carriers that did not exist yet (the tips forensics result generalizes). In weight space the solve up-weighted carriers: design 18.15M → final 19.86M (+9.4%), design mass $110.98B → final $114.79B. Composition: the frame's usual-hours>40 share among earners (18.3% final / similar design) matches the national CPS ORG 2024 donor (17.0% weighted) — the 57,240 frame is not starving high-hours workers. Zero
overtimetargets in the certified diagnostics (untargeted receipt, re-confirmed from the same diagnostics the #496 tool consumes; running the tool with--target overtimerefuses with "No household-entity target matches").(a) Incidence — the dominant leg, and it is structural. The retired construction gates on
hours_worked_last_week > 40: the ASEC reference week (A_HRS1), a single point-in-time draw that is not even inside the retrospective income year the premium is attributed to. Its own archived docstring states the design: "constructs the premium share implied by the reported/reference week, then applies that share to annual employment income" — a mass proxy that collapses incidence to one week's draw. Measured on the certified frame (final weights):The fix (deliberate deviation from the exact retired port — adjudication)
derive_flsa_overtime_premiumbecomes a two-signal estimator over columns already on the frame:weekly_hours_worked_before_lsr← ASEC HRSWK) above the threshold makes the worker a carrier, with the usual-hours share as the annualizer. A single hot reference week no longer overrides a measured above-threshold usual schedule (measured −$1.4B on the 10.65M double-positive carriers), and a light reference week no longer erases a regular overtime schedule. Year-consistency is channel-scoped: on ASEC-channel rows HRSWK shares the retrospective year withemployment_income/weeks_worked(the reference week does not); on PUF-support rows both hours signals are donor-ASEC schedule proxies attached to PUF income — the support design's property, inherited equally by the retired reference-week leg on those same rows. Channel split of the recovered carriers: 3.08M ASEC / 4.04M PUF-support.positive_without_overtimeinvariant now requires both measured signals ≤ threshold, read through the estimator's non-finite-as-absent transform. Stage lineage (source_stages.json) documents which operations remain the exact 9a823603 port (donor build, QRF, union assignment, occupation carries — all of them) and why the premium gate deviates. Known inherited edge documented: both hours measures count all jobs while FLSA §7 applies per employer.Measured on the certified N frame (final weights):
Every shipped carrier is preserved (the shipped set is a strict subset). Probe measured with the certified-N recipe (store copy + column recompute + the real
us_reform_coverage_smoke_gateat 2026 law; shipped store re-scored in the same engine as the control — certified evidence records −$16.86B, the small drift is engine-version, which the same-engine pair isolates away). Build O in-flight cross-check (22bd902 sparse): same 8,748-row shipped surface, −$16.89B probe — the undercount rides into O unchanged; this fix lands at the next release build. Carrier units reach 88.7% of the >29M floor; the named residual is (i) the occasional-overtime universe the one-week snapshot structurally under-counts (unfixable without a week-level donor — SIPP monthly hours is the candidate if it is ever needed), (ii) the claims-side over-inclusion edge, (iii) TY2024-vs-TY2025 growth.Treasury count-anchor mintability (adjudicated NOT mintable — re-verified, sharper reason)
The prior lane's finding ("floors are not mintable as point facts") re-verified against the schema itself:
consumer_fact.v1hasassertion ∈ {observation, source_projection}and a scalarvalue— no bound semantics. The sharper reason: Treasury never published a point value. sb0517 (June 2026) and the first-year-results release sb0553 both publish only rounded inequalities ("over 29 million filers … average deduction of over $3,100", 75% under $100k). Mintingvalue = 29,000,000would assert publisher precision the publisher never stated. No IRS SOI TY2025 line item exists yet (filing-season weeklies are volume-only;26inweek*.xlsxstill 404 per #499). The mintable object arrives when SOI publishes the TY2025 qualified-overtime deduction line (exact returns + amount) — anobservationat tax_year 2025, aging per the #474 wages-chain path already documented in #499. Until then the floors remain reason-level citations in the coverage manifest (shipped by #499, byte-untouched here).Not done here (by design)
Named consumer
The OBBBA paper (PolicyEngine/obbba-paper, registered against populace#506 with tracking issue obbba-paper#1) currently computes the FLSA premium by hand from stored hours/wages because no lineage ever shipped it. This stage's stored, Treasury-floor-validated
fsla_overtime_premiumcolumn — in dense as well as sparse releases — retires that workaround once an evidence-tier dense artifact publishes (#506); the paper's tip-exemption sibling rides #505.Verification
_flsa_policyinstead of skipping without the us extra): two-signal arithmetic (recovered carrier, persistent-priority, absent-reference-week, occasional-leg-retained, exemptions, hourly-above-HCE), non-finite-as-absent hardening at both the estimator and the gate, gate accepts usual-leg carriers / rejects both-signals-≤40 and fabricated premiums riding on infinite hours, stage end-to-end throughwith_us_org_wages_inputswith signal-conflict rows that fail under swapped wiring in either the fresh or the refresh branch, and the stale-populated-surface refresh (QRF poisoned to prove the skip, stale zeros and NaN premiums converge, consistent surfaces pass through unchanged).packages/populace-buildsuite green in this worktree at the pre-rebase base, and green on the rebased head except thetest_release_target_parity.py::TestRegenerationclass, which fails identically on detached clean origin/main in this venv while main CI is green at the same sha (e56b8ad) — the known local-venv-drift class, arriving with Add JCT tax-expenditure target references: ALD components + CDCC (supersedes #514) #519's new references; this PR's CI is the authoritative run for it.uv run ruff check+ruff format --checkclean on the diff.importorskipleaving the integration tests out of base CI, refresh-branch swap blindness — all addressed in commit 3. Delta-2: all five fixes RESOLVED, no new functional defect; two audit items — the branch had fallen 8 commits behind origin/main (now rebased onto e56b8ad, content unchanged) and the NaN test did not truly pin the fillna regression (a third pass now uses an otherwise-consistent surface with a NaN on a recomputed-zero row, mutation-verified: reintroducing the coercion fails it) — closed in commit 4. Delta-3 final confirmation: MERGE-SAFE — both items RESOLVED (range-diff=on all three replayed commits, byte-identical blobs; the NaN guard verified to fail if the coercion is reintroduced), zero new findings, zero calibration machinery on the exactorigin/main..HEADdiff.Part of populace#451 (item 4 overtime-incidence lane; claim + receipts on the thread).
🤖 Generated with Claude Code