Switch to canonical SPM variable names; populate 2026 baseline#1
Closed
PavelMakarchuk wants to merge 1 commit into
Closed
Switch to canonical SPM variable names; populate 2026 baseline#1PavelMakarchuk wants to merge 1 commit into
PavelMakarchuk wants to merge 1 commit into
Conversation
Methodology: - Use spm_unit_is_in_spm_poverty and spm_unit_is_in_deep_spm_poverty mapped to person, matching policyengine.py outputs/poverty.py conventions (USPovertyType.SPM / SPM_DEEP). - Formulas are mathematically identical to v1 API's person_in_poverty / in_deep_poverty; this only changes the variable name we reference. Local compute: - New scripts/compute_local.py runs the same methodology locally without Modal, writes data/baseline.json and frontend/public/baseline.json. - _resolve_dataset falls back to the documented HuggingFace URL convention when policyengine.countries import fails (e.g. editable installs with a mismatched manifest). Frontend: - Pin Next.js to 15.5.4 (16 + Turbopack hits an RSC bundler bug). - Replace turbopack.root with outputFileTracingRoot. Baseline (2026, current-law, no reforms): - All 52 regions populated, zero errors. - Computed with policyengine 4.3.0 / policyengine-us 1.690.7 / policyengine-us-data 1.17.0. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
4 tasks
Contributor
Author
|
Superseded by #2, which contains these commits plus the deploy and senior-poverty work. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
policyengine.pyvariable names:spm_unit_is_in_spm_povertyandspm_unit_is_in_deep_spm_poverty, both mapped to person entity. Formulas are mathematically identical to v1 API'sperson_in_poverty/in_deep_poverty, but the names matchpolicyengine/outputs/poverty.py(USPovertyType.SPM/SPM_DEEP). Worth noting: the v1 API referencesperson_in_deep_poverty, which no longer exists inpolicyengine-us1.690.7.scripts/compute_local.pyso we can populatedata/baseline.jsonwithout Modal. Falls back to the documented HuggingFace dataset URL convention whenpolicyengine.countriesfails to import (editable installs with mismatched manifests).baseline.jsonfor 2026 (current-law, no reforms): all 52 regions, zero errors.Baseline highlights (2026, SPM, weighted by
person_weight)Computed with
policyengine4.3.0 /policyengine-us1.690.7 /policyengine-us-data1.17.0.Observations worth digging into separately
enhanced_cps_2024.h5and the per-statestates/{XX}.h5files appear to be calibrated differently — possibly different SPM thresholds, or the state files use state-specific thresholds while the national file uses a single national one. We're using the same datasets the API would route to for each region, so the gap reflects the data, not the methodology.person_weighttotals look low (e.g. AL ~2.2M vs real ~5.1M). The state-file weight columns may not be calibrated to full state population — also a property of the data, not this code.Test plan
make dev— page renders 52-row table sorted by child poverty descmodal deploy) flags any package that has moved on PyPI🤖 Generated with Claude Code