Add blog post announcing Enhanced CPS full launch#2708
Merged
Conversation
This post announces the graduation from beta to full launch of the Enhanced CPS, highlighting new features including: - Tip income imputation from SIPP - Overtime premium calculations - Auto loan interest imputation from SCF - Immigration status imputation using ASEC Undocumented Algorithm - Adoption of new microimpute package Also covers upcoming developments: - State and congressional district calibration (Arnold Ventures funded) - Migration to microcalibrate package - Additional data integration plans 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
- Correct calibration targets from "over 100" to "thousands" (9,168 lines) - Fix microimpute description - it automates imputation, not replaces gradient descent - Update data download info to mention Hugging Face automatic downloads - Add links to Arnold Ventures and Nuffield Foundation funding - Replace broken validation dashboard link with GitHub source code link - Make Enhanced CPS the default dataset in the app (remove beta label) - Update switch logic to default to Enhanced CPS when no dataset specified 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Change from "thousands" to the exact count of 9,168 administrative totals 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Update launch date to January 8, 2025 - Remove timeline references that could make content seem dated - Add comprehensive links to PolicyEngine US data documentation - Clarify that SCF and ACS are already integrated for specific uses - Remove dataset selector from UI as Enhanced CPS is now the default for nationwide analysis - Note that Enhanced CPS will become default for state analysis once local calibration is complete 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Add two-stage methodology description with QRF imputation and PyTorch reweighting - Include L0 penalty for sparsity in reweighting description - Add link to methodology flowchart on documentation site - Expand technical details about optimization approach - Include data sources for calibration targets (IRS SOI, Census, CBO/Treasury, JCT) 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Add detailed description of the data processing pipeline - Describe integration of five source datasets (CPS ASEC, IRS PUF, SIPP, SCF, ACS) - Explain the two Enhanced CPS variants created through QRF imputation - Link to full methodology documentation for visualization 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Embed the Enhanced CPS methodology flowchart showing the two-stage process - Flowchart illustrates data integration from five sources through QRF imputation and reweighting 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR announces the full launch of PolicyEngine's Enhanced Current Population Survey (Enhanced CPS), marking its transition from beta to the default dataset for US policy simulations.
- Adds a comprehensive blog post detailing new data enhancements including tip income, overtime premiums, auto loan interest, and immigration status imputations
- Removes the Enhanced CPS dataset selector from the UI since it's now the default dataset
- Documents upcoming developments including state/district calibration and new technical packages
Reviewed Changes
Copilot reviewed 3 out of 5 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
src/posts/posts.json |
Adds metadata entry for the Enhanced CPS launch blog post |
src/posts/articles/enhanced-cps-launch.md |
Creates comprehensive blog post announcing Enhanced CPS features and methodology |
src/pages/policy/PolicyRightSidebar.jsx |
Removes DatasetSelector component and related UI elements since Enhanced CPS is now default |
|
|
||
| **Two-stage methodology**: Our approach combines sophisticated imputation with advanced reweighting techniques. First, we use Quantile Regression Forests (QRF) to impute missing variables from multiple data sources, preserving realistic variation and capturing conditional distribution tails. Second, we apply gradient-based optimization with PyTorch to reweight households, matching administrative targets while maintaining the survey's statistical properties. | ||
|
|
||
|  |
There was a problem hiding this comment.
The referenced image /images/posts/enhanced-cps-launch-flowchart.png is not included in this PR. Ensure the image file exists in the correct location before publishing the blog post.
| "date": "2025-08-08", | ||
| "tags": ["us", "data", "featured"], | ||
| "filename": "enhanced-cps-launch.md", | ||
| "image": "enhanced-cps-launch.png", |
There was a problem hiding this comment.
The referenced image enhanced-cps-launch.png is not included in this PR. Verify the image file exists in the expected location for the blog post metadata.
- Update PolicyRightSidebar tests to expect the enhanced_cps_switch element to not be present - Remove tests that were checking for switch behavior since the selector has been removed 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Collaborator
|
Just to note: we don't default to using the ECPS on API runs at the moment. We never migrated to that as it was not necessary to get the API running. Also, I believe we said it isn't an issue, but does it matter that if a user clicks an old article link that was run off the standard CPS, they'd now be getting ECPS numbers? |
Contributor
Author
|
That's OK nobody but us is using the microsim API. Old posts are outdated anyway but good flag. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Announces the full launch of the Enhanced Current Population Survey (Enhanced CPS), marking its graduation from beta status to PolicyEngine's default dataset.
Key features highlighted
New data enhancements
Technical improvements
microimputepackage for transparent imputation methodologyUpcoming developments
Impact
This launch enables more accurate modeling of recent policy proposals (tips/overtime exemptions) and upcoming local-area policy analysis capabilities.
🤖 Generated with Claude Code