Harden CPS basic ORG loading and caching

[MaxGhenis]

Summary

• fetch CPS basic ORG monthly files via explicit HTTP requests and parse from bytes before selecting columns
• normalize column resolution case-insensitively and add retry coverage for transient bad responses
• serialize ORG cache creation with a file lock and atomic temp-file replace so concurrent builds share one cache artifact

Testing

• UV_PYTHON=3.14 uv run pytest tests/unit/datasets/test_org.py -q
• UV_PYTHON=3.14 uv run ruff check policyengine_us_data/datasets/org/org.py tests/unit/datasets/test_org.py
• python3 -m py_compile policyengine_us_data/datasets/org/org.py tests/unit/datasets/test_org.py
• UV_PYTHON=3.14 uv run python - <<'PY'
  from policyengine_us_data.datasets.org.org import _load_cps_basic_org_month
  month = _load_cps_basic_org_month(2024, "may", max_attempts=1)
  print(month.shape)
  print(month.columns.tolist())
  PY

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
pipeline-diagrams	Error		Apr 12, 2026 3:02pm