Skip to content

Add dedicated Python docs flow#7

Merged
MaxGhenis merged 19 commits intomainfrom
feature/python-interactive-guide
Apr 21, 2026
Merged

Add dedicated Python docs flow#7
MaxGhenis merged 19 commits intomainfrom
feature/python-interactive-guide

Conversation

@PavelMakarchuk
Copy link
Copy Markdown
Contributor

Summary

  • split the country docs into route-based API and Python tracks with a country landing page
  • replace the mixed Python/API content on the API page with API-specific guidance and direct Python cross-links
  • add a dedicated Python docs page that pulls in the household simulation and microsimulation material from the educational notebooks, including sequential sidebar navigation and step-by-step progression
  • remove the broken relative Next asset prefix so nested routes load CSS and JS correctly in local development

Testing

  • npm run build

Notes

  • the microsimulation examples are currently concrete for the US track because the educational notebook material is US-specific
  • the header remains the primary API/Python switch; the duplicate in-page track switcher was removed

@vercel
Copy link
Copy Markdown

vercel Bot commented Mar 31, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
household-api-docs Ready Ready Preview, Comment Apr 21, 2026 1:25pm

Request Review

- Remove intermediate landing pages (/ and /[countryId]) — redirect to API docs
- Fix sidebar layout: use underscore syntax for Tailwind grid-cols, lower breakpoint to md
- Remove side info boxes, fold content into panel body text
- Add real computed outputs below code blocks in notebook style
- Add inline comments from educational notebooks to all code examples
- Replace summary cards with core concepts intro (entities, variables, parameters, periods)
- Remove redundant Situation shape block; add per-entity comments in quickstart code
- Auto-scroll to content top when switching topics
- Add d3-geo and topojson-client dependencies

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Remove sidebar title and description — numbered steps are self-explanatory
- Remove "You reached the end of this sequence" message
- Remove "Before you start" heading from core concepts section
- Use exact DataFrame output with full precision

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Resolve conflict in next.config.mjs — keep function-based config with assetPrefix from main.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Replace state_name with state_code to match analysis notebooks
- Wrap microsim calculate_dataframe() calls with pd.DataFrame() to
  avoid MicroSeries issues in pandas operations
- Add import pandas as pd to microsim examples that use DataFrames
- All 11 code examples verified to run successfully

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove install/import box and simplify subtitle text. Connect
CountrySelector to HeaderActionButtons so the globe appears in
the header.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Reform.from_dict() returns a class that policyengine-uk's Simulation
cannot instantiate due to a missing baseline arg. Passing a plain dict
to reform= works correctly and is simpler.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds entity hierarchy and parameter-type tables, tracing/programmatic/
structural-reform/visualization sections, and a top-level reproducibility
section (pin the bundle, two-manifest architecture, TRACE export).

Every code cell in household, microsim, and reproducibility sections was
executed locally against policyengine==3.4.4 (US 1.602.0, UK 2.74.0).
Fixed code bugs that the notebooks inherited (ctc_value key missing,
Reform class vs instance, UK scenario __init__ override, identity
map_to_entity, missing state_fips on household), replaced fabricated
manifest APIs with the real get_release_manifest / get_data_release_manifest
pair, and updated every documented output to match live results on the
enhanced CPS 2024 and enhanced FRS 2023-24 datasets.

UK time-series dropped from a 10-year to a 5-year window because the
packaged enhanced FRS is only uprated through 2030, which also matches
the OBR forecast horizon. TRACE export wraps DataReleaseManifestUnavailable
with a runtime-only fallback since data manifests are not yet published
to HF for the certified dataset versions.

Output blocks now render as plain pre with only numeric tokens colored
(warm brown), avoiding syntax-highlighter false positives on lines like
"Weighted total EITC: $6.8B".

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
policyengine.py v4 rewrote the user-facing surface (#298):
- import policyengine as pe; pe.us.calculate_household(...)
- no more USHouseholdInput / UKHouseholdInput wrapper classes
- plain dict reforms for both calculate_household (reform=) and
  Simulation (policy=)
- attribute access on typed results (result.tax_unit.income_tax)
- pe.us.model / pe.uk.model replace us_latest / uk_latest
- Simulation.ensure() replaces manual run()/cache plumbing
- extra_variables={entity: [names]} opts in non-default columns
- provenance moved to policyengine.provenance.manifest and
  policyengine.provenance.trace; serialize_trace_tro replaces
  compute_trace_composition_fingerprint
- bundled manifests now pin policyengine-us==1.653.3 and
  policyengine-uk==2.88.0 under bundle ids us-4.0.0 / uk-4.0.0
- strict import-time certification check

Every code cell in household, microsim, and reproducibility sections
was re-executed against policyengine==4.3.0 (US 1.653.3, UK 2.88.0,
core 3.25.1) on the enhanced CPS 2024 and enhanced FRS 2023-24
datasets. All documented outputs updated to live values.

Cells simplified where v4 lets us: variation grids became a loop
over calculate_household (no custom dataset needed); programmatic
situations are plain pydantic dicts; regional is Aggregate with
filter_variable=/filter_variable_eq=.

Known issues surfaced by this pass:
- US economic_impact_analysis references a non-existent payroll_tax
  variable (upstream bug). Guide uses the manual composition path
  with calculate_decile_impacts + per-program ProgramStatistics
  for programs present in the default output set.
- UK Microsimulation scenario handler calls Reform() with no args
  and apply(tbs), so structural-reform subclasses need __init__
  and apply overrides. Documented in the structural reform example.
- The UK dataset is uprated through 2030, so the UK time-series
  example runs a 5-year OBR-style window rather than 10 years.
- A full 10-year US time-series run is memory-heavy; the output
  block documents the shape rather than exact numbers.
- Data release manifests are not yet published on HuggingFace for
  the certified dataset versions, so get_data_release_manifest
  raises DataReleaseManifestUnavailableError; the trace export
  example handles this with a runtime-only fallback path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Commit 56e7532 replaced the /[countryId] landing page with a client-side
useRouter redirect. The Next.js static export (output: 'export') requires
every dynamic segment to export generateStaticParams, but a 'use client'
file can't. The build failed with:

  Page "/[countryId]" is missing "generateStaticParams()" so it cannot be
  used with "output: export" config.

Restore the page as a server component with generateStaticParams and do the
redirect via a meta refresh tag on the generated HTML, which works under
static export on any host.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Trace tab (household analysis):
- Adds an "anatomy of a trace line" reference block (variable<period, branch>
  = [array], max_depth semantics, default-branch note) so readers can decode
  trace output cold without extra context.
- Output now shows a "..." tail so the truncation is explicit instead of
  looking like the full log.

TRACE export tab (reproducibility):
- Expands from a single code block into four structured parts:
  1. Bundle TRO via Python with DataReleaseManifestUnavailableError fallback
  2. CLI one-liners: release-manifest, trace-tro, trace-tro-validate
  3. Per-simulation TRO via build_simulation_trace_tro that chains a bundle
     TRO to a reform payload and results payload, anchored by bundle_tro_url
- Reference block documents the 4+3 artifact composition and the full pe:*
  namespace (pe:certifiedForModelVersion, pe:compatibilityBasis enum,
  pe:emittedIn "local" vs "github-actions" + pe:ciRunUrl / pe:ciGitSha,
  pe:bundleFingerprint / pe:bundleTroUrl on simulation TROs).
- Every output verified by running the code end-to-end against
  policyengine==4.3.0; fingerprints and validation results are real.

Household analysis:
- Renames "Visualization" to a new household tab "Charting a household" and
  shows a real PNG rendered by running the exact variation + Plotly snippet
  with fig.write_image().
- Adds inline comments to the household impact example explaining people
  dicts, entity kwargs, year, and attribute-access on the typed result.

Microsimulation:
- New "Decile impact chart" tab that renders a real decile-level
  absolute-change bar chart from calculate_decile_impacts().
- Inline comments on the setup tab explain ensure_datasets key format and
  pe.us.model / pe.uk.model pinning, plus the Aggregate construct->run()
  lifecycle on the metrics tab.

Headers and hero:
- Shared SectionHeader eyebrow + H2 + subtitle across the three main
  sections (Simulation / Microsimulation / Reproducibility), replacing the
  ad-hoc reproducibility header.
- Hero copy switched from "Learn policyengine through US examples" to
  "policyengine.py for {US} tax and benefit analysis" (declarative
  reference framing) with a tighter subtitle.

Code output formatting:
- Drops the number-highlighting regex in CodeBlock that partially tinted
  hashes and version strings. Output blocks now render as plain monospace
  on the dark background, matching the convention in Python docs / MDN /
  rustdoc.

Assets:
- public/python-guide/{us,uk}-variation-chart.png - household earnings
  variation, rendered by the exact snippet shown in the guide.
- public/python-guide/{us,uk}-decile-impacts-chart.png - microsim decile
  impacts from calculate_decile_impacts().

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Decile impact (microsim) and household variation charts now use the
@policyengine/design-system tokens directly instead of ad-hoc colors:

- primary teal #319795 for the main series / positive bars
- gray #4B5563 for the secondary series / negative bars / zero line
- Inter font stack with explicit sizes (16 title / 13 axis labels /
  12 ticks + legend + hover) to match Recharts defaults in app-v2
- dashed horizontal gridlines only, no vertical, no zeroline
- white plot + paper background, ink color #101828 for text
- centered title; decile title uses the app-v2 dynamic phrasing
  "This reform would increase/decrease the net income of households
  by $X on average"
- value labels outside each decile bar, cliponaxis=False

Removed the "PolicyEngine sorts households..." annotation that was
getting cropped on the decile chart. Regenerated all four PNGs
(public/python-guide/{us,uk}-{variation,decile-impacts}-chart.png)
by running the exact snippets shown in the guide.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Plotly horizontal legend was placing the EITC / UC entry right against
the end of the "Household net income" label because entries default to
minimum width. Set entrywidth=180 + entrywidthmode="pixels" so each
legend entry reserves a full 180px for its glyph + label, giving clear
separation between items.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pairs with PolicyEngine/policyengine.py#314 which migrates the
emitted TROv vocabulary URI from the prerelease 2023/05 namespace
to the canonical 0.1 namespace. Landing both together keeps the
docs consistent with what policyengine.py actually emits.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@MaxGhenis MaxGhenis marked this pull request as ready for review April 21, 2026 13:26
@MaxGhenis MaxGhenis merged commit 19d750c into main Apr 21, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants