[Tom] July 31 edits of some old and new lectures

[mmcky]

This PR are edits to old and new lectures in support of work that Tom is currently doing.

[mmcky]

@thomassargent30 just running CI now to check this builds all our assets and then will merge once I get green.

[Copilot]

Pull request overview

This PR updates and extends the lecture materials by adding new MyST/Jupytext lectures and refactoring existing ones to improve the coherence of the VAR/Kalman and LQ permanent income “lecture suite” narratives.

Changes:

• Add two new lectures: var_subsets (VARs for subvectors via Kalman filtering) and lq_robust_bewley (robustness + Bewley economy via observational equivalence).
• Refactor and retitle/relabel significant parts of lq_robust_smoothing and kalman_filter_var to align with the expanded lecture sequence and new follow-on lectures.
• Update the book TOC and bibliography to include new lectures and citations; minor plotting API update in ar1_turningpts.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
lectures/var_subsets.md	New lecture introducing VAR subsystem theory + executable code (includes a blocking syntax error in one f-string).
lectures/lq_robust_smoothing.md	Major restructure/rewrite to serve as the 3rd lecture in a 4-part LQ permanent income sequence, adding scalar closed-form results and reorganizing content.
lectures/lq_robust_bewley.md	New lecture extending robust smoothing results to a Bewley setting with heterogeneous robustness types.
lectures/lq_permanent_income.md	Update lecture-sequence references to reflect 4-part suite and link to the new lq_robust_bewley.
lectures/lq_bewley_complete_markets.md	Update lecture-sequence references to reflect 4-part suite and link to the new lq_robust_bewley.
lectures/kalman_filter_var.md	Add header/contents, tighten definitions, update a worked example section to point to the new var_subsets lecture, and adjust code ordering/comments.
lectures/ar1_turningpts.md	Replace az.plot_trace_dist with az.plot_trace for ArviZ plotting.
lectures/_toc.yml	Add var_subsets and lq_robust_bewley to the Jupyter Book TOC; also add sargent_surico.
lectures/_static/quant-econ.bib	Add bibliography entries (e.g., Sargent–Surico 2011, Lucas 1980, etc.).

[mmcky]

Build failure in sargent_surico.md — arviz 1.x API break

The Build HTML failure in run 30674947077 comes from a single lecture, lectures/sargent_surico.md. Everything else in the PR built and executed fine — sargent_surico.err.log is the only report in the execution-reports-html artifact.

Cause

The lecture installs arviz unpinned (!pip install pandas_datareader numpyro jax arviz), and pip install arviz now resolves arviz 1.2.0 — the rewrite that dispatches to arviz-base / arviz-stats / arviz-plots. Two calls in the lecture use the arviz 0.x API.

1. az.from_dict (line 1287) — this is what failed CI. In arviz 0.x the first positional argument was the posterior group, so a flat {var_name: array} mapping worked. In arviz 1.x the first argument is {group_name: {var_name: array}}, so each of the nine entries in FREE is read as a group and dict_to_dataset() is handed a raw ndarray:

AttributeError: 'numpy.ndarray' object has no attribute 'items'
  .../site-packages/arviz_base/io_dict.py:105 in from_dict

2. hdi_3% / hdi_97% (line 1643) — a second break sitting behind the first. Execution stopped at the cell above, so this one never got the chance to run. arviz 1.x summary() no longer emits hdi_3%/hdi_97%; it defaults to an 89% ETI, with columns mean, sd, eti89_lb, eti89_ub, ess_bulk, ess_tail, r_hat, mcse_mean, mcse_sd. As written it raises KeyError: "['hdi_3%', 'hdi_97%'] not in index".

Fix

Wrap the draws in a posterior group, and ask summary explicitly for the 94% HDI that the arviz 0.x default used to give:

rwmh_idata = az.from_dict(
    {'posterior': {n: kept[:, j][None, :] for j, n in enumerate(FREE)}})

nuts_summary = az.summary(nuts_idata, var_names=FREE,
                          ci_kind='hdi', ci_prob=0.94)
nuts_kept = np.column_stack([np.asarray(mcmc.get_samples()[n]) for n in FREE])
nuts_summary[['mean', 'sd', 'hdi94_lb', 'hdi94_ub', 'ess_bulk', 'r_hat']]

No other arviz calls need changing — az.from_numpyro and az.summary(var_names=...) both still exist in 1.2.0. I reproduced the original AttributeError against arviz 1.2.0 and ran both patched cells plus every downstream consumer of the two summary frames (the worst/nsmallest diagnostics, the compare table, the median-ESS report, and the ESS bar chart) — all clean. ess_bulk comes back as pandas Int64, so the argsort, median and f-string formatting downstream are unaffected.

One unrelated thing worth a second look

The edit at lectures/ar1_turningpts.md:508 swaps az.plot_trace_dist → az.plot_trace. It passes CI, but in arviz 1.x these are two different plots: plot_trace_dist is "1D marginal distributions and iteration versus sampled values" (the equivalent of arviz 0.x's plot_trace), while plot_trace is "iteration versus sampled values" only. The change silently drops the density panels from that figure, so it's worth confirming it was intended rather than a 0.x-from-memory correction.

[mmcky]

Fix pushed — d169164

I've pushed the two arviz 1.x fixes to robust_tom, so CI should re-run on this PR.

location	before (arviz 0.x)	after (arviz 1.x)
sargent_surico.md line 1287	az.from_dict({n: kept[:, j][None, :] ...})	the same mapping wrapped in a 'posterior' group
sargent_surico.md line 1641	az.summary(nuts_idata, var_names=FREE)	plus ci_kind='hdi', ci_prob=0.94
sargent_surico.md line 1644	nuts_summary[[..., 'hdi_3%', 'hdi_97%', ...]]	nuts_summary[[..., 'hdi94_lb', 'hdi94_ub', ...]]

The first one is what failed the Build HTML step. The second and third were sitting behind it — execution stopped at the from_dict cell, so the hdi_3% columns never got the chance to raise KeyError, and would have failed the next run instead. The ci_kind/ci_prob arguments are there to keep the 94% highest-density interval that arviz 0.x gave by default; without them 1.x returns an 89% ETI in columns named eti89_lb/eti89_ub.

Five lines changed in total, no prose edits, and no other arviz calls needed touching — az.from_numpyro and az.summary(var_names=...) both still exist in 1.2.0.

How it was checked

I could not run the lecture end to end locally, since it needs a live FRED pull plus jax and a GPU, so CI on this PR is the real test. What I did verify, against a scratch install of arviz 1.2.0: the original AttributeError reproduces exactly as CI reported it; both patched cells run clean; and every downstream consumer of the two summary frames still works — the worst/nsmallest mixing diagnostics, the compare table with its ESS ratio column, the median-ESS-per-second report, and the ESS bar chart. summary() returns ess_bulk as a pandas Int64 column in 1.x, so the np.argsort, .median() and f-string formatting downstream are all unaffected.

One thing to be aware of for future edits: 1.x summary() pre-formats mean, sd and r_hat into object-dtype columns, because round_to defaults to 'auto'. The compare.round(3) call therefore no-ops on those three columns and rounds only ESS ratio. The displayed table looks right — arviz has already rounded the values sensibly — so I've left it alone, but pass round_to=None if you ever want real floats there.

Follow-up filed

The unrelated az.plot_trace_dist → az.plot_trace change in ar1_turningpts.md is now tracked in #1021. Short version: under arviz 1.x those are two different plots, plot_trace_dist is the one that matches old plot_trace, and the edit in this PR reverts the migration done in #584 — the figure loses its density panels without failing the build. ar1_bayes.md has the same issue on main, left over from #930. I have not touched either lecture here; the issue proposes fixing both together.

[mmcky]

@thomassargent30 applied a couple of fixes to enable this to work with the latest version of arviz

[github-actions]

📖 Netlify Preview Ready!

Preview URL: https://pr-1020--sunny-cactus-210e3e.netlify.app

Commit: d812aa6

📚 Changed Lectures

• ar1_turningpts
• kalman_filter_var
• lq_bewley_complete_markets
• lq_permanent_income
• lq_robust_bewley
• lq_robust_smoothing
• sargent_surico
• var_subsets

---

Build Info

• Workflow: Build Project [using jupyter-book]

[mmcky]

Green — d812aa6

CI passes now. Two commits were needed, and the second one only became visible once the first was fixed.

commit	problem	fix
d169164	az.from_dict and the hdi_3% columns use the arviz 0.x API; unpinned !pip install arviz now gives 1.2.0	wrap the draws in a 'posterior' group; ci_kind='hdi', ci_prob=0.94 with hdi94_lb/hdi94_ub
d812aa6	the NUTS cell exceeded the 2400s per-cell timeout on the GPU runner, confirmed by CellTimeoutError in the execution report	jax.config.update('jax_platform_name', 'cpu')

The second one is the interesting one. That cell takes 157 seconds locally on a laptop CPU, and more than 2400 seconds on the g4dn.2xlarge T4. Pinning jax to the CPU took the whole lecture from not-finishing to 14m52s on the same runner, and the notebook-execution step from 58m02s to 26m10s.

Why the GPU is so much slower has not been established — jax_enable_x64 on a T4, kernel launch overhead on 10x10 matrices, and chain_method='sequential' are all plausible and none is confirmed. I've opened #1022 to look into it properly, since it likely affects any lecture that pairs jax_enable_x64 with a small model. The pin is the pragmatic fix for this PR either way.

Unrelated and still open: the az.plot_trace_dist → az.plot_trace change in ar1_turningpts.md, tracked in #1021.

[mmcky]

✅ Translation sync completed (zh-cn)

Target repo: QuantEcon/lecture-python.zh-cn
Translation PR: QuantEcon/lecture-python.zh-cn#230
Files synced (9):

• lectures/ar1_turningpts.md
• lectures/kalman_filter_var.md
• lectures/lq_bewley_complete_markets.md
• lectures/lq_permanent_income.md
• lectures/lq_robust_bewley.md
• lectures/lq_robust_smoothing.md
• lectures/sargent_surico.md
• lectures/var_subsets.md
• lectures/_toc.yml