Add PLAN-QELD-PACKAGE.md: the consumer-side data package, re-scoped as a URL resolver

[mmcky]

Distils a design session into an operative plan for qeld, and lands the call-site audit that drives it. Nothing is implemented — this is the development plan we build the tooling from.

What the package is

A URL resolver, not a fetch-and-cache library. qeld.url('mpd2020.xlsx') returns a context-aware URL and the lecture reads it with pandas, in the open, so the reading step is never obfuscated. Two functions: url() and info().

It exists for three concrete reasons: the same file is read via github.com/…/raw/main/… in lecture-python-intro and raw.githubusercontent.com/… in lecture-wasm because the first fails CORS (that split is repoint rules 5–6); pyodide_http.patch_all() sits in every wasm lecture; and the data.quantecon.org cutover should be one constant rather than an edit in every lecture.

What changed from the first design pass

The version pins the key→URL contract, not bytes. CalVer is retired for semver — >=1,<2 in install cells, == in env files. The qbn==1.6 precedent does not transfer: quantecon_book_networks ships its data inside the wheel, so its pin is immortal because nothing is fetched. qeld inverts that exactly. As a house pattern it also does not generalise — of intro's 16 install cells the only exact pins are the two qbn cells.

No fetch(), load(), cache, or runtime integrity. After the format convention is applied, dataBHS.mat is the only file in the whole endgame that cannot be read from a URL. That is what justifies dropping fetch() rather than growing out of it.

Context detection may change transport, never semantics. Two jobs only: the CORS-correct URL form, and the emscripten shim that retires pyodide_http.

Integrity moves to CI. Measured against the right baseline, this removes nothing — a bare URL read has no integrity guarantee today either. The committed-bytes leg already landed in #56. The remaining gap is a live serving-URL check, which belongs post-merge and on a schedule, because audit-dashboard.yml gates deploy on github.event_name != 'pull_request' so a PR branch's bytes are never on the serving host.

The call-site rule was measured, not asserted

Every static data read in six repos — 115 sites across 40 lectures — was swept and clustered by idiom. The worklist lands as scripts/qeld_callsites.yml.

Three candidate syntaxes were tested against every idiom. Substituting in place of the URL expression the lecture already uses is the only form that wins everywhere: inlining destroys reused variables across 47 sites, and always-two-step turns one line into two across the 18 inline literals. It also gives the property that matters for a ~78-site sweep — the diff is always and only the URL expression, so a reviewer verifies a migration PR by reading the changed lines alone.

The honest case

13 sites are structural, where qeld deletes real logic; 70 are cosmetic shortenings of literals adopted in the recent repoint PRs, which a plain one-line literal would win back without a package. The package earns its place on those 13, the wasm shim, the host cutover, and the relative-path portability bugs that break Colab today — not on the read-site tally. Section 5.2 says so plainly so the plan can be weighed on that basis.

Two latent bugs turned up in passing and are worth fixing regardless of qeld: inequality.md imports pyodide_http and never calls patch_all(), and short_path.md calls requests.get under Pyodide with no shim at all.

Reading order

Section 7 is the development plan (Q1–Q7 with gates). Section 8 carries the ambiguities, each tagged with the phase that needs it — including one the session changed under us: the .npy → CSV conversion was free when analysed and is now a breaking change, because the A3 set was repointed in #49.

Section 9 lists corrections to the original design report so they are not re-proposed.

scripts/qeld_callsites.yml is explicitly a snapshot, not a live view — both this repo and lecture-python-intro moved during the session that produced it, and it says so at the top. Regenerate before the sweep.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR adds the design-and-rollout plan for the proposed qeld consumer-side URL resolver package, along with a snapshot YAML worklist of all audited data read call-sites across lecture repositories to drive the future conversion sweep.

Changes:

• Add PLAN-QELD-PACKAGE.md, documenting the settled design (URL resolver, semver pinning, CI-based integrity) and a phased rollout plan.
• Add scripts/qeld_callsites.yml, a machine-readable snapshot of audited call-sites clustered by idiom and tagged with conversion actions/wins.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
PLAN-QELD-PACKAGE.md	Adds an end-to-end design/rollout plan for qeld, including conventions, CI/integrity stance, and phased work items.
scripts/qeld_callsites.yml	Adds the audited call-site worklist used to scope and order the eventual conversion sweep.

Suppressed comments (3)

scripts/qeld_callsites.yml:525

• This callsite entry duplicates an earlier identical entry for the same repo/lecture/filename (see lines 472–478). Keeping both will inflate any automated counts and can cause the conversion sweep to touch the same site twice. Remove the duplicate entry.

  - repo: "QuantEcon"
    lecture: "lectures/heavy_tails.md:854"
    filename: "cities_us.csv"
    idiom: "C-inline-literal"
    verdict: "simpler"

scripts/qeld_callsites.yml:532

• This callsite entry duplicates an earlier identical entry for the same repo/lecture/filename (see lines 479–485). Keeping both will inflate any automated counts and can cause the conversion sweep to touch the same site twice. Remove the duplicate entry.

  - repo: "QuantEcon"
    lecture: "lectures/heavy_tails.md:855"
    filename: "cities_brazil.csv"
    idiom: "C-inline-literal"
    verdict: "simpler"

scripts/qeld_callsites.yml:539

• This callsite entry duplicates an earlier identical entry for the same repo/lecture/filename (see lines 486–492). Keeping both will inflate any automated counts and can cause the conversion sweep to touch the same site twice. Remove the duplicate entry.

  - repo: "QuantEcon"
    lecture: "lectures/heavy_tails.md:879"
    filename: "forbes-billionaires.csv"
    idiom: "C-inline-literal"
    verdict: "simpler"

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.