Skip to content

inequality's data.ipynb raises under the pinned pandas 3, and nothing runs it to notice #835

Description

@mmcky

lectures/_static/lecture_specific/inequality/data.ipynb cannot run under this repo's own pinned environment. It is linked from inequality.md:618 as "This notebook can be used to compute this information over the full dataset", and it is served — so a reader who follows that link and runs it gets an exception.

The defect

Line 88 is y = np.asarray(y), and line 90 is rd.shuffle(y). Under pandas 3's copy-on-write default, np.asarray() of a Series returns a read-only view, so the shuffle raises. Reproduced against anaconda=2026.07 (pandas 3.0.5, numpy 2.5.2):

np.asarray(series).flags.writeable        = False
rd.shuffle(...)  ->  ValueError: assignment destination is read-only

np.asarray(series).copy().flags.writeable = True
rd.shuffle(...)  ->  OK

The same bug was already fixed in the lecture, ten weeks ago

inequality.md:288 reads y = np.asarray(y).copy() today — fixed by #776 on 2026-06-26 ("pandas 3.0 returns a read-only array from np.asarray() of a Series (Copy-on-Write), so rd.shuffle(y) raised … during the anaconda=2026.06 build"). The notebook carries the identical idiom and never received the fix, because the two copies of this code are not linked by anything.

Why nothing caught it

lectures/_config.yml:24-25 sets exclude_patterns: ['_static/*'], so the build never executes this notebook. It is a served artifact with zero execution coverage — the failure is only reachable by a reader.

That is the more interesting half of this issue: the lecture and the notebook contain the same algorithm, one is executed on every build and one is executed by nobody, and they have now silently diverged on a real API change. Any future environment bump can do this again.

Suggested fix

One line — y = np.asarray(y).copy() — matching the lecture.

Worth considering alongside it: whether this notebook should stay a hand-maintained twin of inequality.md's code at all, given nothing keeps them in step.

What this does not block

The dataset it builds, usa-gini-nwealth-tincome-lincome.csv, has migrated to QuantEcon/data-lectures, and a frozen copy of this notebook landed there as its provenance record (builders/usa-gini-nwealth-tincome-lincome.ipynb, builder_status: committed-frozen). That copy is deliberately not patched: it is committed verbatim as the record of what produced those bytes, and editing it would destroy the property that makes it worth keeping. The fix belongs here, in the copy that readers actually open.

Note also that the builder is non-deterministic independently of this bug — rd.shuffle is unseeded and qe.gini_coefficient accumulates in a prange loop, so re-running produces different low-order digits and cannot reproduce the committed CSV byte-for-byte. Fixing the ValueError makes it run; it does not make it reproducible. That is recorded in the manifest's integrity.upstream and is a separate question from this one.

Related: #776 (the original fix), #833 (unseeded RNG in heavy_tails and inequality, a neighbouring instance of the same reproducibility theme).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions