Skip to content

fix(build): use an sdist allowlist so the zarr sdist stops shipping subpackages - #289

Open
d-v-b wants to merge 8 commits into
mainfrom
claude/zarr-sdist-allowlist-414698
Open

fix(build): use an sdist allowlist so the zarr sdist stops shipping subpackages#289
d-v-b wants to merge 8 commits into
mainfrom
claude/zarr-sdist-allowlist-414698

Conversation

@d-v-b

@d-v-b d-v-b commented Aug 13, 2026

Copy link
Copy Markdown
Owner

🤖 AI text below 🤖

The problem

The root sdist config was a blocklist:

[tool.hatch.build.targets.sdist]
exclude = ["/.github", "/bench", "/docs"]

Naming three directories to drop means shipping everything else that happens to sit in the repository root. Top level of zarr-3.3.1.dev23.tar.gz as built from main:

.claude/  .git-blame-ignore-revs  .gitattributes  .markdownlint-cli2.jsonc
.pre-commit-config.yaml  .pyup.yml  .readthedocs.yaml  FUNDING.yml  TEAM.md
changes/  ci/  codecov.yml  design/  lychee.toml  packages/  uv.lock  ...

packages/ is the headline: 2.9M of zarr-indexing, zarr-metadata and zarr-http-server sources — three distributions with their own PyPI releases and their own release workflows — inside every zarr sdist.

The fix

An explicit allowlist, matching what packages/zarr-indexing and packages/zarr-metadata already do.

Sdist goes 1.4M → 763K, and the top level is now exactly:

LICENSE.txt  PKG-INFO  README.md  docs/  examples/  mkdocs.yml
mkdocs_hooks.py  pyproject.toml  src/  tests/  .gitignore

(pyproject.toml, README.md, LICENSE.txt, PKG-INFO and .gitignore are added by hatchling itself.)

/docs is now included, which fixes a second bug

tests/test_docs.py walks docs/ at import time and testpaths collects docs/user-guide. With /docs excluded, the shipped test suite died at collection:

tests/test_docs.py:64: in _session_params
    for example in find_examples(str(root)):
E   ValueError: Not a file or directory: '.../zarr-3.3.1.dev23/docs'
!!!!! Interrupted: 2 errors during collection !!!!!

mkdocs.yml and mkdocs_hooks.py were also already shipping, orphaned, with no docs/ for them to build. examples/ earns its place because the pages under docs/user-guide/examples/ pull their source out of it via pymdownx snippet includes.

Verification

Built the sdist, unpacked it, and ran the suite from the unpacked tree:

check before after
pytest tests/test_docs.py 2 collection errors 61 passed, 2 skipped
pytest --collect-only (full) — 7581 tests, no errors
sdist size 1.4M 763K
packages/ in sdist yes (2.9M) no

Wheel builds unchanged.

Note, not fixed here

testpaths = ["src", "tests", "docs/user-guide"] is fine — docs/user-guide does exist. No stale paths found.

A companion PR does the same for packages/zarr-http-server, which has no sdist section at all and so defaults to "everything not gitignored".

JOhnsonKC201 and others added 4 commits August 12, 2026 21:29
@JOhnsonKC201 @d-v-b
fix: allow require_array to accept a ZDType (zarr-developers#4189)
6ba6e89 
* fix: allow `require_array` to accept a `ZDType`

AsyncGroup.require_array normalised its dtype with np.dtype(), which
cannot consume a ZDType, so requiring an existing array with one raised
a TypeError. Every sibling creation method already accepts ZDTypeLike.

Widen the annotation and normalise via parse_data_type().to_native_dtype().
parse_data_type(None) resolves to float64 just as np.dtype(None) did, so
the default is unchanged. This leaves numpy.typing unused, so drop it.

* chore: rename changelog fragment to the PR number

* fix: keep the float64 default explicit for mypy

parse_data_type does not accept None, so pass "float64" directly, which
is what np.dtype(None) resolved to before.

* test: parametrize require_array dtype cases over (input, expected) pairs

Covers the `dtype=None` path, which resolves to float64 and was previously
untested, and asserts on the resulting ZDType rather than the native dtype.

---------

Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
@d-v-b
fix: accept numpy integers as chunk sizes (zarr-developers#4257)
d3dc9f5 
* fix: accept numpy integers as chunk sizes

`normalize_chunks_nd` dispatches the scalar convenience form on
`numbers.Integral`, but `normalize_chunks_1d` narrowed on `int`. Numpy
integer scalars satisfy the former and not the latter, so a per-dimension
numpy integer passed the outer dispatch and then fell into the branch
meant for explicit per-dimension chunk sequences, where `list(chunks)`
raised `TypeError: 'numpy.int64' object is not iterable`.

Numpy integers arise naturally whenever a chunk shape is computed rather
than written as a literal, since numpy reductions and elementwise ops
yield numpy scalars.

Narrow on `numbers.Integral` and coerce with `int()`, matching the
caller and the sequence branch, which already accepted `Integral`
elements.

Move the `-1` sentinel check inside that branch. It previously ran on the
raw input, so a numpy array chunk specification made `chunks == -1`
return an array and raise an ambiguous-truth-value error; rectilinear
specs given as numpy arrays now work.

A chunk specification that is neither an integer nor iterable now names
the offending value and its type instead of surfacing an opaque
"object is not iterable" from `list(chunks)`.

Fixes zarr-developers#4255

Assisted-by: ClaudeCode:claude-opus-5

* Rename 4255.bugfix.md to 4257.bugfix.md
@dependabot
chore(deps): bump the actions group with 4 updates (zarr-developers#4258
ecb5814 
)

Bumps the actions group with 4 updates: [CodSpeedHQ/action](https://github.com/codspeedhq/action), [scientific-python/issue-from-pytest-log-action](https://github.com/scientific-python/issue-from-pytest-log-action), [actions/attest](https://github.com/actions/attest) and [zizmorcore/zizmor-action](https://github.com/zizmorcore/zizmor-action).


Updates `CodSpeedHQ/action` from 5.0.1 to 5.0.2
- [Release notes](https://github.com/codspeedhq/action/releases)
- [Changelog](https://github.com/CodSpeedHQ/action/blob/main/CHANGELOG.md)
- [Commits](CodSpeedHQ/action@8847237...0ca9cbb)

Updates `scientific-python/issue-from-pytest-log-action` from 1.6.1 to 1.6.2
- [Release notes](https://github.com/scientific-python/issue-from-pytest-log-action/releases)
- [Commits](scientific-python/issue-from-pytest-log-action@054799b...35b4e0a)

Updates `actions/attest` from 4.2.1 to 4.2.2
- [Release notes](https://github.com/actions/attest/releases)
- [Changelog](https://github.com/actions/attest/blob/main/RELEASE.md)
- [Commits](actions/attest@508db95...1e69f48)

Updates `zizmorcore/zizmor-action` from 0.6.1 to 0.6.2
- [Release notes](https://github.com/zizmorcore/zizmor-action/releases)
- [Commits](zizmorcore/zizmor-action@6fc4b00...3dc1ecc)

---
updated-dependencies:
- dependency-name: CodSpeedHQ/action
  dependency-version: 5.0.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
- dependency-name: scientific-python/issue-from-pytest-log-action
  dependency-version: 1.6.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
- dependency-name: actions/attest
  dependency-version: 4.2.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
- dependency-name: zizmorcore/zizmor-action
  dependency-version: 0.6.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@dependabot @d-v-b
chore(deps): bump the python-dependencies group with 6 updates (zarr-…
52a6380 
…developers#4259)

Bumps the python-dependencies group with 6 updates:

| Package | From | To |
| --- | --- | --- |
| [packaging](https://github.com/pypa/packaging) | `26.2` | `26.3` |
| [typer](https://github.com/fastapi/typer) | `0.27.0` | `0.27.1` |
| [coverage](https://github.com/coveragepy/coveragepy) | `7.15.2` | `7.15.3` |
| [hypothesis](https://github.com/HypothesisWorks/hypothesis) | `6.164.0` | `6.165.2` |
| [uv](https://github.com/astral-sh/uv) | `0.12.0` | `0.12.2` |
| [ruff](https://github.com/astral-sh/ruff) | `0.16.0` | `0.16.1` |


Updates `packaging` from 26.2 to 26.3
- [Release notes](https://github.com/pypa/packaging/releases)
- [Changelog](https://github.com/pypa/packaging/blob/main/CHANGELOG.rst)
- [Commits](pypa/packaging@26.2...26.3)

Updates `typer` from 0.27.0 to 0.27.1
- [Release notes](https://github.com/fastapi/typer/releases)
- [Changelog](https://github.com/fastapi/typer/blob/master/docs/release-notes.md)
- [Commits](fastapi/typer@0.27.0...0.27.1)

Updates `coverage` from 7.15.2 to 7.15.3
- [Release notes](https://github.com/coveragepy/coveragepy/releases)
- [Changelog](https://github.com/coveragepy/coveragepy/blob/main/CHANGES.rst)
- [Commits](coveragepy/coveragepy@7.15.2...7.15.3)

Updates `hypothesis` from 6.164.0 to 6.165.2
- [Release notes](https://github.com/HypothesisWorks/hypothesis/releases)
- [Commits](HypothesisWorks/hypothesis@v6.164.0...v6.165.2)

Updates `uv` from 0.12.0 to 0.12.2
- [Release notes](https://github.com/astral-sh/uv/releases)
- [Changelog](https://github.com/astral-sh/uv/blob/main/CHANGELOG.md)
- [Commits](astral-sh/uv@0.12.0...0.12.2)

Updates `ruff` from 0.16.0 to 0.16.1
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](astral-sh/ruff@0.16.0...0.16.1)

---
updated-dependencies:
- dependency-name: packaging
  dependency-version: '26.3'
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: python-dependencies
- dependency-name: typer
  dependency-version: 0.27.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: python-dependencies
- dependency-name: coverage
  dependency-version: 7.15.3
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: python-dependencies
- dependency-name: hypothesis
  dependency-version: 6.165.2
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: python-dependencies
- dependency-name: uv
  dependency-version: 0.12.2
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: python-dependencies
- dependency-name: ruff
  dependency-version: 0.16.1
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: python-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
This was referenced Aug 13, 2026
Open
Merged
@d-v-b
fix(build): use an sdist allowlist so the zarr sdist stops shipping s…
8881b25 
…ubpackages

The root sdist config was a blocklist naming /.github, /bench and /docs, so
every release shipped whatever else happened to sit in the repository root.
That included the whole packages/ tree — the zarr-indexing, zarr-metadata and
zarr-http-server sources, which are released as their own distributions — plus
ci/, design/, towncrier fragments and other repo furniture. 2.9M of the 1.4M
sdist was other people's packages.

Replace it with an explicit allowlist, matching what packages/zarr-indexing and
packages/zarr-metadata already do.

Including /docs also fixes a second problem: tests/test_docs.py walks docs/ and
testpaths collects docs/user-guide, so with docs/ excluded the shipped test
suite died at collection with 'Not a file or directory'. tests/test_docs.py now
runs green from an unpacked sdist (61 passed, 2 skipped), and full collection
finds 7581 tests with no errors.

Assisted-by: ClaudeCode:claude-opus-5
@d-v-b
d-v-b force-pushed the claude/zarr-sdist-allowlist-414698 branch from 58cfed9 to 8881b25 Compare August 13, 2026 11:08
@d-v-b d-v-b mentioned this pull request Aug 13, 2026
Open
d-v-b added 3 commits August 13, 2026 13:20
@d-v-b
fix(build): add an sdist allowlist to zarr-http-server (zarr-develope…
4aba691 
…rs#4262)

packages/zarr-http-server had no [tool.hatch.build.targets.sdist] section at
all, so hatchling defaulted to 'everything not gitignored' — a blocklist by
another name, and the same shape of problem the root pyproject.toml had.

Add an explicit allowlist matching the ones packages/zarr-indexing and
packages/zarr-metadata already carry. This drops changes/ (towncrier fragments
are consumed into CHANGELOG.md at release time), .readthedocs.yaml (only means
anything in the repository) and uv.lock, and keeps /examples, which the suite
genuinely needs: tests/test_examples.py runs examples/serve.py and executes
every cell of examples/serve_notebook.ipynb.

Verified from an unpacked sdist: 210 passed, tests/test_examples.py green.

Assisted-by: ClaudeCode:claude-opus-5
@d-v-b
Merge branch 'main' into claude/zarr-sdist-allowlist-414698
1d43a97
@d-v-b
docs(build): trim the sdist allowlist comment
0573aae 
Drop the narration of the blocklist this replaced -- that history lives in
git -- and keep only the durable rationale and the reason each entry is on
the list.

Assisted-by: ClaudeCode:claude-opus-5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@d-v-b @JOhnsonKC201