Skip to content

fix: accept numpy integers as chunk sizes - #288

Open
d-v-b wants to merge 4 commits into
mainfrom
claude/zarr-python-4255-8cc4db
Open

fix: accept numpy integers as chunk sizes#288
d-v-b wants to merge 4 commits into
mainfrom
claude/zarr-python-4255-8cc4db

Conversation

@d-v-b

@d-v-b d-v-b commented Aug 12, 2026

Copy link
Copy Markdown
Owner

🤖 AI text below 🤖

Fixes zarr-developers#4255.

Problem

Since 3.3.0, a per-dimension chunk size given as a numpy integer raises a TypeError:

zarr.zeros((4, 4), dtype="f4", chunks=(np.int64(2), np.int64(2)))
# TypeError: 'numpy.int64' object is not iterable

Two functions in src/zarr/core/chunk_grids.py disagreed about what counts as an integer:

  • normalize_chunks_nd dispatches the scalar convenience form on isinstance(chunks, numbers.Integral)
  • normalize_chunks_1d narrowed on isinstance(chunks, int)

np.int64 is numbers.Integral but is not an int subclass, so a numpy integer passed the outer dispatch, failed the inner narrowing, and fell into the else branch meant for explicit per-dimension chunk sequences — where list(chunks) was called on a scalar.

This matters because numpy integers arise naturally whenever a chunk shape is computed rather than written as a literal; any numpy reduction or elementwise op yields numpy scalars. The reporter hit it through ultrack, which sizes chunks from the array shape and is currently pinned to zarr<3.3.

Fix

  • Narrow on numbers.Integral and coerce with int(), matching the caller. This is consistent with the rest of the module: guess_chunks already ends with tuple(int(x) for x in chunks), and the sequence branch already accepts Integral elements and coerces them.
  • Move the -1 sentinel check inside that branch. It previously ran on the raw input, so a numpy array chunk specification made chunks == -1 evaluate to an array and raise an ambiguous-truth-value error. Rectilinear specs given as numpy arrays now work.
  • Name the offending value when the spec is neither an integer nor iterable, instead of surfacing an opaque 'float' object is not iterable from list(chunks). The issue called out that the old error named neither chunks nor the bad value.

Verification

All of these previously raised and now work:

chunks=(np.int64(2), np.int64(2))         → (2, 2)
chunks=(1, 3, np.int64(16), np.int64(16)) → (1, 3, 16, 16)
chunks=np.int32(10)                       → (10,)
chunks=(np.int64(-1), 5)                  → (10, 5)
chunks=np.array([5, 5])                   → (5, 5)
shards=(np.int64(32), np.int64(32))       → (32, 32)

Regression cases added to test_normalize_chunks (scalar, per-dimension, mixed python/numpy, -1 sentinel, numpy arrays, rectilinear numpy arrays) and error cases for np.int64(0) and the non-iterable scalar.

One existing assertion in tests/test_api.py asserted the old opaque 'float' object is not iterable message and was updated to the new one.

Full test suite: 6357 passed, 373 skipped, 4 xfailed. mypy and ruff clean.

JOhnsonKC201 and others added 2 commits August 12, 2026 21:29
* fix: allow `require_array` to accept a `ZDType`

AsyncGroup.require_array normalised its dtype with np.dtype(), which
cannot consume a ZDType, so requiring an existing array with one raised
a TypeError. Every sibling creation method already accepts ZDTypeLike.

Widen the annotation and normalise via parse_data_type().to_native_dtype().
parse_data_type(None) resolves to float64 just as np.dtype(None) did, so
the default is unchanged. This leaves numpy.typing unused, so drop it.

* chore: rename changelog fragment to the PR number

* fix: keep the float64 default explicit for mypy

parse_data_type does not accept None, so pass "float64" directly, which
is what np.dtype(None) resolved to before.

* test: parametrize require_array dtype cases over (input, expected) pairs

Covers the `dtype=None` path, which resolves to float64 and was previously
untested, and asserts on the resulting ZDType rather than the native dtype.

---------

Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
`normalize_chunks_nd` dispatches the scalar convenience form on
`numbers.Integral`, but `normalize_chunks_1d` narrowed on `int`. Numpy
integer scalars satisfy the former and not the latter, so a per-dimension
numpy integer passed the outer dispatch and then fell into the branch
meant for explicit per-dimension chunk sequences, where `list(chunks)`
raised `TypeError: 'numpy.int64' object is not iterable`.

Numpy integers arise naturally whenever a chunk shape is computed rather
than written as a literal, since numpy reductions and elementwise ops
yield numpy scalars.

Narrow on `numbers.Integral` and coerce with `int()`, matching the
caller and the sequence branch, which already accepted `Integral`
elements.

Move the `-1` sentinel check inside that branch. It previously ran on the
raw input, so a numpy array chunk specification made `chunks == -1`
return an array and raise an ambiguous-truth-value error; rectilinear
specs given as numpy arrays now work.

A chunk specification that is neither an integer nor iterable now names
the offending value and its type instead of surfacing an opaque
"object is not iterable" from `list(chunks)`.

Fixes zarr-developers#4255

Assisted-by: ClaudeCode:claude-opus-5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Regression in 3.3.0: numpy integer chunk sizes raise TypeError: 'numpy.int64' object is not iterable

2 participants