Skip to content

LazyExpr getitem with a broadcasting operand and a non-leading integer index gives the wrong shape #688

Description

@FrancescAlted

Slicing a LazyExpr with an integer in any position other than the first
returns the wrong shape — or raises — whenever one of the operands broadcasts.
The result is silently wrong, not an error, in the first case below.

Reproducer (blosc2 4.10.1.dev0, NumPy 2.5.1):

import numpy as np, blosc2

a_np = np.arange(24, dtype="f8").reshape(2, 3, 4)
b_np = np.arange(2, dtype="f8").reshape(2, 1, 1) + 100
a, b = blosc2.asarray(a_np), blosc2.asarray(b_np)

expr = blosc2.lazyexpr("a + b", {"a": a, "b": b})

print(expr[:, 0].shape)        # (2, 2, 4)  -- NumPy gives (2, 4)
print(expr[..., 0].shape)      # (2, 2, 3)  -- NumPy gives (2, 3)
print((a_np + b_np)[:, 0].shape)   # (2, 4)

# with a lower-rank operand it raises instead
b_np = np.arange(3, dtype="f8").reshape(3, 1) + 100
expr = blosc2.lazyexpr("a + b", {"a": a, "b": blosc2.asarray(b_np)})
expr[..., 0]
# ValueError: operands could not be broadcast together with shapes (2,3) (3,1)

The values are wrong too, not just the shape — the extra axis is not a
harmless dummy dimension.

Scope:

  • Needs both a broadcasting operand and an integer index that is not the
    leading one. expr[0] is fine, and so is any index when all operands have
    the full shape.
  • Only the numexpr path. The same expression as a LazyUDF returns (2, 4)
    and (2, 3) correctly.
  • Not a __getitem__-only accounting problem: expr.compute()[:][:, 0] gives
    the correct (2, 4), so the sliced-evaluation path itself produces the bad
    array.

Probable area: slices_eval_getitem in src/blosc2/lazyexpr.py slices
broadcasting operands through compute_smaller_slice using the raw index,
while the result shape is derived from _slice_bcast (integers widened to
slice(i, i+1)). For a full-shape operand these agree; for a broadcasting one
they do not.

Found while fixing #403. The reshape added there deliberately does not paper
over this: the element counts do not match, so it declines to touch the result
rather than hide a wrong one.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions