Skip to content

[Data] Fix error message and off-by-one bounds check in operators#64576

Open
jiangxt2 wants to merge 1 commit into
ray-project:masterfrom
jiangxt2:fix/data-operator-bounds-check
Open

[Data] Fix error message and off-by-one bounds check in operators#64576
jiangxt2 wants to merge 1 commit into
ray-project:masterfrom
jiangxt2:fix/data-operator-bounds-check

Conversation

@jiangxt2

@jiangxt2 jiangxt2 commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Why are these changes needed?

Three minor bugs in Ray Data operators:

  1. _concatenate_chunked_arrays prints type: None in its assertion error message instead of the actual array type. The f-string uses {type_} which is always None at that point in the loop — it is initialized to None and only reassigned after the assertion.

  2. UnionOperator._add_input_inner uses <= instead of < in its bounds check, allowing input_index == len(self._input_dependencies). This is semantically incorrect — input_index is 0-based, so valid values are [0, len-1]. The <= allows an out-of-bounds index to pass the assertion.

  3. ZipOperator._add_input_inner has the same off-by-one bounds check.

Both off-by-one bugs are corrected to match the pattern used by MixOperator._add_input_inner (line 95), which correctly uses strict <.

What changes are made?

  • python/ray/data/_internal/arrow_ops/transform_pyarrow.py: Change {type_} to {arr.type} in assertion error message
  • python/ray/data/_internal/execution/operators/union_operator.py: Change <= to < in bounds check
  • python/ray/data/_internal/execution/operators/zip_operator.py: Change <= to < in bounds check
  • python/ray/data/tests/unit/test_fix_operator_bounds_check.py: Add 3 tests covering the fixes

Related issues

N/A

Checks

  • I've signed off every commit (using git commit -s)
  • I've run pre-commit hooks (ruff, black, semgrep, pydoclint) to lint the changes
  • I've added tests for the new behavior

@jiangxt2
jiangxt2 requested a review from a team as a code owner July 7, 2026 16:53

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request addresses three bugs: it corrects an assertion error message in _concatenate_chunked_arrays to display the actual tensor type instead of None, and fixes off-by-one bounds checking errors in both UnionOperator and ZipOperator input addition methods. It also introduces corresponding unit tests. The feedback suggests refactoring the _concatenate_chunked_arrays test to use the more idiomatic pytest.raises context manager instead of a try-except block with pytest.fail.

Comment thread python/ray/data/tests/unit/test_fix_operator_bounds_check.py Outdated
Fix three minor bugs in Ray Data operators:

1. _concatenate_chunked_arrays prints "type: None" instead of the actual
   array type in its assertion error message. Changed {type_} to {arr.type}.
2. UnionOperator._add_input_inner uses <= instead of < in bounds check,
   allowing out-of-bounds index access under python -O.
3. ZipOperator._add_input_inner has the same off-by-one bounds check.

Both off-by-one bugs are corrected to match the pattern used by
MixOperator._add_input_inner.

Signed-off-by: jiangxt2 <jiangxt2@vip.qq.com>
@jiangxt2
jiangxt2 force-pushed the fix/data-operator-bounds-check branch from 13dcbd7 to f227ae9 Compare July 7, 2026 17:02
@ray-gardener ray-gardener Bot added data Ray Data-related issues community-contribution Contributed by the community labels Jul 7, 2026
@github-actions

Copy link
Copy Markdown

This pull request has been automatically marked as stale because it has not had
any activity for 14 days. It will be closed in another 14 days if no further activity occurs.
Thank you for your contributions.

You can always ask for help on our discussion forum or Ray's public slack channel.

If you'd like to keep this open, just leave any comment, and the stale label will be removed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-contribution Contributed by the community data Ray Data-related issues stale The issue is stale. It will be closed within 7 days unless there are further conversation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant