[Data]Improve hash partition for columns contains Struct/list/map

[laysfire]

Description

To improve the performance of hash partitioning, operations that retrieve table columns need to be moved out of the loop.
Use the following script to verify:

import time
import pyarrow as pa
from ray.data._internal.arrow_ops.transform_pyarrow import hash_partition

idx = list(range(50000000))
ints = list(range(50000000))
t = pa.Table.from_pydict(
   {
       "idx": pa.array(idx),
       "ints": pa.array(ints),
   }
)

start = time.time()
hash_partition(t, hash_cols=["idx", "ints"], num_partitions=10)
end = time.time()
print(end - start)

The test result is:

CPU spec	Code Version	Time consumed
Apple M4	original	60s
Apple M4	optimized	15s

[gemini-code-assist]

Code Review

This pull request refactors the hash partitioning logic in transform_pyarrow.py by replacing manual column indexing with a zip-based iteration. Feedback suggests using enumerate for a more idiomatic Python implementation and highlights a potential behavioral discrepancy when partitioning a table with no columns.

[gemini-code-assist]

Using enumerate is more idiomatic in Python than manually managing an index variable.

Also, note that zip(*table.columns) will result in an empty iterator if the table has no columns, causing the loop to be skipped entirely. In this case, partitions will remain all zeros. This differs from the previous implementation which would fill the array with hash(()) % num_partitions. While this edge case (partitioning on zero columns) is rare, you might want to handle it explicitly if exact backward compatibility is required.

Suggested change

	i = 0
	for _tuple in zip(*table.columns):
	partitions[i] = hash(_tuple) % num_partitions
	i += 1
	for i, _tuple in enumerate(zip(*table.columns)):
	partitions[i] = hash(_tuple) % num_partitions

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Reviewed by Cursor Bugbot for commit d1c5c4d. Configure here.}

[laysfire]

Since this pr #62757 propose a better solution, close this pr.

[owenowenisme]

Sorry I didn't see your PR, thanks for your contribution anyway! If you need other stuff to work on you can DM me on ray slack.