Skip to content

Add unit tests for core/architecture/sendsemantics partitioners #4663

@Yicong-Huang

Description

@Yicong-Huang

Task Summary

Backfill unit tests for amber/src/main/python/core/architecture/sendsemantics/. This is the partitioner family that routes tuples between operators (range / hash / round-robin / broadcast / single / one-to-one) and is currently the largest under-tested module in the Python codebase per the latest Codecov report.

Codecov snapshot (commit f59e9e5, 2026-05-02):

file cov hits/lines
range_based_shuffle_partitioner.py 40.8% 20/49
broad_cast_partitioner.py 45.5% 20/44
round_robin_partitioner.py 46.1% 18/39
hash_based_shuffle_partitioner.py 47.5% 19/40
single_blocking_io.py (in core/models) 66.7% 36/54
module total 56.6% 133/235

102 lines unexecuted across 6 files. Goal: bring sendsemantics/ to ≥90% line coverage and ensure every partitioning branch is exercised.

Test ideas:

  • Empty input batch routing for each partitioner
  • Single-partition / single-receiver edge case
  • Range partitioner: boundary keys, out-of-range keys, descending ranges
  • Hash partitioner: hash collisions, null/None keys, deterministic routing across runs
  • Round-robin: state continuity across multiple add() calls, restart behavior
  • Broadcast: ensure tuple is emitted to all receivers, not duplicated to same receiver
  • Backpressure / flush semantics if applicable to the partitioner contract
  • Schema mismatch / type coercion errors

Out of scope: integration tests with the engine; end-to-end shuffle correctness tests live in amber/'s sbt test suite. This task is unit-test only, runs under pytest in the existing python CI job.

Task Type

  • Testing / QA

Source: Codecov report at https://app.codecov.io/gh/apache/texera (commit f59e9e5e686fc34cbdfb8f4976706552cac01347).

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No fields configured for Task.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions