Describe the bug
DataFusion compiled to wasm32-unknown-unknown panics on UNION ALL and any other multi-partition query whose collect funnels through CoalescePartitionsExec (aggregations over partitioned scans, sorts across partitions, etc.).
CoalescePartitionsExec::execute spawns one task per input partition via JoinSet::spawn on the _ => (multi-input) branch. JoinSet::spawn needs a tokio reactor that isn't installed under wasm-bindgen-futures. The panic:
there is no reactor running, must be called from the context of a Tokio 1.x runtime
The single-input branch of CoalescePartitionsExec::execute works — it passes the child stream through directly. Only the multi-input branch spawns.
Follow-up to #24275, which fixed the same class of panic at a different call site (collect_partitioned).
To Reproduce
Add this to datafusion/wasmtest/src/lib.rs:
#[wasm_bindgen_test(unsupported = tokio::test)]
async fn test_union_all() {
let ctx = get_ctx();
ctx.sql("SELECT 1 AS n UNION ALL SELECT 2 AS n")
.await.unwrap()
.collect().await.unwrap();
}
Run from the repo root:
RUSTFLAGS='--cfg getrandom_backend="wasm_js"' \
wasm-pack test --headless --chrome datafusion/wasmtest
The test panics:
panicked at datafusion/common-runtime/src/join_set.rs:69:20:
there is no reactor running, must be called from the context of a Tokio 1.x runtime
Expected behavior
Multi-partition collect should succeed under wasm-bindgen-futures when the session has asked for single-threaded execution (target_partitions == 1). Sequential drain of the input partitions is a valid interleaving — CoalescePartitionsExec's own docstring states "No guarantees are made about the order of the resulting partition."
Additional context
Related: #24274, #24275, #13815
Describe the bug
DataFusion compiled to
wasm32-unknown-unknownpanics onUNION ALLand any other multi-partition query whose collect funnels throughCoalescePartitionsExec(aggregations over partitioned scans, sorts across partitions, etc.).CoalescePartitionsExec::executespawns one task per input partition viaJoinSet::spawnon the_ =>(multi-input) branch.JoinSet::spawnneeds a tokio reactor that isn't installed underwasm-bindgen-futures. The panic:The single-input branch of
CoalescePartitionsExec::executeworks — it passes the child stream through directly. Only the multi-input branch spawns.Follow-up to #24275, which fixed the same class of panic at a different call site (
collect_partitioned).To Reproduce
Add this to
datafusion/wasmtest/src/lib.rs:Run from the repo root:
The test panics:
Expected behavior
Multi-partition collect should succeed under
wasm-bindgen-futureswhen the session has asked for single-threaded execution (target_partitions == 1). Sequential drain of the input partitions is a valid interleaving —CoalescePartitionsExec's own docstring states "No guarantees are made about the order of the resulting partition."Additional context
Related: #24274, #24275, #13815