Skip to content

UNION ALL panics on wasm32-unknown-unknown #24886

Description

@kentkwu

Describe the bug

DataFusion compiled to wasm32-unknown-unknown panics on UNION ALL and any other multi-partition query whose collect funnels through CoalescePartitionsExec (aggregations over partitioned scans, sorts across partitions, etc.).

CoalescePartitionsExec::execute spawns one task per input partition via JoinSet::spawn on the _ => (multi-input) branch. JoinSet::spawn needs a tokio reactor that isn't installed under wasm-bindgen-futures. The panic:

there is no reactor running, must be called from the context of a Tokio 1.x runtime

The single-input branch of CoalescePartitionsExec::execute works — it passes the child stream through directly. Only the multi-input branch spawns.

Follow-up to #24275, which fixed the same class of panic at a different call site (collect_partitioned).

To Reproduce

Add this to datafusion/wasmtest/src/lib.rs:

#[wasm_bindgen_test(unsupported = tokio::test)]
async fn test_union_all() {
    let ctx = get_ctx();
    ctx.sql("SELECT 1 AS n UNION ALL SELECT 2 AS n")
        .await.unwrap()
        .collect().await.unwrap();
}

Run from the repo root:

RUSTFLAGS='--cfg getrandom_backend="wasm_js"' \
wasm-pack test --headless --chrome datafusion/wasmtest

The test panics:

panicked at datafusion/common-runtime/src/join_set.rs:69:20:
there is no reactor running, must be called from the context of a Tokio 1.x runtime

Expected behavior

Multi-partition collect should succeed under wasm-bindgen-futures when the session has asked for single-threaded execution (target_partitions == 1). Sequential drain of the input partitions is a valid interleaving — CoalescePartitionsExec's own docstring states "No guarantees are made about the order of the resulting partition."

Additional context

Related: #24274, #24275, #13815

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions