PanicExcept/Entered unreachable code with streaming engine

[phobson]

Checks

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of Polars.

Reproducible example

from datetime import  datetime, timedelta

import numpy as np
import polars as pl


DTCOL = "datetime"

precipcol: str = "precip"
obs_period_minutes: int = 5
intervent_dry_hours: int = 2
minimum_depth: float = 0.001
minimum_observations: int = 2


_tstamps = pl.datetime_range(
    datetime(2023, 1, 1),
    datetime(2024, 1, 1),
    interval=timedelta(minutes=obs_period_minutes),
    eager=True
)

rs = np.random.RandomState(37)
precip = rs.uniform(0, 0.3, size=_tstamps.shape[0])
mask = rs.choice([0, 1], size=_tstamps.shape[0], p=[0.75, 0.25])
precip = np.where(mask, precip, 0).round(2)

df = pl.DataFrame({DTCOL: _tstamps, precipcol: precip}).lazy()

depth_to_intensity = 60 / obs_period_minutes
polars_storms = (
    df.filter(pl.col(precipcol).gt(0) & pl.col(precipcol).is_not_null())
    .with_columns(pl.col(DTCOL).diff().alias("tdelta"))
    .with_columns(
        pl.col("tdelta")
        .gt(timedelta(hours=intervent_dry_hours))
        .fill_null(True)
        .alias("is_start")
    )
    .with_columns(pl.col("is_start").cum_sum().alias("storm_id"))
    .group_by("storm_id")
    .agg([
        pl.col(precipcol).sum().alias("storm_total"),
        pl.col(precipcol).max().mul(depth_to_intensity).alias("max_inten_in_per_hr"),
        pl.col(DTCOL).min().alias("storm_start"),
        pl.col(DTCOL).max().alias("storm_stop"),
        ((pl.col(DTCOL).max() - pl.col(DTCOL).min()).dt.total_minutes() / 60).alias(
            "duration_hours"
        ),
        (
            pl.col(precipcol).sum()
            / (pl.col(DTCOL).max() - pl.col(DTCOL).min()).dt.total_hours()
        ).alias("avg_inten_in_per_hour"),
    ])
)

polars_storms.collect(engine="streaming")

Log output

$ POLARS_VERBOSE=1 python scratch_polars_panic.py                                                                            
polars-stream: updating graph state
async thread count: 4
polars-stream: running shift in subgraph
polars-stream: running input-independent-select in subgraph
polars-stream: done running graph phase
polars-stream: updating graph state
polars-stream: running in-memory-source in subgraph
polars-stream: running group-by in subgraph
polars-stream: running select in subgraph
polars-stream: running select in subgraph
polars-stream: running zip in subgraph
polars-stream: running multiplexer in subgraph
polars-stream: running with-columns in subgraph
polars-stream: running filter in subgraph
polars-stream: running select in subgraph
polars-stream: running zip in subgraph
polars-stream: running shift in subgraph
polars-stream: running select in subgraph
polars-stream: running cum_sum in subgraph
polars-stream: running select in subgraph
polars-stream: running simple-projection in subgraph
polars-stream: running multiplexer in subgraph

thread 'async-executor-3' (162392) panicked at crates/polars-expr/src/hot_groups/row_encoded.rs:48:13:
internal error: entered unreachable code
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

thread 'async-executor-7' (162396) panicked at crates/polars-expr/src/hot_groups/row_encoded.rs:48:13:
internal error: entered unreachable code

thread 'async-executor-2' (162391) panicked at crates/polars-expr/src/hot_groups/row_encoded.rs:48:13:
internal error: entered unreachable code

thread 'async-executor-1' (162390) panicked at crates/polars-expr/src/hot_groups/row_encoded.rs:48:13:
internal error: entered unreachable code

thread 'async-executor-5' (162394) panicked at crates/polars-expr/src/hot_groups/row_encoded.rs:48:13:
internal error: entered unreachable code

thread 'async-executor-2' (162391) panicked at crates/polars-expr/src/hot_groups/row_encoded.rs:48:13:
internal error: entered unreachable code

thread 'async-executor-7' (162396) panicked at crates/polars-expr/src/hot_groups/row_encoded.rs:48:13:
internal error: entered unreachable code

thread 'async-executor-1' (162390) panicked at crates/polars-expr/src/hot_groups/row_encoded.rs:48:13:
internal error: entered unreachable code
Traceback (most recent call last):
  File "/home/paul/sources/hec-drip-edms/scratch_polars_panic.py", line 59, in <module>
    polars_storms.collect(engine="streaming")
  File "/home/paul/sources/hec-drip-edms/.venv/lib/python3.11/site-packages/polars/_utils/deprecation.py", line 97, in wrapper
    return function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/paul/sources/hec-drip-edms/.venv/lib/python3.11/site-packages/polars/lazyframe/opt_flags.py", line 330, in wrapper
    return function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/paul/sources/hec-drip-edms/.venv/lib/python3.11/site-packages/polars/lazyframe/frame.py", line 2407, in collect
    return wrap_df(ldf.collect(engine, callback))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pyo3_runtime.PanicException: internal error: entered unreachable code

Issue description

Hey folks,

I just tried the new 1.33.1 version of polars. And get the following error message when using the streaming engine (in-memory works)

Previously (v1.32.3), this query worked with both the in-memory and streaming engines.

In addition to the POLARS_VERBOSE=1 output, I've attached the stderr output with RUST_BACKTRACE as a file

Expected behavior

The streaming engine should produce identical results to the in-memory engine and not throw a PanicException

Installed versions

Details

--------Version info---------
Polars:              1.33.1
Index type:          UInt32
Platform:            Linux-6.6.87.2-microsoft-standard-WSL2-x86_64-with-glibc2.35
Python:              3.11.11 | packaged by conda-forge | (main, Dec  5 2024, 14:17:24) [GCC 13.3.0]
LTS CPU:             False

----Optional dependencies----
Azure CLI            2.74.0
adbc_driver_manager  <not installed>
altair               <not installed>
azure.identity       1.21.0
boto3                <not installed>
cloudpickle          <not installed>
connectorx           <not installed>
deltalake            <not installed>
fastexcel            <not installed>
fsspec               2025.3.2
gevent               <not installed>
google.auth          <not installed>
great_tables         0.18.0
matplotlib           <not installed>
numpy                2.2.5
openpyxl             3.1.5
pandas               2.2.3
polars_cloud         <not installed>
pyarrow              20.0.0
pydantic             2.11.3
pyiceberg            <not installed>
sqlalchemy           <not installed>
torch                <not installed>
xlsx2csv             <not installed>
xlsxwriter           <not installed>

rust_backtrace.log

[cmdlineluser]

Attempted MRE:

df = pl.LazyFrame({"x": [1, 2]})

df.group_by((pl.col.x > 1).cum_sum()).agg().collect(engine="streaming")
# thread 'async-executor-6' (20877275) panicked at crates/polars-expr/src/hot_groups/row_encoded.rs:48:13:
# internal error: entered unreachable code

[phobson]

@cmdlineluser way simpler! thanks for boiling that down. You'll have to take my word for it that I stripped away a lot of complexity from a terribly intertwined test suite 😅

[ritchie46]

@orlp can you take a look?

[coastalwhite]

My guess is that this is related to #24289.

[orlp]

@coastalwhite The dtype from #23977 was incorrect.