Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regression bug on 0.13.49 - filter with apply on list[cat] not working #3784

Closed
thobai opened this issue Jun 23, 2022 · 0 comments · Fixed by #3788
Closed

Regression bug on 0.13.49 - filter with apply on list[cat] not working #3784

thobai opened this issue Jun 23, 2022 · 0 comments · Fixed by #3788
Labels
bug Something isn't working

Comments

@thobai
Copy link

thobai commented Jun 23, 2022

What language are you using?

Python

Have you tried latest version of polars?

  • [yes]

What version of polars are you using?

0.13.49

What operating system are you using polars on?

MacOS 11.6.7

What language version are you using

python 3.10.4

Describe your bug.

Filter with apply on list of categorical that works on version 0.13.48 an earlier fails on version 0.13.49.

What are the steps to reproduce the behavior?

import polars as pl

df = pl.DataFrame(
    {
        'str': ['A', 'B', 'A', 'B', 'C'],
        'group': [1,1,2,1,2]
    }
).lazy()
df = df.with_column(pl.col('str').cast(pl.Categorical))

df_groups = df.groupby('group').agg([pl.col('str').list().alias('str_list')])

pre = "A"
succ = "B"

(
    df_groups
    .filter(pl.col("str_list").apply(
        lambda variant: 
            pre in variant and succ in variant and 
            variant.to_list().index(pre) < variant.to_list().index(succ)
    ))
    .collect()
)

Example

import polars as pl
import numpy as np

# Create a simple dataset on which we can reproduce the bug.
pl.DataFrame({
    "foo": [None, 1, 2],
    "bar": np.arange(3)
})

What is the actual behavior?

thread '<unnamed>' panicked at 'could not find supertype of binary expr: ComputeError("Failed to determine supertype of List(Categorical(None)) and Boolean")', local_dependencies/polars-lazy/src/logical_plan/optimizer/type_coercion.rs:279:30```

What is the expected behavior?

This is the expected output that you get in version 0.13.48 and earlier:

group | str_list
-- | --
i64 | list[cat]
1 | ["A", "B", "B"]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant