Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Filter has inconsistent null handling #295

Closed
ritchie46 opened this issue May 16, 2021 · 0 comments · Fixed by #296
Closed

Filter has inconsistent null handling #295

ritchie46 opened this issue May 16, 2021 · 0 comments · Fixed by #296
Labels

Comments

@ritchie46
Copy link
Contributor

Describe the bug
When creating a boolean mask from a comparison operation on an array with null values, the filter kernel yields a different result than when that boolean mask, including null values, is created directly.

To Reproduce

    fn test_null_mask() -> Result<()> {
        use crate::compute::kernels::comparison;
        let a: PrimitiveArray<Int64Type> = PrimitiveArray::from(vec![Some(1), Some(2), None]);
        let mask0 = comparison::eq(&a, &a)?;
        let out0 = filter(&a, &mask0)?;
        let out_arr0 = out0.as_any().downcast_ref::<PrimitiveArray<Int64Type>>().unwrap();

        let mask1 = BooleanArray::from(vec![Some(true), Some(true), None]);
        let out1 = filter(&a, &mask1)?;
        let out_arr1 = out1.as_any().downcast_ref::<PrimitiveArray<Int64Type>>().unwrap();
        assert_eq!(mask0, mask1); // assert succes
        assert_eq!(out_arr0, out_arr1);  // assert false
        Ok(())
    }

Expected behavior
Null values in the predicate mask should be ignored and not yield a result. Which the kernel does correctly on boolean mask that are created directly.

@ritchie46 ritchie46 added the bug label May 16, 2021
@ritchie46 ritchie46 changed the title Fitler has inconsistent null handling Filter has inconsistent null handling May 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant