Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Ruby] Table.slice uses :emit_null not :drop for FilterOptions #34953

Closed
heronshoes opened this issue Apr 7, 2023 · 1 comment · Fixed by #34954
Closed

[Ruby] Table.slice uses :emit_null not :drop for FilterOptions #34953

heronshoes opened this issue Apr 7, 2023 · 1 comment · Fixed by #34954

Comments

@heronshoes
Copy link
Contributor

Describe the bug, including details regarding any error messages, version, and platform.

Table.slice behaves as FilterOptions.null_selection_behavior = :emit_null for backward compatibility.

This is a differnt behavior from Table#filter and Slicer::ColumnCondition#select
which use default option :drop.

Is it better to align with default option :drop for Table.slice ?

Example:

table = Arrow::Table.new(index: [*1..3], string: ["A", "B", nil])
=>
#<Arrow::Table:0x7fc7c71b61b0 ptr=0x5612150e1b40>
	index	string
0	    1	A     
1	    2	B     
2	    3	(null)

This case:

table.slice { |slicer| slicer.string == "A" }
=>
#<Arrow::Table:0x7fc7cf2bee10 ptr=0x5612165c5220>
	 index	string
0	     1	A     
1	(null)	(null)

Reference 1:

table.filter([true, nil, false])
=>
#<Arrow::Table:0x7fc7cf366a48 ptr=0x561216608250>
	index	string
0	    1	A     

Reference 2:

table.slice { |slicer| slicer.string.select { |value| value == "A" } }
=>
#<Arrow::Table:0x7fc7c6f7f2e8 ptr=0x5612166326e0>
	index	string
0	    1	A    

Component(s)

Ruby

kou pushed a commit that referenced this issue Apr 10, 2023
…:drop` (#34954)

### Rationale for this change

`Table.slice` behaves as `FilterOptions.null_selection_behavior = :emit_null` for backward compatibility.
But this is a differnt behavior from `Table#filter` and `Slicer::ColumnCondition#select` 
which use default option `:drop`.

This request will change the behavior of `Table#slice` to align with the default of `FilterOptions`.

### What changes are included in this PR?

Set `FilterOptions.null_selection_behavior` to the default value `:drop` from `:emit_null` .

### Are these changes tested?

Yes. Rebased after #34952.

### Are there any user-facing changes?

Yes.

**This PR includes breaking changes to public APIs.**

* Closes: #34953

Authored-by: Hirokazu SUZUKI <heronshoes877@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@kou kou added this to the 12.0.0 milestone Apr 10, 2023
@heronshoes
Copy link
Contributor Author

Thhanks!

liujiacheng777 pushed a commit to LoongArch-Python/arrow that referenced this issue May 11, 2023
…` to `:drop` (apache#34954)

### Rationale for this change

`Table.slice` behaves as `FilterOptions.null_selection_behavior = :emit_null` for backward compatibility.
But this is a differnt behavior from `Table#filter` and `Slicer::ColumnCondition#select` 
which use default option `:drop`.

This request will change the behavior of `Table#slice` to align with the default of `FilterOptions`.

### What changes are included in this PR?

Set `FilterOptions.null_selection_behavior` to the default value `:drop` from `:emit_null` .

### Are these changes tested?

Yes. Rebased after apache#34952.

### Are there any user-facing changes?

Yes.

**This PR includes breaking changes to public APIs.**

* Closes: apache#34953

Authored-by: Hirokazu SUZUKI <heronshoes877@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
ArgusLi pushed a commit to Bit-Quill/arrow that referenced this issue May 15, 2023
…` to `:drop` (apache#34954)

### Rationale for this change

`Table.slice` behaves as `FilterOptions.null_selection_behavior = :emit_null` for backward compatibility.
But this is a differnt behavior from `Table#filter` and `Slicer::ColumnCondition#select` 
which use default option `:drop`.

This request will change the behavior of `Table#slice` to align with the default of `FilterOptions`.

### What changes are included in this PR?

Set `FilterOptions.null_selection_behavior` to the default value `:drop` from `:emit_null` .

### Are these changes tested?

Yes. Rebased after apache#34952.

### Are there any user-facing changes?

Yes.

**This PR includes breaking changes to public APIs.**

* Closes: apache#34953

Authored-by: Hirokazu SUZUKI <heronshoes877@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
rtpsw pushed a commit to rtpsw/arrow that referenced this issue May 16, 2023
…` to `:drop` (apache#34954)

### Rationale for this change

`Table.slice` behaves as `FilterOptions.null_selection_behavior = :emit_null` for backward compatibility.
But this is a differnt behavior from `Table#filter` and `Slicer::ColumnCondition#select` 
which use default option `:drop`.

This request will change the behavior of `Table#slice` to align with the default of `FilterOptions`.

### What changes are included in this PR?

Set `FilterOptions.null_selection_behavior` to the default value `:drop` from `:emit_null` .

### Are these changes tested?

Yes. Rebased after apache#34952.

### Are there any user-facing changes?

Yes.

**This PR includes breaking changes to public APIs.**

* Closes: apache#34953

Authored-by: Hirokazu SUZUKI <heronshoes877@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants