Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change filter_blank_rows when_any parameter to have a more user-friendly type #7935

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -588,6 +588,8 @@
- [Implemented `Table.expand_to_rows` for the in-memory backend.][8029]
- [Added XML support for `.to Table` and `.expand_column`.][8083]
- [Added `Previous_Value` option to `fill_nothing` and `fill_empty`.][8105]
- [Implemented new selector for when parameter in `filter_blank_rows`,
`select_blank_columns`, `remove_blank_columns`][7935]

[debug-shortcuts]:
https://github.com/enso-org/enso/blob/develop/app/gui/docs/product/shortcuts.md#debug
Expand Down Expand Up @@ -839,6 +841,7 @@
[8029]: https://github.com/enso-org/enso/pull/8029
[8083]: https://github.com/enso-org/enso/pull/8083
[8105]: https://github.com/enso-org/enso/pull/8105
[7935]: https://github.com/enso-org/enso/pull/7935

#### Enso Compiler

Expand Down
36 changes: 19 additions & 17 deletions distribution/lib/Standard/Database/0.0.0-dev/src/Data/Table.enso
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ from Standard.Base.Widget_Helpers import make_delimiter_selector
import Standard.Table.Data.Calculations.Column_Operation.Column_Operation
import Standard.Table.Data.Column_Ref.Column_Ref
import Standard.Table.Data.Constants.Previous_Value
import Standard.Table.Data.Blank_Selector.Blank_Selector
import Standard.Table.Data.Expression.Expression
import Standard.Table.Data.Expression.Expression_Error
import Standard.Table.Data.Join_Condition.Join_Condition
Expand Down Expand Up @@ -271,9 +272,9 @@ type Table
rows are present, all columns are considered blank.

Arguments:
- when_any: By default, only columns consisting of all blank cells are
selected. If set to `True`, columns with one or more blank values are
selected.
- when: By default, only columns consisting of all blank cells are
selected. If set to Blank_Selector.Any_Cell_Blank, columns with one or
more blank values are selected.
- treat_nans_as_blank: specified whether `Number.nan` is considered as
blank. By default, it is not.

Expand All @@ -284,9 +285,9 @@ type Table
Select completely blank columns from a table.

table.select_blank_columns
select_blank_columns : Boolean -> Boolean -> Table
select_blank_columns self (when_any : Boolean = False) (treat_nans_as_blank : Boolean = False) =
new_columns = self.columns_helper.select_blank_columns_helper when_any treat_nans_as_blank
select_blank_columns : Blank_Selector -> Boolean -> Table
select_blank_columns self (when:Blank_Selector = Blank_Selector.All_Cells_Blank) treat_nans_as_blank=False =
new_columns = self.columns_helper.select_blank_columns_helper when treat_nans_as_blank
if new_columns.length == 0 then Error.throw (No_Output_Columns) else
self.updated_columns new_columns

Expand All @@ -297,9 +298,9 @@ type Table
rows are present, all columns are considered blank.

Arguments:
- when_any: By default, only columns consisting of all blank cells are
selected. If set to `True`, columns with one or more blank values are
selected.
- when: By default, only columns consisting of all blank cells are
selected. If set to Blank_Selector.Any_Cell_Blank, columns with one or
more blank values are selected.
- treat_nans_as_blank: specified whether `Number.nan` is considered as
blank. By default, it is not.

Expand All @@ -310,9 +311,9 @@ type Table
Remove completely blank columns from a table.

table.remove_blank_columns
remove_blank_columns : Boolean -> Boolean -> Table
remove_blank_columns self (when_any : Boolean = False) (treat_nans_as_blank : Boolean = False) =
new_columns = self.columns_helper.select_blank_columns_helper when_any treat_nans_as_blank invert_selection=True
remove_blank_columns : Blank_Selector -> Boolean -> Table
remove_blank_columns self (when:Blank_Selector = Blank_Selector.All_Cells_Blank) treat_nans_as_blank=False =
new_columns = self.columns_helper.select_blank_columns_helper when treat_nans_as_blank invert_selection=True
if new_columns.length == 0 then Error.throw (No_Output_Columns) else
self.updated_columns new_columns

Expand Down Expand Up @@ -2111,15 +2112,16 @@ type Table
Remove rows which are all blank or containing blank values.

Arguments:
- when_any: If `True`, then remove any row containing any blank values.
If `False`, then only remove rows with all blank values.
- when: If Blank_Selector.Any_Cell_Blank, then remove any row containing
any blank values.
If Blank_Selector.All_Cells_Blank, then only remove rows with all blank values.
- treat_nans_as_blank: If `True`, then `Number.nan` is considered as blank.

? Blank values
Blank values are `Nothing`, `""` and depending on setting `Number.nan`.
filter_blank_rows : Boolean -> Boolean -> Table
filter_blank_rows self when_any=False treat_nans_as_blank=False =
Table_Helpers.filter_blank_rows self when_any treat_nans_as_blank
filter_blank_rows : Blank_Selector -> Boolean -> Table
filter_blank_rows self (when:Blank_Selector = Blank_Selector.All_Cells_Blank) treat_nans_as_blank=False =
Table_Helpers.filter_blank_rows self when treat_nans_as_blank

## ALIAS count
GROUP Standard.Base.Metadata
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
## TODO Documents
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should add a piece of docs here.

type Blank_Selector
## Blank_Selector is used as a constructor for other functions.
Any_Cell_Blank

All_Cells_Blank
Cassandra-Clark marked this conversation as resolved.
Show resolved Hide resolved
35 changes: 19 additions & 16 deletions distribution/lib/Standard/Table/0.0.0-dev/src/Data/Table.enso
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ from Standard.Base.Widget_Helpers import make_delimiter_selector

import project.Data.Aggregate_Column.Aggregate_Column
import project.Data.Calculations.Column_Operation.Column_Operation
import project.Data.Blank_Selector.Blank_Selector
import project.Data.Column as Column_Module
import project.Data.Column.Column
import project.Data.Column_Ref.Column_Ref
Expand Down Expand Up @@ -52,6 +53,7 @@ import project.Internal.Split_Tokenize
import project.Internal.Table_Helpers
import project.Internal.Table_Helpers.Table_Column_Helper
import project.Internal.Widget_Helpers

from project.Data.Column import get_item_string, normalize_string_for_display
from project.Data.Type.Value_Type import Auto, Value_Type
from project.Errors import all
Expand Down Expand Up @@ -412,9 +414,9 @@ type Table
rows are present, all columns are considered blank.

Arguments:
- when_any: By default, only columns consisting of all blank cells are
selected. If set to `True`, columns with one or more blank values are
selected.
- when: By default, only columns consisting of all blank cells are
selected. If set to Blank_Selector.Any_Cell_Blank, columns with one or
more blank values are selected.
- treat_nans_as_blank: specifies whether `Number.nan` is considered as
blank. By default, it is not.

Expand All @@ -425,9 +427,9 @@ type Table
Select completely blank columns from a table.

table.select_blank_columns
select_blank_columns : Boolean -> Boolean -> Table ! No_Output_Columns
select_blank_columns self (when_any : Boolean = False) (treat_nans_as_blank : Boolean = False) =
new_columns = self.columns_helper.select_blank_columns_helper when_any treat_nans_as_blank
select_blank_columns : Blank_Selector -> Boolean -> Table ! No_Output_Columns
select_blank_columns self (when:Blank_Selector = Blank_Selector.All_Cells_Blank) (treat_nans_as_blank : Boolean = False) =
new_columns = self.columns_helper.select_blank_columns_helper when treat_nans_as_blank
if new_columns.length == 0 then Error.throw (No_Output_Columns) else
Table.new new_columns

Expand All @@ -438,8 +440,8 @@ type Table
rows are present, all columns are considered blank.

Arguments:
- when_any: By default, only columns consisting of all blank cells are
selected. If set to `True`, columns with one or more blank values are
- when By default, only columns consisting of all blank cells are
selected. If set to Blank_Selector.Any_Cell_Blank, columns with one or more blank values are
selected.
- treat_nans_as_blank: specified whether `Number.nan` is considered as
blank. By default, it is not.
Expand All @@ -451,9 +453,9 @@ type Table
Remove completely blank columns from a table.

table.remove_blank_columns
remove_blank_columns : Boolean -> Boolean -> Table ! No_Output_Columns
remove_blank_columns self (when_any : Boolean = False) (treat_nans_as_blank : Boolean = False) =
new_columns = self.columns_helper.select_blank_columns_helper when_any treat_nans_as_blank invert_selection=True
remove_blank_columns : Blank_Selector -> Boolean -> Table ! No_Output_Columns
remove_blank_columns self (when:Blank_Selector = Blank_Selector.All_Cells_Blank) (treat_nans_as_blank : Boolean = False) =
new_columns = self.columns_helper.select_blank_columns_helper when treat_nans_as_blank invert_selection=True
if new_columns.length == 0 then Error.throw (No_Output_Columns) else
Table.new new_columns

Expand Down Expand Up @@ -2004,15 +2006,16 @@ type Table
Remove rows which are all blank or containing blank values.

Arguments:
- when_any: If `True`, then remove any row containing any blank values.
If `False`, then only remove rows with all blank values.
- when: If Blank_Selector.Any_Cell_Blank, then remove any row containing
any blank values.
If Blank_Selector.All_Cells_Blank, then only remove rows with all blank values.
- treat_nans_as_blank: If `True`, then `Number.nan` is considered as blank.

? Blank values
Blank values are `Nothing`, `""` and depending on setting `Number.nan`.
filter_blank_rows : Boolean -> Boolean -> Table
filter_blank_rows self when_any=False treat_nans_as_blank=False =
Table_Helpers.filter_blank_rows self when_any treat_nans_as_blank
filter_blank_rows : Blank_Selector -> Boolean -> Table
filter_blank_rows self when=Blank_Selector.Any_Cell_Blank treat_nans_as_blank=False =
Table_Helpers.filter_blank_rows self when treat_nans_as_blank

## ALIAS count
GROUP Standard.Base.Metadata
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import Standard.Base.Errors.Illegal_Argument.Illegal_Argument
import Standard.Base.Errors.Illegal_State.Illegal_State

import project.Data.Aggregate_Column.Aggregate_Column
import project.Data.Blank_Selector.Blank_Selector
import project.Data.Column.Column
import project.Data.Position.Position
import project.Data.Set_Mode.Set_Mode
Expand Down Expand Up @@ -196,14 +197,15 @@ type Table_Column_Helper
completely blank or have some blanks.

Arguments:
- when_any: By default, only columns consisting of all blank cells are
selected. If set to `True`, columns with one or more blank values are
-TODO docs
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The TODO should be removed

- when: By default, only columns consisting of all blank cells are
selected. If set to Blank_Selector.Any_Cell_Blank, columns with one or more blank values are
selected.
- treat_nans_as_blank: If `True`, then `Number.nan` is considered as
blank.
- invert_selection: If `True`, then the selection is inverted.
select_blank_columns_helper : Boolean -> Boolean -> Boolean -> Vector
select_blank_columns_helper self when_any:Boolean treat_nans_as_blank:Boolean invert_selection:Boolean=False =
select_blank_columns_helper : Blank_Selector -> Boolean -> Boolean -> Vector
select_blank_columns_helper self (when:Blank_Selector = Blank_Selector.All_Cells_Blank) treat_nans_as_blank:Boolean invert_selection:Boolean=False =
Cassandra-Clark marked this conversation as resolved.
Show resolved Hide resolved
blanks = self.internal_columns.map_with_index ix-> internal_column->
column = self.make_column internal_column
blank_indicator = column.is_blank treat_nans_as_blank
Expand All @@ -222,7 +224,9 @@ type Table_Column_Helper
just_indicators = table_with_blank_indicators.select_columns (blanks.map .name) on_problems=Problem_Behavior.Report_Error

# Maximum is equivalent to Exists and Minimum is equivalent to Forall.
col_aggregate = if when_any then Aggregate_Column.Maximum _ else Aggregate_Column.Minimum _
col_aggregate = case when of
Blank_Selector.Any_Cell_Blank -> Aggregate_Column.Maximum _
Blank_Selector.All_Cells_Blank -> Aggregate_Column.Minimum _
aggregates = blanks.map blanks_col-> col_aggregate blanks_col.name

aggregate_result = just_indicators.aggregate aggregates on_problems=Problem_Behavior.Report_Error
Expand Down Expand Up @@ -440,14 +444,15 @@ resolve_order_by internal_columns column_selectors problem_builder =
## PRIVATE
A helper method gathering the common logic for constructing expressions that
can filter out blank rows.
filter_blank_rows : Table -> Boolean -> Boolean -> Table
filter_blank_rows table when_any treat_nans_as_blank =
filter_blank_rows : Table -> Blank_Selector -> Boolean -> Table
filter_blank_rows table when treat_nans_as_blank =
cols = table.columns
case cols.not_empty of
True ->
merge = if when_any then (||) else (&&)
missing_mask = cols.map (_.is_blank treat_nans_as_blank) . reduce col1-> col2->
merge col1 col2 . rename "blank_indicator"
merge = case when of
Blank_Selector.Any_Cell_Blank -> (||)
Blank_Selector.All_Cells_Blank -> (&&)
missing_mask = cols.map (_.is_blank treat_nans_as_blank) . reduce merge
non_missing_mask = missing_mask.not
table.filter non_missing_mask Filter_Condition.Is_True
False -> table
Expand Down
2 changes: 2 additions & 0 deletions distribution/lib/Standard/Table/0.0.0-dev/src/Main.enso
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ from Standard.Base import all

import project.Data.Aggregate_Column.Aggregate_Column
import project.Data.Calculations.Column_Operation.Column_Operation
import project.Data.Blank_Selector.Blank_Selector
import project.Data.Column.Column
import project.Data.Column_Ref.Column_Ref
import project.Data.Column_Vector_Extensions
Expand Down Expand Up @@ -31,6 +32,7 @@ from project.Extensions.Table_Conversions import all

export project.Data.Aggregate_Column.Aggregate_Column
export project.Data.Calculations.Column_Operation.Column_Operation
export project.Data.Blank_Selector.Blank_Selector
export project.Data.Column.Column
export project.Data.Column_Ref.Column_Ref
export project.Data.Column_Vector_Extensions
Expand Down
Loading
Loading