Skip to content

Commit

Permalink
Change filter_blank_rows when_any parameter to have a more user-frien…
Browse files Browse the repository at this point in the history
…dly type (#7935)

Added Blank_Selector constructor and applied to remove_blank_columns, select_blank_columns, filter_blank_rows for #7931 . Changed when_any to when for readability.
  • Loading branch information
Cassandra-Clark committed Nov 1, 2023
1 parent d467683 commit b5d6628
Show file tree
Hide file tree
Showing 9 changed files with 87 additions and 66 deletions.
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -588,6 +588,8 @@
- [Implemented `Table.expand_to_rows` for the in-memory backend.][8029]
- [Added XML support for `.to Table` and `.expand_column`.][8083]
- [Added `Previous_Value` option to `fill_nothing` and `fill_empty`.][8105]
- [Implemented new selector for when parameter in `filter_blank_rows`,
`select_blank_columns`, `remove_blank_columns`][7935]

[debug-shortcuts]:
https://github.com/enso-org/enso/blob/develop/app/gui/docs/product/shortcuts.md#debug
Expand Down Expand Up @@ -839,6 +841,7 @@
[8029]: https://github.com/enso-org/enso/pull/8029
[8083]: https://github.com/enso-org/enso/pull/8083
[8105]: https://github.com/enso-org/enso/pull/8105
[7935]: https://github.com/enso-org/enso/pull/7935

#### Enso Compiler

Expand Down
36 changes: 19 additions & 17 deletions distribution/lib/Standard/Database/0.0.0-dev/src/Data/Table.enso
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ from Standard.Base.Widget_Helpers import make_delimiter_selector
import Standard.Table.Data.Calculations.Column_Operation.Column_Operation
import Standard.Table.Data.Column_Ref.Column_Ref
import Standard.Table.Data.Constants.Previous_Value
import Standard.Table.Data.Blank_Selector.Blank_Selector
import Standard.Table.Data.Expression.Expression
import Standard.Table.Data.Expression.Expression_Error
import Standard.Table.Data.Join_Condition.Join_Condition
Expand Down Expand Up @@ -271,9 +272,9 @@ type Table
rows are present, all columns are considered blank.

Arguments:
- when_any: By default, only columns consisting of all blank cells are
selected. If set to `True`, columns with one or more blank values are
selected.
- when: By default, only columns consisting of all blank cells are
selected. If set to Blank_Selector.Any_Cell_Blank, columns with one or
more blank values are selected.
- treat_nans_as_blank: specified whether `Number.nan` is considered as
blank. By default, it is not.

Expand All @@ -284,9 +285,9 @@ type Table
Select completely blank columns from a table.

table.select_blank_columns
select_blank_columns : Boolean -> Boolean -> Table
select_blank_columns self (when_any : Boolean = False) (treat_nans_as_blank : Boolean = False) =
new_columns = self.columns_helper.select_blank_columns_helper when_any treat_nans_as_blank
select_blank_columns : Blank_Selector -> Boolean -> Table
select_blank_columns self (when:Blank_Selector = Blank_Selector.All_Cells_Blank) treat_nans_as_blank=False =
new_columns = self.columns_helper.select_blank_columns_helper when treat_nans_as_blank
if new_columns.length == 0 then Error.throw (No_Output_Columns) else
self.updated_columns new_columns

Expand All @@ -297,9 +298,9 @@ type Table
rows are present, all columns are considered blank.

Arguments:
- when_any: By default, only columns consisting of all blank cells are
selected. If set to `True`, columns with one or more blank values are
selected.
- when: By default, only columns consisting of all blank cells are
selected. If set to Blank_Selector.Any_Cell_Blank, columns with one or
more blank values are selected.
- treat_nans_as_blank: specified whether `Number.nan` is considered as
blank. By default, it is not.

Expand All @@ -310,9 +311,9 @@ type Table
Remove completely blank columns from a table.

table.remove_blank_columns
remove_blank_columns : Boolean -> Boolean -> Table
remove_blank_columns self (when_any : Boolean = False) (treat_nans_as_blank : Boolean = False) =
new_columns = self.columns_helper.select_blank_columns_helper when_any treat_nans_as_blank invert_selection=True
remove_blank_columns : Blank_Selector -> Boolean -> Table
remove_blank_columns self (when:Blank_Selector = Blank_Selector.All_Cells_Blank) treat_nans_as_blank=False =
new_columns = self.columns_helper.select_blank_columns_helper when treat_nans_as_blank invert_selection=True
if new_columns.length == 0 then Error.throw (No_Output_Columns) else
self.updated_columns new_columns

Expand Down Expand Up @@ -2122,15 +2123,16 @@ type Table
Remove rows which are all blank or containing blank values.

Arguments:
- when_any: If `True`, then remove any row containing any blank values.
If `False`, then only remove rows with all blank values.
- when: If Blank_Selector.Any_Cell_Blank, then remove any row containing
any blank values.
If Blank_Selector.All_Cells_Blank, then only remove rows with all blank values.
- treat_nans_as_blank: If `True`, then `Number.nan` is considered as blank.

? Blank values
Blank values are `Nothing`, `""` and depending on setting `Number.nan`.
filter_blank_rows : Boolean -> Boolean -> Table
filter_blank_rows self when_any=False treat_nans_as_blank=False =
Table_Helpers.filter_blank_rows self when_any treat_nans_as_blank
filter_blank_rows : Blank_Selector -> Boolean -> Table
filter_blank_rows self (when:Blank_Selector = Blank_Selector.All_Cells_Blank) treat_nans_as_blank=False =
Table_Helpers.filter_blank_rows self when treat_nans_as_blank

## ALIAS count
GROUP Standard.Base.Metadata
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
## TODO Documents
type Blank_Selector
## Blank_Selector is used as a constructor for other functions.
Any_Cell_Blank

All_Cells_Blank
35 changes: 19 additions & 16 deletions distribution/lib/Standard/Table/0.0.0-dev/src/Data/Table.enso
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ from Standard.Base.Widget_Helpers import make_delimiter_selector

import project.Data.Aggregate_Column.Aggregate_Column
import project.Data.Calculations.Column_Operation.Column_Operation
import project.Data.Blank_Selector.Blank_Selector
import project.Data.Column as Column_Module
import project.Data.Column.Column
import project.Data.Column_Ref.Column_Ref
Expand Down Expand Up @@ -52,6 +53,7 @@ import project.Internal.Split_Tokenize
import project.Internal.Table_Helpers
import project.Internal.Table_Helpers.Table_Column_Helper
import project.Internal.Widget_Helpers

from project.Data.Column import get_item_string, normalize_string_for_display
from project.Data.Type.Value_Type import Auto, Value_Type
from project.Errors import all
Expand Down Expand Up @@ -412,9 +414,9 @@ type Table
rows are present, all columns are considered blank.

Arguments:
- when_any: By default, only columns consisting of all blank cells are
selected. If set to `True`, columns with one or more blank values are
selected.
- when: By default, only columns consisting of all blank cells are
selected. If set to Blank_Selector.Any_Cell_Blank, columns with one or
more blank values are selected.
- treat_nans_as_blank: specifies whether `Number.nan` is considered as
blank. By default, it is not.

Expand All @@ -425,9 +427,9 @@ type Table
Select completely blank columns from a table.

table.select_blank_columns
select_blank_columns : Boolean -> Boolean -> Table ! No_Output_Columns
select_blank_columns self (when_any : Boolean = False) (treat_nans_as_blank : Boolean = False) =
new_columns = self.columns_helper.select_blank_columns_helper when_any treat_nans_as_blank
select_blank_columns : Blank_Selector -> Boolean -> Table ! No_Output_Columns
select_blank_columns self (when:Blank_Selector = Blank_Selector.All_Cells_Blank) (treat_nans_as_blank : Boolean = False) =
new_columns = self.columns_helper.select_blank_columns_helper when treat_nans_as_blank
if new_columns.length == 0 then Error.throw (No_Output_Columns) else
Table.new new_columns

Expand All @@ -438,8 +440,8 @@ type Table
rows are present, all columns are considered blank.

Arguments:
- when_any: By default, only columns consisting of all blank cells are
selected. If set to `True`, columns with one or more blank values are
- when By default, only columns consisting of all blank cells are
selected. If set to Blank_Selector.Any_Cell_Blank, columns with one or more blank values are
selected.
- treat_nans_as_blank: specified whether `Number.nan` is considered as
blank. By default, it is not.
Expand All @@ -451,9 +453,9 @@ type Table
Remove completely blank columns from a table.

table.remove_blank_columns
remove_blank_columns : Boolean -> Boolean -> Table ! No_Output_Columns
remove_blank_columns self (when_any : Boolean = False) (treat_nans_as_blank : Boolean = False) =
new_columns = self.columns_helper.select_blank_columns_helper when_any treat_nans_as_blank invert_selection=True
remove_blank_columns : Blank_Selector -> Boolean -> Table ! No_Output_Columns
remove_blank_columns self (when:Blank_Selector = Blank_Selector.All_Cells_Blank) (treat_nans_as_blank : Boolean = False) =
new_columns = self.columns_helper.select_blank_columns_helper when treat_nans_as_blank invert_selection=True
if new_columns.length == 0 then Error.throw (No_Output_Columns) else
Table.new new_columns

Expand Down Expand Up @@ -2016,15 +2018,16 @@ type Table
Remove rows which are all blank or containing blank values.

Arguments:
- when_any: If `True`, then remove any row containing any blank values.
If `False`, then only remove rows with all blank values.
- when: If Blank_Selector.Any_Cell_Blank, then remove any row containing
any blank values.
If Blank_Selector.All_Cells_Blank, then only remove rows with all blank values.
- treat_nans_as_blank: If `True`, then `Number.nan` is considered as blank.

? Blank values
Blank values are `Nothing`, `""` and depending on setting `Number.nan`.
filter_blank_rows : Boolean -> Boolean -> Table
filter_blank_rows self when_any=False treat_nans_as_blank=False =
Table_Helpers.filter_blank_rows self when_any treat_nans_as_blank
filter_blank_rows : Blank_Selector -> Boolean -> Table
filter_blank_rows self when=Blank_Selector.Any_Cell_Blank treat_nans_as_blank=False =
Table_Helpers.filter_blank_rows self when treat_nans_as_blank

## ALIAS count
GROUP Standard.Base.Metadata
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import Standard.Base.Errors.Illegal_Argument.Illegal_Argument
import Standard.Base.Errors.Illegal_State.Illegal_State

import project.Data.Aggregate_Column.Aggregate_Column
import project.Data.Blank_Selector.Blank_Selector
import project.Data.Column.Column
import project.Data.Position.Position
import project.Data.Set_Mode.Set_Mode
Expand Down Expand Up @@ -196,14 +197,15 @@ type Table_Column_Helper
completely blank or have some blanks.

Arguments:
- when_any: By default, only columns consisting of all blank cells are
selected. If set to `True`, columns with one or more blank values are
-TODO docs
- when: By default, only columns consisting of all blank cells are
selected. If set to Blank_Selector.Any_Cell_Blank, columns with one or more blank values are
selected.
- treat_nans_as_blank: If `True`, then `Number.nan` is considered as
blank.
- invert_selection: If `True`, then the selection is inverted.
select_blank_columns_helper : Boolean -> Boolean -> Boolean -> Vector
select_blank_columns_helper self when_any:Boolean treat_nans_as_blank:Boolean invert_selection:Boolean=False =
select_blank_columns_helper : Blank_Selector -> Boolean -> Boolean -> Vector
select_blank_columns_helper self (when:Blank_Selector = Blank_Selector.All_Cells_Blank) treat_nans_as_blank:Boolean invert_selection:Boolean=False =
blanks = self.internal_columns.map_with_index ix-> internal_column->
column = self.make_column internal_column
blank_indicator = column.is_blank treat_nans_as_blank
Expand All @@ -222,7 +224,9 @@ type Table_Column_Helper
just_indicators = table_with_blank_indicators.select_columns (blanks.map .name) on_problems=Problem_Behavior.Report_Error

# Maximum is equivalent to Exists and Minimum is equivalent to Forall.
col_aggregate = if when_any then Aggregate_Column.Maximum _ else Aggregate_Column.Minimum _
col_aggregate = case when of
Blank_Selector.Any_Cell_Blank -> Aggregate_Column.Maximum _
Blank_Selector.All_Cells_Blank -> Aggregate_Column.Minimum _
aggregates = blanks.map blanks_col-> col_aggregate blanks_col.name

aggregate_result = just_indicators.aggregate aggregates on_problems=Problem_Behavior.Report_Error
Expand Down Expand Up @@ -440,14 +444,15 @@ resolve_order_by internal_columns column_selectors problem_builder =
## PRIVATE
A helper method gathering the common logic for constructing expressions that
can filter out blank rows.
filter_blank_rows : Table -> Boolean -> Boolean -> Table
filter_blank_rows table when_any treat_nans_as_blank =
filter_blank_rows : Table -> Blank_Selector -> Boolean -> Table
filter_blank_rows table when treat_nans_as_blank =
cols = table.columns
case cols.not_empty of
True ->
merge = if when_any then (||) else (&&)
missing_mask = cols.map (_.is_blank treat_nans_as_blank) . reduce col1-> col2->
merge col1 col2 . rename "blank_indicator"
merge = case when of
Blank_Selector.Any_Cell_Blank -> (||)
Blank_Selector.All_Cells_Blank -> (&&)
missing_mask = cols.map (_.is_blank treat_nans_as_blank) . reduce merge
non_missing_mask = missing_mask.not
table.filter non_missing_mask Filter_Condition.Is_True
False -> table
Expand Down
2 changes: 2 additions & 0 deletions distribution/lib/Standard/Table/0.0.0-dev/src/Main.enso
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ from Standard.Base import all

import project.Data.Aggregate_Column.Aggregate_Column
import project.Data.Calculations.Column_Operation.Column_Operation
import project.Data.Blank_Selector.Blank_Selector
import project.Data.Column.Column
import project.Data.Column_Ref.Column_Ref
import project.Data.Column_Vector_Extensions
Expand Down Expand Up @@ -31,6 +32,7 @@ from project.Extensions.Table_Conversions import all

export project.Data.Aggregate_Column.Aggregate_Column
export project.Data.Calculations.Column_Operation.Column_Operation
export project.Data.Blank_Selector.Blank_Selector
export project.Data.Column.Column
export project.Data.Column_Ref.Column_Ref
export project.Data.Column_Vector_Extensions
Expand Down
Loading

0 comments on commit b5d6628

Please sign in to comment.