Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Renamed lookup_and_replace to merge and renamed Table.replace to text… #8564

Merged
merged 15 commits into from
Dec 20, 2023
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -593,6 +593,7 @@
- [Added `Table.format` for the in-memory backend.][8150]
- [Implemented truncate `Date_Time` for database backend (Postgres only).][8235]
- [Initial Enso Cloud APIs.][8006]
- [Renamed `Table.replace` to `text_replace` and renamed `Table.lookup_and_replace` to `Table.merge`.][8564]
- [Errors thrown inside `map` are wrapped in `Map_Error`.][8307]
- [Support for loading big Excel files.][8403]
- [Added new `Filter_Condition`s - `Equal_Ignore_Case`, `Is_Nan`, `Is_Infinite`
Expand Down Expand Up @@ -852,6 +853,7 @@
[8105]: https://github.com/enso-org/enso/pull/8105
[8150]: https://github.com/enso-org/enso/pull/8150
[8235]: https://github.com/enso-org/enso/pull/8235
[8564]: https://github.com/enso-org/enso/pull/8564
[8307]: https://github.com/enso-org/enso/pull/8307
[8403]: https://github.com/enso-org/enso/pull/8403
[8539]: https://github.com/enso-org/enso/pull/8539
Expand Down
23 changes: 12 additions & 11 deletions distribution/lib/Standard/Database/0.0.0-dev/src/Data/Table.enso
Original file line number Diff line number Diff line change
Expand Up @@ -1360,10 +1360,11 @@ type Table

## GROUP Standard.Base.Calculations
ICON join
Replaces values in this table by values from a lookup table.
Merges this table with a lookup table.
New values are looked up in the lookup table based on the `key_columns`.
Columns found in the lookup table values are replaced by values from the
lookup. Columns not found are left unchanged.
Columns that exist in the lookup table where a match was found are
replaced by values from the lookup table. Columns not found are left
unchanged.
This operation is similar to `Table.update_rows`, but just returns a new
`Table` instance, instead of updating the table in-place (which is only
possible for Database tables).
Expand Down Expand Up @@ -1412,10 +1413,10 @@ type Table
- If `add_new_columns` is `False` and the lookup table has columns
that are not present in this table, an `Unexpected_Extra_Columns`.
@key_columns Widget_Helpers.make_column_name_vector_selector
lookup_and_replace : Table -> (Vector (Integer | Text | Regex) | Text | Integer | Regex) -> Boolean -> Boolean -> Problem_Behavior -> Table ! Missing_Input_Columns | Non_Unique_Key | Unmatched_Rows_In_Lookup
lookup_and_replace self lookup_table:Table key_columns:(Vector (Integer | Text | Regex) | Text | Integer | Regex) add_new_columns:Boolean=True allow_unmatched_rows:Boolean=True on_problems:Problem_Behavior=Problem_Behavior.Report_Warning =
Helpers.ensure_same_connection "table" [self, lookup_table] <|
Lookup_Query_Helper.build_lookup_query self lookup_table key_columns add_new_columns allow_unmatched_rows on_problems
merge : Table -> (Vector (Integer | Text | Regex) | Text | Integer | Regex) -> Boolean -> Boolean -> Problem_Behavior -> Table ! Missing_Input_Columns | Non_Unique_Key | Unmatched_Rows_In_Lookup
merge self lookup_table:Table key_columns:(Vector (Integer | Text | Regex) | Text | Integer | Regex) add_new_columns:Boolean=False allow_unmatched_rows:Boolean=True on_problems:Problem_Behavior=Problem_Behavior.Report_Warning =
_ = [lookup_table, key_columns, add_new_columns, allow_unmatched_rows, on_problems]
Error.throw (Unsupported_Database_Operation.Error "Table.merge is not implemented yet for the Database backends.")

## ALIAS join by row position
GROUP Standard.Base.Calculations
Expand Down Expand Up @@ -2595,12 +2596,12 @@ type Table
> Example
Replace dashes with underscores.

table.replace ["col0", "col1"] "-" "_"
table.text_replace ["col0", "col1"] "-" "_"

> Example
Remove leading and trailing spaces from cells.

table.replace ["col.*".to_regex] "^\s*(.*?)\s*$".to_regex "$1"
table.text_replace ["col.*".to_regex] "^\s*(.*?)\s*$".to_regex "$1"

> Example
Replace texts in quotes with parentheses.
Expand All @@ -2609,8 +2610,8 @@ type Table
@columns Widget_Helpers.make_column_name_vector_selector
@term Widget_Helpers.make_column_ref_or_text_value_selector
@new_text Widget_Helpers.make_column_ref_or_text_value_selector
replace : Vector (Integer | Text | Regex) | Text | Integer | Regex -> Text | Column | Column_Ref | Regex -> Text | Column | Column_Ref -> Case_Sensitivity -> Boolean -> Column
replace self columns (term : Text | Column | Column_Ref | Regex = "") (new_text : Text | Column | Column_Ref = "") case_sensitivity=Case_Sensitivity.Sensitive only_first=False =
text_replace : Vector (Integer | Text | Regex) | Text | Integer | Regex -> Text | Column | Column_Ref | Regex -> Text | Column | Column_Ref -> Case_Sensitivity -> Boolean -> Column
text_replace self columns (term : Text | Column | Column_Ref | Regex = "") (new_text : Text | Column | Column_Ref = "") case_sensitivity=Case_Sensitivity.Sensitive only_first=False =
table_ref = Table_Ref.from self
resolved_term = table_ref.resolve term
resolved_new_text = table_ref.resolve new_text
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ Table.select_into_database_table self connection (table_name : Text) primary_key
Table.update_rows : Database_Table | Table -> Update_Action -> Vector Text | Nothing -> Boolean -> Problem_Behavior -> Database_Table ! Table_Not_Found | Unmatched_Columns | Missing_Input_Columns | Column_Type_Mismatch | SQL_Error | Illegal_Argument
Table.update_rows self (source_table : Database_Table | Table) (update_action : Update_Action = Update_Action.Update_Or_Insert) (key_columns : Vector | Nothing = Nothing) (error_on_missing_columns : Boolean = False) (on_problems : Problem_Behavior = Problem_Behavior.Report_Warning) =
_ = [source_table, update_action, key_columns, error_on_missing_columns, on_problems]
Error.throw (Illegal_Argument.Error "Table.update_rows modifies the underlying table, so it is only supported for Database tables - in-memory tables are immutable. Consider using `join` or `lookup_and_replace` for a similar operation that creates a new Table instead.")
Error.throw (Illegal_Argument.Error "Table.update_rows modifies the underlying table, so it is only supported for Database tables - in-memory tables are immutable. Consider using `join` or `merge` for a similar operation that creates a new Table instead.")

## GROUP Standard.Base.Output
ICON data_output
Expand Down
17 changes: 9 additions & 8 deletions distribution/lib/Standard/Table/0.0.0-dev/src/Data/Table.enso
Original file line number Diff line number Diff line change
Expand Up @@ -1871,10 +1871,11 @@ type Table

## GROUP Standard.Base.Calculations
ICON join
Replaces values in this table by values from a lookup table.
Merges this table with a lookup table
New values are looked up in the lookup table based on the `key_columns`.
Columns found in the lookup table values are replaced by values from the
lookup. Columns not found are left unchanged.
Columns that exist in the lookup table where a match was found are
replaced by values from the lookup table. Columns not found are left
unchanged.
This operation is similar to `Table.update_rows`, but just returns a new
`Table` instance, instead of updating the table in-place (which is only
possible for Database tables).
Expand Down Expand Up @@ -1923,8 +1924,8 @@ type Table
- If `add_new_columns` is `False` and the lookup table has columns
that are not present in this table, an `Unexpected_Extra_Columns`.
@key_columns Widget_Helpers.make_column_name_vector_selector
lookup_and_replace : Table -> (Vector (Integer | Text | Regex) | Text | Integer | Regex) -> Boolean -> Boolean -> Problem_Behavior -> Table ! Missing_Input_Columns | Non_Unique_Key | Unmatched_Rows_In_Lookup
lookup_and_replace self lookup_table:Table key_columns:(Vector (Integer | Text | Regex) | Text | Integer | Regex) add_new_columns:Boolean=True allow_unmatched_rows:Boolean=True on_problems:Problem_Behavior=Problem_Behavior.Report_Warning =
merge : Table -> (Vector (Integer | Text | Regex) | Text | Integer | Regex) -> Boolean -> Boolean -> Problem_Behavior -> Table ! Missing_Input_Columns | Non_Unique_Key | Unmatched_Rows_In_Lookup
merge self lookup_table:Table key_columns:(Vector (Integer | Text | Regex) | Text | Integer | Regex) add_new_columns:Boolean=False allow_unmatched_rows:Boolean=True on_problems:Problem_Behavior=Problem_Behavior.Report_Warning =
lookup_columns = Lookup_Helpers.prepare_columns_for_lookup self lookup_table key_columns add_new_columns allow_unmatched_rows on_problems

java_descriptions = lookup_columns.map make_java_lookup_column_description
Expand Down Expand Up @@ -2588,7 +2589,7 @@ type Table
> Example
Replace dashes with underscores.

table.replace "-" "_"
table.text_replace "-" "_"

> Example
Remove leading and trailing spaces from cells.
Expand All @@ -2602,8 +2603,8 @@ type Table
@columns Widget_Helpers.make_column_name_vector_selector
@term Widget_Helpers.make_column_ref_or_text_value_selector
@new_text Widget_Helpers.make_column_ref_or_text_value_selector
replace : Vector (Integer | Text | Regex) | Text | Integer | Regex -> Text | Column | Column_Ref | Regex -> Text | Column | Column_Ref -> Case_Sensitivity -> Boolean -> Column
replace self columns (term : Text | Column | Column_Ref | Regex = "") (new_text : Text | Column | Column_Ref = "") case_sensitivity=Case_Sensitivity.Sensitive only_first=False =
text_replace : Vector (Integer | Text | Regex) | Text | Integer | Regex -> Text | Column | Column_Ref | Regex -> Text | Column | Column_Ref -> Case_Sensitivity -> Boolean -> Column
text_replace self columns (term : Text | Column | Column_Ref | Regex = "") (new_text : Text | Column | Column_Ref = "") case_sensitivity=Case_Sensitivity.Sensitive only_first=False =
table_ref = Table_Ref.from self
resolved_term = table_ref.resolve term
resolved_new_text = table_ref.resolve new_text
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ type Lookup_Column
It also reports any errors or warnings related to selecting these columns.
prepare_columns_for_lookup base_table lookup_table key_columns_selector add_new_columns allow_unmatched_rows on_problems =
key_columns = base_table.select_columns key_columns_selector . column_names . catch No_Output_Columns _->
Error.throw (Illegal_Argument.Error "At least one key column must be provided for `lookup_and_replace`.")
Error.throw (Illegal_Argument.Error "At least one key column must be provided for `merge`.")
lookup_table_key_columns = lookup_table.select_columns key_columns . catch Missing_Input_Columns error->
Error.throw (Missing_Input_Columns.Error error.criteria "the lookup table")

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -333,16 +333,16 @@ spec setup =
actual.at "col1" . to_vector . should_equal [1000, 200, 1000, 400, 500, 1000]
actual.column_names . should_equal ["col0", "col_between", "col1", "def"]

Test.group prefix+"Table.replace" <|
Test.group prefix+"Table.text_replace" <|
Test.specify "should allow to replace values in a table" <|
with_mixed_columns_if_supported [["col0", ["abc", "def", "ghi"]], ["col1", ["nabc", "ndef", "asdf"]]] t->
actual = t.replace ["col0", "col1"] "ab" "xy"
actual = t.text_replace ["col0", "col1"] "ab" "xy"
actual.at "col0" . to_vector . should_equal ["xyc", "def", "ghi"]
actual.at "col1" . to_vector . should_equal ["nxyc", "ndef", "asdf"]

Test.specify "should allow to replace values in a table with a regex" <|
t = table_builder [["col0", ["abc", "def", "ghi"]], ["col1", ["nabc", "ndef", "asdf"]]]
actual = t.replace ["col0", "col1"] "[bdi]".to_regex "xy"
actual = t.text_replace ["col0", "col1"] "[bdi]".to_regex "xy"
case actual.is_error && setup.is_database of
True ->
actual.should_fail_with Unsupported_Database_Operation
Expand All @@ -352,7 +352,7 @@ spec setup =

Test.specify "should allow to replace values in a table with a column" <|
t = table_builder [["col0", ["abc", "def", "ghi"]], ["col1", ["nabc", "ndef", "asdf"]], ["col2", ["xy", "yx", "zz"]]]
actual = t.replace ["col0", "col1"] "[bdi]".to_regex (t.at "col2")
actual = t.text_replace ["col0", "col1"] "[bdi]".to_regex (t.at "col2")
case actual.is_error && setup.is_database of
True ->
actual.should_fail_with Unsupported_Database_Operation
Expand All @@ -363,7 +363,7 @@ spec setup =

Test.specify "should allow to use Column_Ref in replace" <|
t = table_builder [["txt", ["abc", "def", "ghi"]], ["term", ["b", "d", "i"]], ["new", ["X", "Y", "Z"]]]
t1 = t.replace "txt" (Column_Ref.Name "term") (Column_Ref.Name "new")
t1 = t.text_replace "txt" (Column_Ref.Name "term") (Column_Ref.Name "new")
case t1.is_error && setup.is_database of
True ->
t1.should_fail_with Unsupported_Database_Operation
Expand Down
Loading
Loading