Snowpark backend #1133

sfc-gh-lfallasavendano · 2024-01-08T23:21:31Z

Adds a new backend for generating Scala code that uses Snowpark .

Terms

THIS SOFTWARE IS CONTRIBUTED SUBJECT TO THE TERMS OF THE TERMS OF THE CCLA DATED 2017-11-07 WITH FINOS/LINUX FOUNDATION (FORMERLY THE SYMPHONY SOFTWARE FOUNDATION CCLA).

THIS SOFTWARE IS LICENSED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE AND ANY WARRANTY OF NON-INFRINGEMENT, ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. THIS SOFTWARE MAY BE REDISTRIBUTED TO OTHERS ONLY BY EFFECTIVELY USING THIS OR ANOTHER EQUIVALENT DISCLAIMER IN ADDITION TO ANY OTHER REQUIRED LICENSE TERMS.

Generate empty files for ELM modules in the IR

* Add initial implementation of a 'mapping' context This context element has information about type definitions. * Code review suggestions

Add support to the literals using the snowpark function lit for boolean, string, char, float and Int. Add test for lit literals.

* Adds basic generation of record wrappers * Fix missing `extends` generation

Considers expressions like: a.b a AConstructor Close finos#23 and close finos#24

Also adds local variables for records representing tables. Related to finos#20

Support List.member function into Snowpark using the "in" function

…s#48) * Add support for Snowpark generation of basic PatternMatch cases The following cases are supported: - Cases with literal values - Cases with constructors of union types without parameters

* Add support for some list functions Adds basic support for: - `List.map` with lambda returning record - `List.map` with lambda returning a single value - `List.filter` with lambda predicate - `List.sum` with `List.map` as its collection * Fixes for Code review comments

Generate a basic support for the add, subtract, multiply and divide operators in Snowpark

Support the comparison operators for snowpark ===, =!=, <, >, <=, >=

* Add support for a couple of elements - Adds basic support for case/of with Union types with params ``` case x of First a b c -> a Second y -> y ``` - Adds support for Maybe case/of patterns ``` case x of Just w -> w Nothing -> 19 ``` - Adds basic support for conversion of `Maybe.Just` and `Maybe.Nothing` - Adds a new Scala literal for `null` * Code review changes

Add support for the and (&&) or (||) and not (!) operators in snowpark.

Add support for the floor and module functions in Snowpark

* Add support `let` and `Maybe` functions Adds support for: - Basic `let` without function definitions - `Maybe.map` - `Maybe.withDefault` * Fix Nothing comparisons Also fixes tagging of fields with aliases.

Add support to access field of a record in the way .field

- Fix creation of union type instance with parameters - Fix generation of parameter names with Scala reserved words - Fix calls to use curry-like syntax in scala

* Add generation of wrappers for keeping record references Now we don't remove parameters representing records from functions . This fixes problems where the a column may be lead to ambiguities. * Code review suggestions

Changes to allow the conversion of must of the subset for Unsecured.elm. Adds basic unpacking of JSON results. Fixes finos#75

Adds support for converting tuple creation: let t = (1,2,3) And matching: case t of (a,b,c) -> c

…inos#88) * Add support for the groupBy and aggregate from Morphir.SDK Add support to the pattern of groupBy and Aggregate from the Moprhir.SDK in Snowpark * Support multiple variables in the lambda's call for the SDK Aggregate function Support multiple variable in the lambda's call for the SDK aggregate function * Support the alias function from the column in order to generate a custom column name * Refactoring Aggregate mapping * Fix alias generation and refactoring constants and map functions file

Fix alias name when is added in the aggregate function for the column

)

Fix passing lambda functions as arguments.

* Changes to improve compilation for `Calculations.elm` Adds support for: - Different set of function mappings when converting a function identified as Scala-only - Convert empty DataFrame (still pending issue with session object) - Adds several simple mappings for `Calculations.elm`: `List.maximum` , `max`, `min` * Code review fixes and fix List.sum with null values

* Add support for the Aggregate when returns a record Support the return type as a record in the Aggregate function. * Support records and handle error asign in the record field

* First commit for decorations * Add support for `inlining`

First support for the join feature in elm. Mapping the innerJoin function from Elm

* Add issue generation report - Generates `GenerationReport.elm` - Adds propagation of issue list - Changes uses of `Value` for `TypedValue` * Code review suggestions

* In-progress documentation and fixes/coverage * Add minor updates * Add missing elements * Add latest changes * Split initial documents * Code review comments

- Fixes some pattern matching scenarios - Adds pattern matching test - Adds function mapping tests - Adds support for some function scenarios

Adds command line options for the `Snowpark` backend to the CLI2 project.

- Add Test for the join feature.

* Adding and fixing documentation for Snowpark backend

- Remove unused code - Fix Boolean expression generation in pattern matching code

* Add documentation for the join feature * Fixing documentation

Fixes: - `List.filter` with a local function reference - Binary operation generated as part of a projection

* Code review suggestion Changes string literal to "throwing an exception" when processing a not supported element. * Code review suggestion Add support for aggregate errors

src/Morphir/Scala/AST.elm

sfc-gh-aramirezfuentes and others added 30 commits January 8, 2024 16:18

Generate a new backend for Snowpark (finos#33)

f886f15

Generate empty files for ELM modules in the IR

Generate empty functions with a to do literal (finos#36)

5b2d4fe

Add initial implementation of a 'mapping' context (finos#38)

6a7243b

* Add initial implementation of a 'mapping' context This context element has information about type definitions. * Code review suggestions

Support the literals into Snowpark (finos#39)

55598ea

Add support to the literals using the snowpark function lit for boolean, string, char, float and Int. Add test for lit literals.

Adds basic generation of record wrappers (finos#40)

58ec90e

* Adds basic generation of record wrappers * Fix missing `extends` generation

Add conversion for identifier expressions (finos#41)

7881a59

Considers expressions like: a.b a AConstructor Close finos#23 and close finos#24

Add conversion for function parameters and return type (finos#42)

dda3620

Also adds local variables for records representing tables. Related to finos#20

Support member list function in Snowpark (finos#46)

aacb84e

Support List.member function into Snowpark using the "in" function

Add support for Snowpark generation of basic PatternMatch cases (fino…

55e13e1

…s#48) * Add support for Snowpark generation of basic PatternMatch cases The following cases are supported: - Cases with literal values - Cases with constructors of union types without parameters

Support basic arithmetic operators for Snowpark (finos#56)

ef3d8b6

Generate a basic support for the add, subtract, multiply and divide operators in Snowpark

Support the comparison operators for Snowpark (finos#59)

cbfd482

Support the comparison operators for snowpark ===, =!=, <, >, <=, >=

Support basic logical operators (finos#68)

77105f2

Add support for the and (&&) or (||) and not (!) operators in snowpark.

Add Support for floor and module functions (finos#72)

e76bf3b

Add support for the floor and module functions in Snowpark

Add support let and Maybe functions (finos#69)

3206866

* Add support `let` and `Maybe` functions Adds support for: - Basic `let` without function definitions - `Maybe.map` - `Maybe.withDefault` * Fix Nothing comparisons Also fixes tagging of fields with aliases.

Add support to access field of a record (finos#74)

31140fd

Add support to access field of a record in the way .field

Misc fixes for compiling LCR assets reduced (finos#76)

3d62b91

- Fix creation of union type instance with parameters - Fix generation of parameter names with Scala reserved words - Fix calls to use curry-like syntax in scala

Small changes LCR's Unsecured (finos#83)

a2abc15

Changes to allow the conversion of must of the subset for Unsecured.elm. Adds basic unpacking of JSON results. Fixes finos#75

Add tuple creation and matching (finos#84)

c732ffd

Adds support for converting tuple creation: let t = (1,2,3) And matching: case t of (a,b,c) -> c

Fix in the aggregate function with alias (finos#90)

03efda3

Fix alias name when is added in the aggregate function for the column

Add initial support for concat and concatMap (finos#89)

74f5ae5

Change use of intermediate JSON objects to intermediate arrays (finos#91

74c71e2

)

Fix lambda functions passed as arguments (finos#92)

0d2e165

Fix passing lambda functions as arguments.

Sf/add filtermap cases (finos#93)

84c5294

Support Aggregate function that returns a Record (finos#98)

00d7444

* Add support for the Aggregate when returns a record Support the return type as a record in the Aggregate function. * Support records and handle error asign in the record field

Add decorations processing (finos#99)

25f87a3

* First commit for decorations * Add support for `inlining`

sfc-gh-lfallasavendano and others added 18 commits January 8, 2024 16:19

Code review suggestions (finos#100)

8e709dd

Basic implementation for the innerjoin function (finos#101)

0c74626

First support for the join feature in elm. Mapping the innerJoin function from Elm

Add issue generation report (finos#102)

0705aee

* Add issue generation report - Generates `GenerationReport.elm` - Adds propagation of issue list - Changes uses of `Value` for `TypedValue` * Code review suggestions

Add Antiques example mappings and minor fixes (finos#104)

b7bb967

Sf/basicdocumentation (finos#106)

6db15bf

* In-progress documentation and fixes/coverage * Add minor updates * Add missing elements * Add latest changes * Split initial documents * Code review comments

Add unit test coverage and some bug fixes (finos#107)

7093d1f

- Fixes some pattern matching scenarios - Adds pattern matching test - Adds function mapping tests - Adds support for some function scenarios

Add documentations, tests and a small example (finos#109)

f98f826

Add Snowpark backend support to CLI2 (#1)

4d02e60

Adds command line options for the `Snowpark` backend to the CLI2 project.

- Add support to the multiple Join. (#2)

40eec96

- Add Test for the join feature.

Sf/docreview (#3)

3b02ca5

* Adding and fixing documentation for Snowpark backend

Fix boolean expression generation in pattern matching code

1c9f205

- Remove unused code - Fix Boolean expression generation in pattern matching code

Add documentation for the join feature (#5)

77afb1a

* Add documentation for the join feature * Fixing documentation

Avoid generating duplicated imports (#6)

921b43c

Add test for Aggregate and Values Operators (#7)

b8a701f

Some improvements for the inline decoration (#8)

e9bb030

Apply the elm-formatter in the Snowpark files (#9)

054fd06

Fix a couple of problems (#10)

8c61b78

Fixes: - `List.filter` with a local function reference - Binary operation generated as part of a projection

Code review suggestion (#11)

4689058

* Code review suggestion Changes string literal to "throwing an exception" when processing a not supported element. * Code review suggestion Add support for aggregate errors

DamianReeves mentioned this pull request Jan 11, 2024

2024-01-11 (11 Jan 2024) - Morphir Project Meeting finos/morphir#225

Open

17 tasks

stephengoldbaum reviewed Jan 22, 2024

View reviewed changes

src/Morphir/Scala/AST.elm Show resolved Hide resolved

stephengoldbaum approved these changes Jan 23, 2024

View reviewed changes

stephengoldbaum merged commit a697113 into finos:main Jan 23, 2024
4 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Snowpark backend #1133

Snowpark backend #1133

sfc-gh-lfallasavendano commented Jan 8, 2024

Snowpark backend #1133

Snowpark backend #1133

Conversation

sfc-gh-lfallasavendano commented Jan 8, 2024

Terms