Better LINQ Integration

[EntilZha]

Creating to discuss potential better LINQ integration for 0.6.0. From the milestone summery:

Another possible focus is on LINQ. This could take the form of implementing a limited SQL parser and optimizer using pyparsing. This might also be giving select, where, and related methods more definition. For example, if the LINQ functions are used using calls like select("atr").filter("atr == 1") be smarter about how they are executed. This is a wide open door, looking for thoughts and suggestions on what is of value. The basic concept is to start working on smarter ways of reading data, although this might tread into the territory of much more mature libraries like pandas its dataframes.

[EntilZha]

After thinking about this more, this seems like it might be worth doing. It looks like the benefits could be:

• Some brevity offered by SQL
• Further differentiate capabilities of the package from other packages
• It is an interesting problem and maybe it comes useful down the line for running on structured data.

I have been experimenting with pyparsing as I have time here and there. Code is in an ipython notebook gist with most of a SQL parser implementation complete.

Next steps will be:

• Do more testing to make sure the parser works correctly
• See if the SQL subset currently there is worth cutting down further
• Implement a parsed SQL to ScalaFunctional transformations compiler (this is the hardest one and I need to read more to find what is the best way to do this).
• Separate out different expressions so they can be used independently to parse some of the LINQ functions when the input is a string (query).

[EntilZha]

Closing this since it seems far off. I like the idea, but this will require writing a SQL parser/compiler which seems beyond the scope of just this project. If this happens it would be in a separate package and PyFunctional would use it as a library.