Extremely similar charter to Ibis

[wesm]

For about 3.5 years we have been developing a unified expression-based front end system for SQL systems as well as pandas: https://github.com/ibis-project/ibis. I suspect there are ripe opportunities for collaboration on this

cc @cpcloud @kszucs

[wesm]

Note that the documentation/READMEs and web properties for Ibis are in need of some revamping -- development over the last 2 years has been invested in non-Hadoop-related areas like BigQuery, Clickhouse, pandas, MapD, and other backend engines

[devin-petersohn]

Thanks @wesm, there are almost certainly opportunities for collaboration here. We have research interests that we would like to explore in this space and it may make sense to combine efforts on these goals if they align.

I'm generally familiar with ibis, though I have admittedly not used it myself. I am also not familiar with the internals of ibis, though I will start looking into it.

Primarily our interest with this project is eventually be able to automatically choose a compute engine for the user's workflow. Allowing for an elastic set of resources rather than always connecting to a static database is also an area of interest for us. We also have a set of systems-level research interests, and these are hard/impossible to integrate into existing technologies (scheduling, partitioning, elastic resource provisioning). Please keep in mind that these research goals are preliminary right now.

Hopefully this gives you a high level overview of what we're hoping to achieve with this project.

[devin-petersohn]

Closing this. Feel free to reopen if the discussion should continue.