Parallelism granularity

[raulcf]

Right now is table to avoid redundant reads.
By splitting data on memory we could provide more fine-granular parallelism, i.e. per column, while still avoiding redundant reads.

[raulcf]

Some evidence: in data.gov 30/10K files contain ~50% data

[sgt101]

Interesting stat.... What does it mean? Amount of data in mbs? Or number
of rows?

We could find these stats for BT's warehouses if that helps?

Simon
On Wednesday, October 26, 2016, Raul notifications@github.com wrote:

Some evidence: in data.gov 30/10K files contain ~50% data

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.<
https://ci3.googleusercontent.com/proxy/3KYElAXuMB7ZvhVBaiLCSFqX09K_1FicMFlo_nJBIe1Szg41zZIW_wI8zXpX3SYuM-KbQPiV6Qww_B3uNG9I1FVDK852yCmrgoesYeMImohyrG-FyiqFPx9tMoo4xrb71SZcS75wYXjFQ1uQvigwM9kP28rb-Q=s0-d-e1-ft#https://github.com/notifications/beacon/AATC_qy74A2hmIBU77xmvpCGfVs0kY50ks5q30zRgaJpZM4JQwVL.gif>

[mansoure]

This is very interesting problem. @raulcf how far are you with this issue? I think I can help here.