Add Apache parquet as an output format for enriched data

[christoph-buente]

In order to use S3 as a queryable data lake, it would be beneficial to store the enriched data in a columnar data format like Apache Parquet [1]. We did some performance tests with Athena, and it seems to perform best for parquet, as opposed to TSV.

Thanks!

[1] https://parquet.apache.org/

[darrenhaken]

I'm interested in picking this up, would a PR get merged for this work?

[BenFradet]

yup sure 👍

[yoelb]

Any update? @darrenhaken are you working on it?

[darrenhaken]

I’ve been away on leave, I can take a look when I’m back in a few weeks. Feel free to pick it up if you want it sooner though!

…

On Fri, 19 Oct 2018 at 17:04, Yoel Benharrous ***@***.***> wrote: Any update? @darrenhaken <https://github.com/darrenhaken> are you working on it? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#87 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA6Me5FQJeKQes8xFMpAhTQ3nq61ocLZks5umYdzgaJpZM4OFQHw> .

[ollikaven]

@darrenhaken any updates?

[sumit-saurabh]

Any update here?

[itgd-techsupport]

is there any progress on this