[BEAM-160] NexMark

[mshields822]

The NexMark integration suit, ported to Beam and generalized to support multiple runners.

[aljoscha]

Wow, that's a big one. Could you give some guidance as to how one should look at this: Where are the starting points, how are benchmarks invoked, what can be the results and what do they mean, how can it be run on different runners, is it for streaming/batch only?

Maybe this is all coming in the README and I'm just to eager ... 😅

[mshields822]

Added README.md to the laundry list :-)

[iemejia]

Excellent work Mark, I am also waiting for the README to see how to run this monster, I discussed briefly with Davor about this benchmark at ApacheCon, and we expect to test it in our local company cluster with all possible runners. In addition I can help you if you need to do something extra to run this over spark too (well the cases we can support with spark streaming).

[iemejia]

Oh and one additional question, the queries 9-12 where do they came from ? I was just checking the website/publication but they are not there (or do I have an old ref).
http://datalab.cs.pdx.edu/niagara/NEXMark/
http://datalab.cs.pdx.edu/niagara/pstream/nexmark.pdf

[mshields822]

Well spotted. We've added some additional 'queries' to fill in some gaps,
for example session windows, interaction with external systems which makes
'closing' windows very expensive, working purely in processing time.
They've generally been inspired by various customer scenarios we've
encountered here at Google.

I'll be getting back to the README and adding an InProcessDriver once we've
gotten our current release out the door. Adding a SparkDriver should be a
5min exercise.

I should mention my recipe for bringing this up on a flink-on-google-cloud
cluster is very ugly so I'll be asking for feedback on where I've taken the
long way unnecessarily.

Thx for your patience,
-m

On Wed, May 25, 2016 at 6:32 AM, Ismael Mejia notifications@github.com
wrote:

Oh and one additional question, the queries >9 where do they came from ? I
was just checking the website/paper but they are not there (or do I have an
old ref).
http://datalab.cs.pdx.edu/niagara/NEXMark/
http://datalab.cs.pdx.edu/niagara/pstream/nexmark.pdf

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
#366 (comment)

[mshields822]

Is the mvn failure me? Very weird.

[mshields822]

R: @davorbonaci
Parking with you Davor.
I will leave this as is on my account.

[mxm]

Great work @mshields822! This looks like it is almost ready to be merged. I think I found the culprit that currently prevents the Flink Runner from completing the benchmark. I'd like to try this out on GCE and give you feedback within the next week.

[iemejia]

Hello this one is a really interesting PR in particular to compare runners, I suppose it was closed because it was inactive for a long time, but I think we should revive it, in particular now that runners have matured. Any other person interested ?
[I still have in my personal TODO list to test this with the spark runner]

[mxm]

This is resolved.

[mxm]

I'd still be interested as well to help solving any remaining problems. Since this uses the Beam API and doesn't integrate with the core, it should still be usable. It just needs some love :)

[davorbonaci]

It would be great to have this! However, I think the amount of work needed is quite significant. If there's anybody who could take this over, I'd love to assist with that.

[mxm]

I assumed this was already at a pretty mature state (mature enough to merge it and keep improving). Perhaps we can find someone from the community to take this over?