Performance Issues for streams with many messages

[JulianFeinauer]

Currently, we observe a performance problem when processing many messages in a Stream with man evaluation functions or with evaluation functions that "look" at many signals.
I already did some analysis in the branch "performance-test".

The class TestCrunchPerformanceNT contains the main to run the tests (and also a setup to run it with JMH with the IntelliJ Plugin).

The file referenced there is 30MB large and thus not in the repo but is shared in the crunch-oss channel in Slack (or downloaded here: https://pragmaticminds.slack.com/files/U1Q8RMFDL/FFEAR60TS/sample2.txt)

[ottobackwards]

any chance you can host this outside your company slack?

[JulianFeinauer]

Hi @ottobackwards... This is indeed a valuable concern which we'd like to address. A good and publicly available infra is one of the reasons why we'd like to move to asf :)

[ottobackwards]

Well, I wanted to run the test, but I'm not sure I want to join your slack

[JulianFeinauer]

Yeah I agree. And we already have a fix now but it's not really rolled out because we had other issues we focused on... Von meinem Mobiltelefon gesendet

…

-------- Ursprüngliche Nachricht -------- Betreff: Re: [pragmaticminds/crunch] Performance Issues for streams with many messages (#10) Von: Otto Fowler An: pragmaticminds/crunch Cc: Julian Feinauer ,Author Well, I wanted to run the test, but I'm not sure I want to join your slack — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#10?email_source=notifications&email_token=AECOQCQ7J7P2E56GINA3Q2LQGFMRXA5CNFSM4GQ5QJW2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5CDAAA#issuecomment-524562432>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AECOQCTORCUHVOZACCTFQA3QGFMRXANCNFSM4GQ5QJWQ>.