Some bench improvements and greetings from the Tremor team :) #10
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hi Alan,
I came over your blog post on optimizing away 700 CPUs with rust, it's a really good read and was fascinating to see since our team at Wayfair ran into exactly the same issue and came up with nearly the same solution :D.
I went through the benchmarks today and noticed you're selling yourself short, the node.js version did only half the logic that the rust version did (it ignored newlines) I twiddled something together to make them more the same on the logical front.
I also changed the benchmark script to emit some filtered metrics (to see the impact on that), I hope you approve of that change.
With the updated results I ran your benchmark through tremor and notices something interesting, your code is quicker for single-core use but not quite as fast when it comes to multi-core setups. (I'll put my local benchmark below ran with
taskset -c 1
andtaskset -c 1,2,3
for single and 3c respectively).We donated tremor to the CNCF last year, so I'm curious if you're open to chatting, sharing some experiences, and possibly expanding the use case - neither your nor our employer is in the game of writing this kind of software as part of their core business so we're probably both very produced focused which seems to align well :). If you're interested I'll leave you a link to our community server.
PS: we found a bug in our stated decoder when trying this :D thank you!