Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some bench improvements and greetings from the Tremor team :) #10

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

Licenser
Copy link

@Licenser Licenser commented May 10, 2021

Hi Alan,

I came over your blog post on optimizing away 700 CPUs with rust, it's a really good read and was fascinating to see since our team at Wayfair ran into exactly the same issue and came up with nearly the same solution :D.

I went through the benchmarks today and noticed you're selling yourself short, the node.js version did only half the logic that the rust version did (it ignored newlines) I twiddled something together to make them more the same on the logical front.

I also changed the benchmark script to emit some filtered metrics (to see the impact on that), I hope you approve of that change.

With the updated results I ran your benchmark through tremor and notices something interesting, your code is quicker for single-core use but not quite as fast when it comes to multi-core setups. (I'll put my local benchmark below ran with taskset -c 1 and taskset -c 1,2,3 for single and 3c respectively).

We donated tremor to the CNCF last year, so I'm curious if you're open to chatting, sharing some experiences, and possibly expanding the use case - neither your nor our employer is in the game of writing this kind of software as part of their core business so we're probably both very produced focused which seems to align well :). If you're interested I'll leave you a link to our community server.


nodejs nodejs (3c) rust_pxy rust_pxy (3c) tremor tremor (3c)
Median(us) 110 153 78 137 97 94
P95(us) 125 183 103 150 112 137

PS: we found a bug in our stated decoder when trying this :D thank you!

Signed-off-by: Heinz N. Gies <heinz@licenser.net>
Signed-off-by: Heinz N. Gies <heinz@licenser.net>
Signed-off-by: Heinz N. Gies <heinz@licenser.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant