Repository navigation
Low performance of C++ multi-channel streaming and unary scenarios on nightly benchmarks #14899
Description
Please answer these questions before submitting your issue.
Should this be an issue in the gRPC issue tracker?
Yes
What version of gRPC and what language are you using?
c++, grpc 1.10
What operating system (Linux, Windows, …) and version?
Linux CentOS 7.2, kernel 3.10
What runtime / compiler are you using (e.g. python version or version of gcc)
gcc 7.2
What did you do?
Perform insecure unary throughput qps test using grpc's c++ qps driver
What did you expect to see?
Reasonable qps result.
What did you see instead?
According to the grpc performance test page for latest release version (https://performance-dot-grpc-testing.appspot.com/explore?dashboard=5636470266134528), c++ unary and stream performance is much poorer than Java's (less than half of java's).
This was not the case when I last checked this page for grpc 1.6. C++ qps was twice as much as Java's back then, now it's less than 1/2. What's with this drastic drop in throughout performance?
I also conducted the unary qps test myself with grpc's qps driver. The default poll engine is now epoll1 for me, and I observe heavy load on a single thread named "gpr_executor", while other threads are relatively idle. The qps result is only 250k on two machines with 64 logic cores.
Activity
disturbing commentedon Apr 12, 2018
Agreed - I just noticed this as well...
ykameshrao commentedon Sep 24, 2018
Whats the status on this?
jtattermusch commentedon Oct 31, 2018
CC @jtattermusch
moiyer commentedon Nov 11, 2018
Have the same question. Why on gRPC official benchmark, c++ performance is much pooler than Java?
13 remaining items
smahapatra1 commentedon Sep 21, 2019
Can you please elaborate? Did you not find a regression on network communication? Can we be confident that C++ async program will perform well?
jtattermusch commentedon Sep 23, 2019
The reason why we think it's a problem of the test setup and not an actual gRPC C++ performance regression is that we have a set of internal benchmarks that perform much better than the ones available externally and the internal benchmarks the problem described in this issue. (Unfortunately we cannot publish the internal numbers because details of the internal google infrastructure are confidential). We're still investigating the root cause of the unexpectedly low test results in some C++ scenarios in the external benchmarks suite - it's quite complicated. We'll update this issue once we find something.
stale commentedon May 6, 2020
This issue/PR has been automatically marked as stale because it has not had any update (including commits, comments, labels, milestones, etc) for 30 days. It will be closed automatically if no further update occurs in 7 day. Thank you for your contributions!
xunliu commentedon Apr 26, 2021
this is still a problem - let's leave open.