Performance comparison

[amir-hadi]

Hi guys,

I found this library because we are experimenting with gRPC in our company. I love Python, but I am quite new to the asyncio world of it. I executed the helloworld proto example with grpclib as the only library supporting asyncio and I was wondering about the performance of it. We are getting 2500 req/s using this library.

With NodeJS we are getting 17000 req/s on our developer machines.

This shall not end in a comparison of python vs node or anything faster, I really want to stick with Python (and use FastAPI + Starlette as framework), but wanted to checkin first, if we are doing something wrong with this library.

Thanks in advance,
Amir

[vmagamedov]

I don't know why Nodejs implementation is so much faster, I compared only grpclib and official grpcio project. Maybe Nodejs implementation consumes all available CPU cores, when grpclib/grpcio uses only single CPU core.

grpcio is a C-based library and it is at least 2x faster than grpclib (on CPython). But grpclib on PyPy is sensibly faster than grpcio on CPython.

grpclib is based on h2 library and most of the CPU time is spent inside h2 code. So in order to speed-up grpclib we need to speedup h2 library.

And finally, my personal goal is to make it possible to return response in a sub-millisecond time, it is already possible with grpclib. Slowest part of a service is not a server or framework, slowest part – is a user's code – business logic, and it doesn't matter how fast your server can serve helloworld responses. This is just my opinion. But I would like to make grpclib faster anyway.

[amir-hadi]

Please don't get me wrong, I am super happy that this project exists. I don't understand why the grpc project is not using asyncio as well (or at least provide an option for it).

I just wanted to make sure that we are not using it wrong. But I didn't see that you have your own benchmarks already and I know understand more.

I checked some more benchmarks from the gRPC people and it's always slowest in Python. Of course this doesn't tell us much, since the application logic is usually slow (waiting for database etc). In my case though, I am just listening on a socket and whatever comes in needs to be forwarded (without much processing), so the faster the better.

Thanks for giving me this info! Very helpful.

Best,
Amir

[vmagamedov]

Seams like @grpc folks already started to work on asyncio support:

• gRPC Async API gRFC grpc/grpc#19494
• asyncio Implementation Design Doc grpc/grpc#19495
• [Aio] Server Implementation grpc/grpc#19496
• [Aio] Client Implementation grpc/grpc#19497

Maybe you need some sort of a proxy server? Something like Envoy or Nginx? gRPC is based on HTTP/2 and any HTTP/2-capable server (there are tons of them) can be used for this (routing, filtering, auth, retries, ...).

[erwinkinn]

I don't know why Nodejs implementation is so much faster, I compared only grpclib and official grpcio project. Maybe Nodejs implementation consumes all available CPU cores, when grpclib/grpcio uses only single CPU core.

grpcio is a C-based library and it is at least 2x faster than grpclib (on CPython). But grpclib on PyPy is sensibly faster than grpcio on CPython.

grpclib is based on h2 library and most of the CPU time is spent inside h2 code. So in order to speed-up grpclib we need to speedup h2 library.

And finally, my personal goal is to make it possible to return response in a sub-millisecond time, it is already possible with grpclib. Slowest part of a service is not a server or framework, slowest part – is a user's code – business logic, and it doesn't matter how fast your server can serve helloworld responses. This is just my opinion. But I would like to make grpclib faster anyway.

@vmagamedov, since you've researched grpcio, could you please tell me if the asynchronous part of it is done correctly or is it just an asynchronous wrapper for blocking calls implemented in C? I just need to choose a gRPC framework for high reliable production code. Once upon a time I've experienced the disappointment with python official async framework for Azure Servicebus. They've done just async wrapper for the existed libraries where every blocking call is done in thread pool executor. From outside they don't even use a future object -- they just poll for the results. On the other hand grpcio is mostly implemented in cython and unfortunately I'm not skilled enough in it.

[vmagamedov]

@vagarkav There are no blocking sync calls with a thread-pool under the hood, so their asyncio support is as good as possible. But it would be great to understand how grpcio works. They have a C-core library (common for many languages) and all IO-related code is implemented in C, grpcio is a bridge between C-based library with it's own event loop running in it's own thread and asyncio event loop running in the main thread. So you definitely can call grpcio a wrapper or bindings to that C-core library.

grpcio has much better support than grpclib, seems that many people work full-time to support that project. grpcio also more widely used, with contributors from big tech companies. grpcio is also more feature-full. So using grpcio is a safe default, use grpclib only if you have some specific requirements and you understand all the trade-offs.

[erwinkinn]

@vmagamedov, thank you very much, you saved me a week! It's interesting that while using threads grpcio code is still 2x faster than pure acyncio based grpclib.
Are these Cython's advantages really so great that they overcome the poor performance of the GIL? Or it's just due to h2 low performance?

[llucax]

In case anyone else bumps into this issue too and wonders how the performance of grpclib is compared to grpcio nowadays, at the time of writing, and according to my tests, grpclib is 2 times faster than grpcio in CPython. Here are the benchmarks I crafted to test this: https://github.com/llucax/python-grpc-benchmark.