socketify.py Ideas & Updates! #10
Replies: 33 comments 55 replies
-
Preliminary Performance ComparisonTests made using ./http_load_test 40 localhost 3000 from uSockets project OS: Debian GNU/Linux bookworm/sid x86_64 CPU: Intel i7-7700HQ (8) @ 3.800GHz Memory: 32066MiB Average Requests per Second
Relative Performance vs C++
All plaintext tests following the TechEmpower rules for plaintext, with Date header updated each 1 second Average Requests per Second using JSON
JSON Relative Performance vs C++ Plain Text
Packages:Ruby using Oj package (at least 20% faster than json package) Python3 using orjson package (about 10% faster than ujson in this test, zzzjson, json are slower than ujson in python3) PyPy3 using json (2 to 3x faster than ujson, orjson is not supported in PyPy) Node.js using JSON.stringify All JSON tests following the TechEmpower rules for application/json, with Date header updated each 1 second |
Beta Was this translation helpful? Give feedback.
-
|
Added JSON tests with some packages to bring more performance comparisons. Fixed 4 processes Avg for node.js. asyncio is really slow, uvloop is not working in PyPy3, i am work in an CFFI solution for asyncio.
|
Beta Was this translation helpful? Give feedback.
-
|
First working pip package created + Docker images working ❤️ EDIT: in the future the install experience will be better |
Beta Was this translation helpful? Give feedback.
-
|
@AmirHmZz was i promised, here some techempower to you compare, is not an valid benchmark in my view because its too short to GC collect something, payload size is too small and other problems, but here are the numbers, i change some code too to updated Ruby and Python version and tried to add some pypy benchs too. I will write an article about benchmarking and give an deep dive in all this tests and more.
|
Beta Was this translation helpful? Give feedback.
-
|
The only thing that made me surprised is that |
Beta Was this translation helpful? Give feedback.
-
|
@AmirHmZz aspnetcore is really a monster, i rewrite the benchmark to an code that i use here in my company and the results are basically the same. My machine details: |
Beta Was this translation helpful? Give feedback.
-
|
@AmirHmZz my performance target are at least close to GoLang Fiber, likely the bottlenecks in an WebApps will be in another place not in the webserver framework. In JSON PyPy3 performs better than a version I wrote in C# but it's a matter of using another JSON framework or strategy like caching. An bottleneck much more plausible are IO (network, disk, bandwidth etc) and other frameworks used for JSON, database etc. Arquiteture and Scaling are much more important than get more than 1.8 millions of requests in this short test, with such an unrealistic payload. |
Beta Was this translation helpful? Give feedback.
-
|
I'd like to add to this table aiohttp and fastapi and specify versions for each framework/lib, Python 3.8 is more like a language version for me than a framework/library version |
Beta Was this translation helpful? Give feedback.
-
|
@eirnym this are results using TechEmPower benchmarks you can find more info here https://www.techempower.com/benchmarks/#section=data-r20, I will add more benchmarks when the first initial release happens, this is just preliminary results |
Beta Was this translation helpful? Give feedback.
-
|
Added a lot of examples in https://github.com/cirospaciari/socketify.py/tree/main/tests/examples updated Docker images in: PS: python3-alpine is better than python3 (2x faster in average) due to be more stable in 1% and 5% response times, but 97.5% is the same Examples of fetching are using aiohttp, but i will change to a CFFI solution if i find something faster and well maintained HTTPS still not working but i will add boringssl version of uSockets to fix this with priority |
Beta Was this translation helpful? Give feedback.
-
|
Added a lot of utilities features for post/upload, still missing try_end and for_each_header, but cookies was added as an extra https://github.com/cirospaciari/socketify.py/blob/main/tests/examples/upload_or_post.py Next step is streaming files, and related helpers, after this fix SSL build and add WS methods :D |
Beta Was this translation helpful? Give feedback.
-
|
Development updates: Loop bugs fixed, cork is working flawless now in CPython and PyPy (an definitive solution will come using HPy) async def home(res, req):
res.write_header("Content-Type", "audio/mpeg")
filename = "./file_example_MP3_5MG.mp3"
total = os.stat(filename).st_size
async with aiofiles.open(filename, "rb") as fd:
while not res.aborted:
buffer = await fd.read(16384) #16kb buffer
(ok, done) = await res.send_chunk(buffer, total)
if not ok or done: #if cannot send probably aborted
break I will add in samples how to stream files and an helper to static files that uses Range header to send partial content too. We are close to the first milestone and we will end MVP this year. |
Beta Was this translation helpful? Give feedback.
-
|
Some static files performance data from my personal machine (Debian 12/testing, i7-7700HQ, 32GB RAM, Samsung 970 PRO NVME) using oha -c 400 -z 5s http://localhost:3000/
Conclusions: Anyway we really recommends using NGINX or similar + CDN for production like everybody else |
Beta Was this translation helpful? Give feedback.
-
|
Finally we have windows x64, macos arm and x64 and linux x64, with pre built binaries using github actions, in the future windows on arm and linux arm will be supported too. |
Beta Was this translation helpful? Give feedback.
-
|
Still get the same error after |
Beta Was this translation helpful? Give feedback.
-
I think is some build problem i created an Issue, and tomorrow will be testing on an mac ARM i have one M1 at work. |
Beta Was this translation helpful? Give feedback.
-
Don't worry I will figure it out the problems until there you can use the docker examples in src/examples/docker sorry for the inconveniences we will make this better together |
Beta Was this translation helpful? Give feedback.
-
|
@seanr3 can you test again? I Think the problem is solved now, if work for you i will close the issue #34 if not i will do more investigation and fixes |
Beta Was this translation helpful? Give feedback.
-
|
Still errors on M1 #34 added link to similar issues |
Beta Was this translation helpful? Give feedback.
-
|
Without proper asyncio integration (fully integrated like uvloop) we got an performance hit on PyPy3, my last test i got 770k req/s in plaintext vs 1,293k req/s without asyncio run_once solution, is still ok for an first solution but will be not the last solution, i will fully integrate asyncio witth libuv, file IO included, because this performance hit is not acceptable in the long run. #18 I will do some improvements to bring performance up ASAP |
Beta Was this translation helpful? Give feedback.
-
|
Removing asyncio workarounds we get a performance boost from 770k to 880k req/s still much lower than 1.3 million req/s, i will do some performance checks in older version but at first seems that having a bigger Response and Request object with more functions is impacting performance (but we need all this features), so we will integrate all features first, measure the impact again, integrate your own aiofiles and fetch API with libuv and after this i will start an "uvloop" like project to fully integrate asyncio with libuv, removing the workarounds. Giving it at least 12% to 30% more raw performance. Adding Http3 will improve much more performance than replacing the workaround right now, and its way less effort for some one like me that is working only a few hours per week in this project (until i get support). Migration to HPy in future will bring Python3 performance up a lot. |
Beta Was this translation helpful? Give feedback.
-
|
Finally support for websockets is done and server names now examples and docs are in progress, soon the first release is coming |
Beta Was this translation helpful? Give feedback.
-
|
@AmirHmZz @eirnym Included benchmarks with falcon, uvicorn and robyn, did not include vibora or japronto because there are not active projects. in Round 22 robyn, vibora and socketify.py will be featured. This week end i will add docs and release PyPI the first true release v0.1.0 |
Beta Was this translation helpful? Give feedback.
-
|
Really nice work! Does/will it play nicely with database calls like with |
Beta Was this translation helpful? Give feedback.
-
I'm working to integrate into HPy and deeper, and will garantee http performance (fetch API), files performance (async file aio), event loop performance (rewrite of uv loop), and next PostgreSQL, SQLite3 and Maria DB |
Beta Was this translation helpful? Give feedback.
-
|
I will do some articles about catching strategy and more to give performance in real world scenarios, this is not only a toy library, this will be a really complete framework for Python and next Ruby and Lua |
Beta Was this translation helpful? Give feedback.
-
|
@cirospaciari I think Its better that socketify.py just wraps the APIs of uWebsockets. |
Beta Was this translation helpful? Give feedback.
-
At least I need to integrate with event loop like uvloop (to remove workarounds), An slim version will be basically just be an uWebSockets wrapper, in the current state we already have get_data() get_json() get_text(), get_parameters(), get_queries(), cork_end(), get_headers() send_chunk() app.static and one sendfile helper, all needed to answer basic needs for developers without been a pain to do manually. Fetch API is almost done, Files API is needed because PyPy3 don't have any support of any current library, and it's very very easy to do. From the start I always want this to be more than just an wrapper and my first option was drogon not uWebSockets. But I liked more UWS |
Beta Was this translation helpful? Give feedback.
-
|
WebSockets are benchmarked and a better readme that highlights better what's matter including Stargazers and Forkers, @vytas7 feedback from Falcon gitter was really helpful and meaningful. I'm working in the current Documentations including examples with remate engines like Mako e Jinja that are missing, tutoriais and full API doc is on the works too. The first stable release will be publish after the docs |
Beta Was this translation helpful? Give feedback.
-
|
Plain Text JSON I was dumb enough to copy the socketify.rb and forgot to change the language name to Python, i added an PR to TechEmPower to fix that. Socketify is the fastest in active development web framework for Python in TechEmPower. |
Beta Was this translation helpful? Give feedback.
-
|
@kijk2869 for now on i will be using Thanks <3 |
Beta Was this translation helpful? Give feedback.
-
|
I updated https://github.com/cirospaciari/socketify.py/blob/main/CONTRIBUTING.md with this to keep note ;) |
Beta Was this translation helpful? Give feedback.
-
|
I also think we need a discord server Discord server: https://discord.socketify.dev/ |
Beta Was this translation helpful? Give feedback.
-
|
Initial docs: www.socketify.dev or docs.socketify.dev |
Beta Was this translation helpful? Give feedback.
-
|
Just want to say again how awesome this project is and thank you for all your work on it. I’ve got a benchmarking question. I see that the requests per second benchmark really shines, but I’m curious if there have been any benchmarks with large numbers of simultaneous connections. For example: https://unetworkingab.medium.com/millions-of-active-websockets-with-node-js-7dc575746a01. Is there anything about Python’s memory management or anything like that that would preclude being able to get in the ballpark of the article above? |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for your support this really means a lot for the me and the project. I will try to do more benchmarks with more connections, my notebook is old and overheating lol, TechEmPower benchmarks posted here highlights that we are in the same ballpark, is just a difference of bench tools and hardware, but i will compare with uwebsocket.js implementation, bun.sh and others like socketify.py, here some comparisons on my machine:
In python3 (CPython) we have a lot of things that could impact performance (like using CFFI) that we are working on and will improve with time. In PyPy3 is the same, we are always improving. This project just focus in bring the best performance outof uWS and compatibility overall in an fast and reliable way (using CFFI) the next step is stability and optimizations (HPy), The Goal is get CPython (python3) in the same ballpark as Go Fiber. Here another results comparing with uWS.js and uWS C++
|
Beta Was this translation helpful? Give feedback.
-
|
Preparations for the first PyPi package are done, in the next version we should publish on PyPi too ❤️ 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
Implemented option to use object factory for AppRequest() and AppResponse(), and also WeSockets objects: app = App(request_response_factory_max_itens=200_000, websocket_factory_max_itens=1_500_000)TechEmPower plaintext With factory:
Without factory:
More performance information with PyPy + object factory: In CPython the performance increase are less pronounced, up to 10% Benchmarks numbers in README.md will be updated soon |
Beta Was this translation helpful? Give feedback.
-
|
After some tweaks and testing with ASGI protocol now we can power Falcon for WebSockets.
|
Beta Was this translation helpful? Give feedback.
-
|
@seanr3 just want to thank you for your help finding bugs, this means a lot! |
Beta Was this translation helpful? Give feedback.
-
|
asyncio study case: On my machine using more threads push performance up for async. The best results i can get on my machine for this cases bellow are:
EDIT: before socketify.py pypy was 79314, now is 131598 after optimiations, and asgi before was 30157 and now is 53731, also increase 11% more performance in ./http_load_test For comparison some sync numbers:
The socketify advantage is explore async and sync when needed, so you can do work async, cache and respond sync, the Goal is to be the fastest async and sync server and web framework for python. This tests are using oha and this is not using pipelining, we can push way harder than this with wrk or other tools, but this show an more realistic number. The true super power currently is pub/sub in websockets for socketify
WSGI compatibility is also here, but performs really badly, ASGI, SSGI and RSGI will be supported with pub/sub extensions and i will optimize this three, i will not put much effort on WSGI for now maybe in future. |
Beta Was this translation helpful? Give feedback.
-
|
Almost all this performance increase is not commited yet, but will soon be available after some more native optimizations. |
Beta Was this translation helpful? Give feedback.
-
|
Current overhead of socketify ASGI and WSGI for reference using TechEmPower plaintext
|
Beta Was this translation helpful? Give feedback.
-
|
@tiangolo @AmirHmZz just added the numbers for fastapi and added an PR to TechEmPower, should get between 3x and 10x improvements or even more, TechEmpower/FrameworkBenchmarks#7781 |
Beta Was this translation helpful? Give feedback.
-
|
New cli tool to help people to use workers/forks, ASGI, WSGI etc is in the works. python3 -m socketify --helpRunning you can easyly run wsgi, asgi or socketify app python3 -m socketify hello_world_cli:run --port 8080 --workers 2
python3 -m socketify falcon_wsgi:app --port 8080 --workers 2
python3 -m socketify falcon_asgi:app --port 8080 --workers 2Socketify apps requires you to pass an run function, the cli will create the instance for you from socketify import App
# App will be created by the cli with all things you want configured
def run(app: App):
# add your routes here
app.get("/", lambda res, req: res.end("Hello World!"))WebSockets can be in the same or another module, you can still use .ws("/*) to serve Websockets python3 -m socketify hello_world_cli:run --ws hello_world_cli:websocket --port 8080 --workers 2websocket = {
"open": lambda ws: ws.send("Hello World!", OpCode.TEXT),
"message": lambda ws, message, opcode: ws.send(message, opcode),
"close": lambda ws, code, message: print("WebSocket closed"),
}When running ASGI will be served by default but you can disabled it python3 -m socketify falcon_asgi:app --ws none --port 8080 --workers 2When running WSGI or ASGI you can still use socketify.py or ASGI websockets in the same server, mixing all available methods python3 -m socketify falcon_wsgi:app --ws falcon:ws none --port 8080 --workers 2 |
Beta Was this translation helpful? Give feedback.
-
|
Some options will be added soon --uds TEXT Bind to a UNIX domain socket.
--reload Enable auto-reload. This options also disable --workers or -w option.
--reload-dir PATH Set reload directories explicitly, instead of using the current working directory.
--reload-include TEXT Set extensions to include while watching for files.
Includes '.py,.html,.js,.png,.jpeg,.jpg and .webp' by default;
these defaults can be overridden with `--reload-exclude`.
--reload-exclude TEXT Set extensions to include while watching for files.
--reload-delay INT Milliseconds to delay reload between file changes. [default: 1000] |
Beta Was this translation helpful? Give feedback.
-
|
The |
Beta Was this translation helpful? Give feedback.
-
You can also use nginx as reverse proxy without uds but socketify.py already supports this feature I just need to include in CLI, maybe tomorrow this will be in the main branch |
Beta Was this translation helpful? Give feedback.
-
|
Can socketify beeing used as an ASGI server? For example instead of uvicorn for running a FastAPI app? |
Beta Was this translation helpful? Give feedback.
-
Sure I will write tests soon, just need to finish the docs and some more fixes, but tests are in the roadmap. |
Beta Was this translation helpful? Give feedback.
-
|
If socketify passes tests we can then talk to @tiangolo in order to propose socketify as an ASGI server alongside with fastapi so he can add socketify name to this page of fastapi doc! |
Beta Was this translation helpful? Give feedback.
-
This is really great! I will provide tests soon 🔥 |
Beta Was this translation helpful? Give feedback.
-
|
I think focus on fastapi is much more important than any other framework because of its huge community. I think adding Fastapi to benchmarks can be interesting for backend developers! |
Beta Was this translation helpful? Give feedback.
-
Sure I will add a lot of more benchmarks info, fastapi and Django with ASGI are the next, I will provide a game changing performance boost to all framework for sure ⚡🦾 |
Beta Was this translation helpful? Give feedback.
-
|
Just a quick update last Official TechEmPower plaintext results for socketify, we hit 6 million requests per second 🏆 this is a new record for Python! Go Fiber got 5.9 million we are faster than Fiber! 🚀🦾🔥⚡
|
Beta Was this translation helpful? Give feedback.
-
|
@tiangolo and @AmirHmZz I think 🤔 we will get a Pretty good boost in performance in fastapi with socketify.py |
Beta Was this translation helpful? Give feedback.
-
|
@cirospaciari Why didn't you use orjson instead of built-in json module of CPython? |
Beta Was this translation helpful? Give feedback.
-
|
Isn't it better to let developers choose between built-in json and orjson? |
Beta Was this translation helpful? Give feedback.
-
I will with plugins and extensions in next release, socketify.py will not add any unnecessary external dependencies, this will be optional |
Beta Was this translation helpful? Give feedback.
-
|
I also have some python specific advice in order to make socketify.py faster (even a little bit). |
Beta Was this translation helpful? Give feedback.
-
If is to TechEmPower you can just send me before so we can do some tests |
Beta Was this translation helpful? Give feedback.
-
Also create a new option (keep the current version without modifications) falcon do the same with Falcon [orjson] option |
Beta Was this translation helpful? Give feedback.
-
|
This is a little bit quiet because of the holidays but: First version on PyPI: https://pypi.org/project/socketify/ Work in progress:
Edit: 50% progress |
Beta Was this translation helpful? Give feedback.
-
|
DX is getting better and I hope v0.1.0 comes soon! |
Beta Was this translation helpful? Give feedback.
-
|
almost 1 month after this and still have not finished all things, Feb will be a month with few improvements, but hopefully, I will finish native optimizations little by little. |
Beta Was this translation helpful? Give feedback.
-
👋 Welcome Pythonists!
We’re using Discussions as a place to connect with other members of our community. We hope that you:
build together 💪.
Hello people this is the place to get updates, preliminary performance results and more about the state of this project.
Beta Was this translation helpful? Give feedback.
All reactions