Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve sending efficiency for large Groups #62

Closed
lsapan opened this issue Oct 18, 2017 · 2 comments
Closed

Improve sending efficiency for large Groups #62

lsapan opened this issue Oct 18, 2017 · 2 comments

Comments

@lsapan
Copy link

lsapan commented Oct 18, 2017

This is really just the continuation of django/channels#763, as it was determined that the redis channel layer was causing the issue.

At smaller Group sizes (less than 100) it is not really noticeable. Once you get to 200, some messages are occasionally dropped. At around 400+ it becomes very evident and can even approach a 50% failure rate.

I've created a repo to reproduce this issue very easily: https://github.com/lsapan/channels-mass-broadcast

The repo's README explains everything in depth, but here's everything you need to reproduce it:

git clone https://github.com/lsapan/channels-mass-broadcast.git
cd channels-mass-broadcast
docker-compose pull
docker-compose up

That will start daphne, a worker, redis, etc. Then simply navigate to http://localhost:8000/ in 1-3 unique web browsers and watch the number of messages coming in through the console.

@andrewgodwin
Copy link
Member

Right, it being in groups explains why I've never seen the issue when we put 5,000 req/sec through it because we never use that part.

Group sending right now is not implemented very efficiently, and I suspect this is due to the for-loop style of sending falling down. I'm going to look into better group send mechanisms and see what I can come up with to improve it, though that may have to be part of the ongoing rewrite of asgi_redis to be async for channels 2.

@andrewgodwin andrewgodwin changed the title Websocket messages start getting dropped with higher volume Channel Groups Improve sending efficiency for large Groups Feb 2, 2018
@andrewgodwin
Copy link
Member

Closing as duplicate of #7

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants