Request for change: threading model #25

Open
Pinkel opened this Issue Oct 9, 2013 · 2 comments

Comments

Projects
None yet
2 participants
@Pinkel

Pinkel commented Oct 9, 2013

In the current implementation, there is always one thread available per connection, which will have scaling issues on systems with a large number of persistent connections.

Now that browsers and clients are moving more and more to a large number of concurrent connections (6 to 8 seems to be the default), this issue becomes more exposed.
When the client is idle for a longer period it will cause a large number of threads to become idle, while at the same time the number of connections will reach the limit as specified by 'num_threads', causing new connections not to be served.
For the end-user the system will seem to be unresponsive, yet the server might have a very low load.

The only solution currently would be to disable keep_alive or increase the num_threads configuration parameter to a high enough value to serve all expected number of concurrent clients.
Both have a major draw back. Disabling keep_alive introduces a significant overhead for the non-idle connections, increasing the thread amount introduces a whole variety of performance/ scalability‎ challenges.

Ger Hobbelt has done an effort on creating a fork with a different threading model, as described by him:

The thread model there for the original mongoose is simple: set it up with N threads, take N+1 clients, each of which open a persistent (or otherwise long-open) connection and the one client per thread approach of mongoose denies client N+1 any access to the server. For a reasonable amount of threads (< 1000; it's not just Windows which fares better with fewer threads to manage) this is very risky as even embedded servers will see multiple clients at a time when they serve data and the viz is done client-side, for example. Browser X keeps connection for several seconds while loading and building a visual, thus denying browser/client Y access once the threadcount has been reached.
Hence I redid the code such that inactive (not transmitting, not receiving, i.e. no data I/O pending) connections are pushed back onto the queue, freeing the thread(s) to pick up waiting connections and/or other connections which report I/O activity.

His fork, based on an older version of mongoose, can be found at:

https://github.com/GerHobbelt/mongoose

I'd very much like to see civetweb being capable of improving the scalability.

For more information, also see the discussion on the mailing list:
https://groups.google.com/forum/#!topic/civetweb/GwvgmFPBcUg

@sunsetbrew

This comment has been minimized.

Show comment
Hide comment
@sunsetbrew

sunsetbrew Oct 14, 2013

Owner

I will try to deal with this in 1.6.

Owner

sunsetbrew commented Oct 14, 2013

I will try to deal with this in 1.6.

@sunsetbrew

This comment has been minimized.

Show comment
Hide comment
@sunsetbrew

sunsetbrew Oct 21, 2013

Owner

The design for this has been submitted for open discussion.
https://groups.google.com/forum/#!topic/civetweb/sFHF75IDSH0

Owner

sunsetbrew commented Oct 21, 2013

The design for this has been submitted for open discussion.
https://groups.google.com/forum/#!topic/civetweb/sFHF75IDSH0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment