Improve incoming request handling by accepting all requests #73
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently, when the number of active requests is greater than
max_threads
, we dotime.sleep(0.05)
and check again indefinitely untilwe can accept the request.
Also, when this happens, the
unaccepted_requests
counter isincremented.
This condition creates an issue. The number of incoming requests must be
no more that the number of warcprox threads(!) and limits the capacity of
warcprox to handle incoming requests.
If we select to use a small number of
max_threads
and we have afair number of browsers using warcprox, a lot of requests will wait.
Since the
ThreadPoolExecutor
we use has a queue, we can accept allrequests regardless of the number of
max_threads
and we'll serve themwhen we can (The queue is FIFO).
The clients will have their connections accepted and maybe wait a bit for
response but they won't be blocked.
Also, VERY important is that we'll be able to keep the number of
max_threads
low.In my performance tests using a VM with 4 CPU cores, I set
max_threads=20
and I saw
active_requests
reach values ~50 withoutseconds_behind
increasing over 1 sec.In addition, another improvement in this PR is that we convert
PooledMixIn.active_requests
to a simple counter instead of a set().We use this var only to add/remove requests and do
len(self.active_requests)
to report their numbers.
Since we don't do anything with the set contents, there is no point to
keep it in memory.
We'll just do self.active_requests +-1 when necessary.
TODO: If this PR is OK up to this point, I also need to remove
unaccepted_requests
from everywhere.