Please sign in to comment.
Improve fairness/responsiveness of background services
The "changeset" and "highlight" background services use a shared base-class (JSONJobServer) that implements request queueing: a client connects, writes a list of object literals ("requests") in JSON form, and then waits for a list of results (typically copies of the requests with some more information in them.) Before these changes, the order in which requests were serviced was "randomized" by storing all pending requests in a dictionary and then popping one at a time to process them. (The "randomization" achieved by this scheme was not exactly intentional.) Assuming the actual order in which requests were serviced was actually random, this meant that on average, all clients would finish at roughly the same time, and that time would be determined by the client with the longest list of requests. If one client had a huge list of requests, the background service would be unavailable to all clients for a long time. This commit changes the queuing policy by keeping a queue of clients with pending requests. Each time a new job is to be started, a request is picked from the client at the front of the queue, and the client is pushed to the back of the queue. This should achieve a situation where the time it takes for any given client to have all requests processed is something like nr_requests x system_load, which seems fair enough. As a side-effect of the implementation changes, the strategy for dealing with identical requests from multiple clients changed somewhat. The base- class used to group together such requests and run only one job. Now, it first allows the sub-class to intercept a request before it's processed (which allows the sub-class to check if the request is actually a no-op) and second assumes that processing a request is always okay, even though an identical request was processed while the first was queued. The syntax highlight service intercepts requests, since the check is very cheap (checking if a file exists.) The changeset service doesn't, since it needs a database connection to check if a changeset already exists, and only connects to the database in the slave processes today. Changesets are fewer and bigger, though, so the overhead of starting a slave process only to discover the changeset is already in the database is insignificant.
- Loading branch information...
Showing with 165 additions and 97 deletions.