batch tag upserts, remove randint in tag sampling

[DanCech]

This PR complements graphite-project/graphite-web#2125 and switches writer to batch tag updates.

The implementation is pretty basic, series created and updated are added to 2 queues which are then processed in batches in a separate thread. 2 queues are used so that new series are prioritized when writing to the TagDB, and so that they don't compete with updates for space in the queue. Each queue has a maximum length of TAG_QUEUE_SIZE.

The writer will write at most TAG_BATCH_SIZE paths in each request, and will process the queue as quickly as possible until the queue is empty. I'm not sure if we want to add a delay in between batches to avoid overloading graphite-web.

The other change here is to remove the use of random.randint in selecting which updates to send to the tagdb, instead just using a global counter and sending every TAG_UPDATE_INTERVALth path. As long as the total number of series being processed isn't an exact multiple or factor of TAG_UPDATE_INTERVAL this will still result in every series getting upserted periodically.

Finally I updated run_twistd_plugin to enable debug logging when passing --debug on the command line.

As always, any and all suggestions for improvement are appreciated.

H/T to @Civil for the suggestion of batching the tagdb updates.

[deniszh]

Maybe stupid question - why not just call writeTags()? Why assign to written and never check result? And writeTags() should not return anything, right?

[DanCech]

Yeah that was a by-product of some refactoring that I need to clean up

[DanCech]

I realized that state.database.tag() was previously set up not to wait for the http call to complete, the latest commit changes that so that state.database.tag() returns a Deferred and writeTags() waits for it to complete before firing off the next request.