Create message rate store

[rudigiesler]

Create a store that can get called whenever the counter should be incremented, and be able to get the current msgs/s, with configurable bucket sizes.

[imsickofmaps]

Completely without context for what the details of this are :) but if you're using Rabbitmq for the queue it does ship with an API that publishes rate data: https://cdn.rawgit.com/rabbitmq/rabbitmq-management/rabbitmq_v3_5_6/priv/www/api/index.html

[hodgestar]

@imsickofmaps We're planning to include queue sizes and message rates from
the RabbitMQ later. The immediate work is to:

• Set up the infrastructure for such metrics.
• Publish the message rates as the API workers see them (which will look a
  bit different to the RabbitMQ queue view).

[imsickofmaps]

@hodgestar awesome, carry on 😀

[rudigiesler]

@hodgestar Ready for review.

[hodgestar]

Should bucket_size not be passed to the MessageRateStore constructor? That would avoid having to pass it in to the individual methods and prevent people from passing in different values?

[hodgestar]

If we pass the bucket size into the constructor, we can also set the ttl there.

[rudigiesler]

I originally wanted to inherit as much as possible from the BaseStore, but I see now that it is hurting us more than helping us.

The reason I wanted bucket_size to be passed per function is that in the API when we're fetching the msgs/s, inbound and outbound could be configured to have different bucket sizes. This reason is why we had channel_id being passed to every function for the other stores, instead of setting the channel id in the constructor.

The reason that ttl is being set here, is because we want to set the ttl when we increment, but we don't want to set the ttl again when we fetch the value (there we set ttl to None, so that we don't call EXPIRE.

[hodgestar]

Maybe we need a ttl flag that can override self.ttl? Or a reset_ttl=False flag? (for certain redis ops).

[hodgestar]

Left some questions about bucket size and TTL setting, but otherwise looks good.

[rudigiesler]

@hodgestar I've added a ttl keyword argument that can be used to override the TTL value. Thoughts?

[hodgestar]

If self.ttl = 5, setting ttl = None in the kwargs won't override the 5? Just checking whether that is intentional?

[rudigiesler]

Yes this is intentional. None means to use the default (and is the default for all of the methods) that is set when you create the class. I'm not sure if it's possible to make this clearer, maybe if the argument doesn't exist, it should use the default, and if it does exist, it uses that value? It would make things a bit more complicated though. Thoughts?

[hodgestar]

I think to make this work we'd need to use an explicit sentinel value, e.g. ``` USE_DEFAULT_TTL = object() def _redis_op(...): ttl = kwargs.pop('ttl') if ttl is self.USE_DEFAULT_TTL: ttl = self.ttl def get_id(..., ttl=USE_DEFAULT_TTL): ... ``` Too complicated?

[rudigiesler]

I think we can do that, yes. Looks good.

[rudigiesler]

@hodgestar Changes made, ready for another review.

[hodgestar]

👍