All tiles are currently processed without any priority given to them. Priorities would allow us to control processing in more flexible ways, and may even increase throughput.
One way to add this would be to add support for multiple queues, where the queues are queried in a configured order. We would need to update:
- intersect process to enqueue tiles to the appropriate queue
- worker processor that pulls work from the queue should ask for tiles from queues in priority order
- seed process - probably best to seed tiles using the same strategy
Assuming we always want to have tiles enqueued using this strategy, it would be best to create a new tilequeue queue implementation that had this logic baked in. The other components would not need to change then. Even if we did need to support different behavior based on context, we could add in keyword arguments that certain queue implementations could honor, and others would ignore.
All tiles are currently processed without any priority given to them. Priorities would allow us to control processing in more flexible ways, and may even increase throughput.
One way to add this would be to add support for multiple queues, where the queues are queried in a configured order. We would need to update:
Assuming we always want to have tiles enqueued using this strategy, it would be best to create a new tilequeue queue implementation that had this logic baked in. The other components would not need to change then. Even if we did need to support different behavior based on context, we could add in keyword arguments that certain queue implementations could honor, and others would ignore.