Add rq.runtime for information from workers/queues

[joostdevries]

Follow up on #517

Comment from @nvie

There could have easily been a nicer API for that, i.e. imagine a Runtime.get_running_worker_info(), rather than Worker.all() and then misusing Worker instances for only the info. It's been a mis-design :(
I actually like a single rq.runtime module or class, for logging/inspection/monitoring purposes. I've seen a lot of misuse of the Worker/Job/Queue classes in the wild, simply because there is no other/proper way of getting that runtime information. Ideally, those runtime helpers would only pull data from Redis, and not inspect OS processes, etc.

Below a draft of what the API could look like. Contributions are greatly appreciated ;-) @nvie @selwin

API (draft)

• rq.Runtime(redis)
  • rq.Runtime.get_workers()
• rq.WorkerInfo
• rq.QueueInfo
  • rq.QueueInfo.get_jobs()

[ccrvlh]

Moving this to a discussion since it's pretty rough around the edges still, and this topic hasn't been alive for quite a while now.

--

For the record and eventual discussions in the future, personally I find that this design in which there an extra (very specific) functional class can be troublesome and confusing.

What I prefer, is to have a singleton-like entrypoint, and managing things from there.

rq = RQ(connection=Redis())
rq.get_workers()
rq.get_queues()
rq.get_jobs()
rq.enqueue('low', task)  # avoids having to import multiple queues

This centralizes the management tasks into a single place, is a single import, single Redis connection to pass around and a lot easier to use IMO. I actually implement this in my projects, so I would be happy to work on that @selwin

[selwin]

I like the idea of having an RQ entry point. However, I think we'll still need classes such as Queue and Worker as these classes also help users in conceptualizing what the code does. Perhaps something like this:

rq = RQ(connection=Redis())
queue = rq.get_queue('low')
worker = rq.get_worker('default')

[ccrvlh]

I agree, I like the dedicated classes, and think they are an excellent example of great OOP.

[selwin]

Having a better CLI interface to get runtime information is also something that I'd like to have. Perhaps we can build a separate rq-cli package that uses https://github.com/Textualize/rich to present information nicely.

[ccrvlh]

Rich is great indeed, this is something to consider, especially with multiple info alternatives (rq-dashboard, rq-monitor, rq-exporter)

[selwin]

Yeah, but unfortunately I don't think any of them are very actively maintained.