ActiveJob interface

[bglusman]

@sschepens we talked about this and I know on the roadmap, but wanted to document here with an issue, we may tackle this at work in a fork and PR it back, or let me know if you've already started. Toiler's working great for us in production now by the way, much better than Shoryuken was, lower memory and faster processing!

[sschepens]

Toiler's working great for us in production now by the way, much better than Shoryuken was, lower memory and faster processing!

Glad to hear that!

I investigated how ActiveJob works and it's kind of tricky, right now Toiler assumes a queue is bound to only one Worker or Job. ActiveJob supports dispatching multiple jobs to a single queue and it sends in the payload the actual class of the job that created the message to instantiate it and call process on it.

Maybe we could implement a separate Module for ActiveJob, something like including Toiler::Job and have a different logic there that's specific for ActiveJob.

Anyways, this requires a few changes, toiler should serialize jobs to the queue in a specific way to support ActiveJob.

I don't have much time lately, @bglusman if you guys could tackle this it would be great!

[bglusman]

That could work.. I also think it's OK if, for instance, all toiler active job classes require you to use the queue_as option and it's an error to not call that from a class (or to have two classes with the same queue?), but the Toiler::Job thing sounds promising. It does seem like active job vaguely assumes there jobs are enqueued from rails which is not the case for many people using SQS but shoryuken had an active job adapter and I imagine we can borrow or adapt a lot of their logic. Thanks, will let you know, this is probably lead important to us than the other ticket Abe's now of s nice to have/maybe we can give something back to help toiler adoption/community, so we'll probably timebox and see how it goes.