Draft: Oban makes it so easy, you don't even stop to think if you should

[vereis]

Oban makes it so easy, you don't even stop to think if you should

In my opinion, Oban and Oban Pro are absolutely foundational libraries in the Elixir ecosystem, not unlike Ecto, Phoenix, or Absinthe.

I started using Oban because it was already used as a simple background job processing library at Vetspire. Since my first foray into using Oban however, we've really gotten to grips with it and started using it in anger.

Before trying it out, a lot of the stuff I do today with Oban was done via traditional message queues like RabbitMQ/Kafka/SQS, or with serialized gen_statems stored in Mnesia or Postgres which were re-initialized on application boot up.

So many small, low-level wants and needs are so perfectly fulfilled by Oban (even when it's not quite the right tool for the job), that I've realised two things:

One—I seldom ever need to really think hard about lower-levell/fun/distributed/resilient code anymore;
Two—Oban makes things so easy, you don't even necessarily think twice before coercing it to do incredibly cursed things 🤣

The following points are a few fun things I've used Oban for at both Vetspire and personal projects. They're fun recipes which seem to be doing okay in production environments/workloads, but they're moreso fun examples of what can be done rather than what should be done.

Definitely don't copy these ad-hoc, but here we go! We'll start off with the least cursed item and end with the most cursed item!

Oban as a retry mechanism

Outside of having truly background background tasks, Oban can be leveraged to make any function call you're expecting to make more resilient by leveraging its retry functionality.

If an Oban job returns an error, or crashes for whatever reason, by default Oban will retry the job up to 20 times (with exponential backoff, but all of this is very configurable).

One of the main issues with doing this is that Oban jobs are run, well... as background jobs. But assuming we can live with the fact that anything you use Oban for will be asynchronous (for now...), simply wrapping any function call in an Oban job will let you make that call more resilient.

Vetspire integrates with a lot of third-party code. No matter what, even if your integrations are rock solid, due to the nature of HTTP and distributed systems in general, stuff will fail sometimes and there is very little recourse/predictability associated with this.

As a result, a lot of our very critical integrations have their communications wrapped in Oban jobs so that if they fail for random reasons, we'll try again a little later.

A pattern I personally like is having a more generic retry worker template that looks like the following:

defmodule MyApp.Integration.Submitter do
  use Oban.Worker, max_attempts: 5

  def perform(_job) do
    case MyApp.Integration.submit() do
      {:ok, %DataSyncLog{} = datasync_log} ->
        {:ok, datasync_log}

      {:error, something_totally_expected_to_fail} ->
        {:discard, error: something_totally_expected_to_fail}

      ... repeat as needed ...

      error ->
        error
    end
  end
end

Since these integrations can often return completely valid errors, I usually enumerate and handle them all explicitly in the job. Anything that isn't explicitly handled will cause a retry (even if your BEAM VM crashes, Oban will retry it!), but anything that is totally expected and BAU will be transformed into an Oban :discard instead.

I typically like co-locating my domain-specific workers in the same context as the modules which use them. I'll also usually create schedule_ functions which utilise them for me "automagically" like so:

defmodule MyApp.Integration do
  def schedule_submit!, do: Oban.insert!(MyApp.Integration.Submitter.new(%{}))

  def submit do
    HTTPoison.post(...)
    ...
  end
end

This way, instead of calling submit/0 in the example above, a caller can choose to call schedule_submit!/0 for the same effect, just more robust!

A lot of early Vetspire code would utilise spawn/3 to offload work in the background as well. Depending on the actual performance implications of doing so, we've ended up refactoring a lot of that to this exact pattern!

Oban as state

Another cool thing Oban can do is literally act as state. This is one of those things that I genuinely don't condone doing in production, but if you're using Oban anyway, Oban is a great way to have short-lived ad-hoc distributed state across your application/cluster.

This works because Oban allows you to define unique constraints for Oban jobs. Typically, unique constraints are used to protect against double enqueueing jobs, but the actual fields you define as unique are pretty flexible.

You can mark jobs are unique by their :args, :queue, :meta fields, etc. See (unique fields, options, and states)[https://hexdocs.pm/oban/Oban.Job.html#t:unique_field/0] for more information.

Coupling a job with some unique constraint involving its fields and its age, Oban literally becomes a cache. This relies on the fact that you can store arbitrary metadata in a job's :meta field: in some Oban cronjob you can perform expensive computations and store the result in the job's :meta field. Once this is done, your application can try reading from the job queue to read whatever result was cached until the job gets pruned or re-written.

You can combine this with upserting jobs via the replace option when creating new jobs to do all sorts of gross cursed stuff!

Honestly, it isn't that bad though, definitely not much different from rolling your own GenServer or Agent based cache with the advantage that its available to any app that has access to your database. If creating a new database table is too much work, you need distributed state, and you don't want to use :mnesia (who does! 😆) then this might be an approach?

Then again... probably not the best idea, even if it isn't the worst 😅

Oban as a throttle mechanism

Oban as a Flow replacement

Oban as an async/await runner