Documentation request: Oban #18
Description
Hey Mark! Awesome repo! I'm working on adding this to a project and it all seems pretty straightforward except for Oban.
The Problem
For those who may not be familiar, Oban allows you to do job processing with Postgres as its data store. Normally you configure Oban with something like this in your config/config.exs file:
config :my_app, Oban,
repo: MyApp.Repo,
plugins: [Oban.Plugins.Pruner],
queues: [default: 10]After installing fly_postgres_elixir, Oban fails to start with the following exception:
** (Mix) Could not start application my_app: MyApp.Application.start(:normal, []) returned an error: shutdown: failed to start child: Oban
** (EXIT) an exception was raised:
** (ArgumentError) expected :repo to be an Ecto.Repo, got: MyApp.Repo
(oban 2.10.1) lib/oban/config.ex:193: Oban.Config.validate_opt!/1
(elixir 1.12.3) lib/enum.ex:930: Enum."-each/2-lists^foreach/1-0-"/2
(oban 2.10.1) lib/oban/config.ex:55: Oban.Config.new/1
(oban 2.10.1) lib/oban.ex:151: Oban.start_link/1
(stdlib 3.15.2) supervisor.erl:414: :supervisor.do_start_child_i/3
(stdlib 3.15.2) supervisor.erl:400: :supervisor.do_start_child/2
(stdlib 3.15.2) supervisor.erl:384: anonymous fn/3 in :supervisor.start_children/2
(stdlib 3.15.2) supervisor.erl:1234: :supervisor.children_map/4
(stdlib 3.15.2) supervisor.erl:350: :supervisor.init_children/2
(stdlib 3.15.2) gen_server.erl:423: :gen_server.init_it/2
(stdlib 3.15.2) gen_server.erl:390: :gen_server.init_it/6
(stdlib 3.15.2) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
If I change the configuration to point to MyApp.Repo.Local then the application will start up, but if I understand correctly this will cause problems down the line. Oban jobs are inserted into the database using Oban.insert/2 which would then try to insert into the local database which may or may not be a read-only replica.
My Solution
I would love feedback on this solution as I'm sure there might be a better way to do it, but I updated my application module to look like this:
defmodule MyApp.Application do
use Application
def start(_type, _args) do
children = [
{Fly.RPC, []},
MyApp.Repo.Local,
{Fly.Postgres.LSN.Tracker, repo: MyApp.Repo.Local},
...
oban(),
MyAppWeb.Endpoint
]
opts = [strategy: :one_for_one, name: MyApp.Supervisor]
children
|> Enum.reject(&is_nil/1)
|> Supervisor.start_link(opts)
end
...
defp oban do
if Fly.is_primary?() do
config = Application.fetch_env!(:my_app, Oban)
{Oban, config}
end
end
endThen I created a new Background module that uses a macro to wrap all the Oban functions so that they will all be called against the primary region every time.
defmodule MyApp.Background do
@moduledoc """
This module serves as a wrapper around Oban so that it will work properly
with `Fly.Repo` instead of an `Ecto.Repo`.
"""
for {func, arity} <- Oban.__info__(:functions), func not in [:child_spec, :init, :start_link] do
args = Macro.generate_arguments(arity, __MODULE__)
@doc """
See documentation for Oban.#{func}/#{arity}
"""
def unquote(func)(unquote_splicing(args)) do
if Fly.is_primary?() do
Oban.unquote(func)(unquote_splicing(args))
else
Fly.RPC.rpc_region(Fly.primary_region(), Oban, unquote(func), unquote(args))
end
end
end
endAm I overlooking a simpler solution? I was thinking about making a video about how to do this, but I was wondering if it might make sense to have it live in the official documentation to make it easier for others to find. I also considered making the Background module a supervisor and have it determine whether or not to start the Oban process so that everything would be contained within that one module. Then I think it could maybe make sense to release it as a separate library if that would have value to others. I wasn't sure if it made sense to do that considering the relative simplicity of it. But I would love to know others' thoughts.