DynamicQueues Opt issues when upgrade ObanPro from 0.11.0 to 0.13.2

[mattheusmarzola]

Environment

• Oban Version
  2.14.2

• Oban Pro Version
  0.13.2

• PostgreSQL Version
  psql (PostgreSQL) 14.5

• Elixir & Erlang/OTP Versions (elixir --version)

Erlang/OTP 25 [erts-13.1.4] [source] [64-bit] [smp:10:10] [ds:10:10:10] [async-threads:1] [jit] [dtrace]
Elixir 1.14.3 (compiled with Erlang/OTP 25)

Current Behavior

When we updated from Oban 2.12.1 to 2.14.2 and ObanPro 0.11.0 to 0.13.2 we are faced an issue with DynamicQueues, and the global_limit opt, from queues that already exists on Database.

We found a difference in the global_limit option structure. For example, the data from oban_queues.opts column:
Before:
{"paused": null, "rate_limit": null, "local_limit": 1, "global_limit": 1}
After:
{"paused": null, "rate_limit": null, "local_limit": 1, "global_limit": {"allowed": 1, "tracked": {}, "partition": null}, "retry_backoff": 1000, "retry_attempts": 5}

When we tried to run the application we got the error traces below:

SELECT o0."name", o0."lock_version", o0."only", o0."opts", o0."inserted_at", o0."updated_at" FROM "public"."oban_queues" AS o0 ORDER BY o0."inserted_at" []
[error] GenServer {Oban.Registry, {Oban, {:plugin, Oban.Pro.Plugins.DynamicQueues}}} terminating
** (ArgumentError) cannot load `1` as type {:parameterized, Ecto.Embedded, %Ecto.Embedded{cardinality: :one, field: :global_limit, owner: Oban.Pro.Queue.Opts, related: Oban.Pro.Queue.Opts.GlobalLimit, on_cast: nil, on_replace: :update, unique: true, ordered: true}} for field `global_limit` in schema Oban.Pro.Queue.Opts
    (ecto 3.9.4) lib/ecto/schema/loader.ex:76: anonymous fn/5 in Ecto.Schema.Loader.unsafe_load/4
    (elixir 1.14.3) lib/enum.ex:2468: Enum."-reduce/3-lists^foldl/2-0-"/3
    (ecto 3.9.4) lib/ecto/embedded.ex:64: Ecto.Embedded.load/3
    (ecto 3.9.4) lib/ecto/type.ex:919: Ecto.Type.process_loaders/3
    (ecto 3.9.4) lib/ecto/repo/queryable.ex:411: Ecto.Repo.Queryable.struct_load!/6
    (ecto 3.9.4) lib/ecto/repo/queryable.ex:243: anonymous fn/5 in Ecto.Repo.Queryable.preprocessor/3
    (elixir 1.14.3) lib/enum.ex:1658: Enum."-map/2-lists^map/1-0-"/2
    (ecto 3.9.4) lib/ecto/repo/queryable.ex:234: Ecto.Repo.Queryable.execute/4
Last message: {:continue, :start}
State: %Oban.Pro.Plugins.DynamicQueues.State{conf: %Oban.Config{dispatch_cooldown: 5, engine: Oban.Pro.Queue.SmartEngine, get_dynamic_repo: nil, log: :debug, name: Oban, node: "menhir@pop-os", notifier: Oban.Notifiers.Postgres, peer: Oban.Peers.Postgres, plugins: [{Oban.Pro.Plugins.Relay, []}, {Oban.Pro.Plugins.DynamicQueues, [queues: []]}, {Oban.Plugins.Cron, [timezone: "Europe/Lisbon", crontab: []]}, {Oban.Pro.Plugins.Lifeline, []}, {Oban.Web.Plugins.Stats, []}, {Oban.Plugins.Gossip, []}], prefix: "public", queues: [default: [limit: 10], unlock_user: [limit: 1]], repo: Coverflex.Database.Repo, shutdown_grace_period: 15000, stage_interval: 1000, testing: :disabled}, name: {:via, Registry, {Oban.Registry, {Oban, {:plugin, Oban.Pro.Plugins.DynamicQueues}}}}, timer: nil, interval: :infinity, queues: []}

Do you have any suggestions or already exists a migration to update old queue records?

Thank you!

Expected Behavior

Maybe a migration to update existing queue records

[sorentwo]

Thanks for the report! Sorry for your trouble.

The simplest way to change the schema is to declare those queues in the DynamicQueues list again. If that's not an option, then the following SQL will fix it:

update oban_queues
set opts = jsonb_set(opts, '{global_limit}', jsonb_build_object('allowed', opts->'global_limit', 'partition', null))
where opts->'global_limit' <> 'null'::jsonb;

[mattheusmarzola]

Thanks @sorentwo !