Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MatchError on a DB transaction in Oban.Peers.Postgres #1007

Closed
whatyouhide opened this issue Jan 4, 2024 · 2 comments
Closed

MatchError on a DB transaction in Oban.Peers.Postgres #1007

whatyouhide opened this issue Jan 4, 2024 · 2 comments
Labels
area:oss Related to Oban OSS kind:bug Something isn't working

Comments

@whatyouhide
Copy link
Contributor

Environment

  • main
  • 13.x
  • Elixir & Erlang/OTP Versions (elixir --version)
Erlang/OTP 26 [erts-14.0.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit]

Elixir 1.16.0 (compiled with Erlang/OTP 24)

Current Behavior

We've seen this in Sentry:

** (MatchError) no match of right hand side value: {:error, :rollback}
  lib/oban/peers/postgres.ex in anonymous fn/2 in Oban.Peers.Postgres.handle_info/2 at line 93
  <redacted>/deps/telemetry/src/telemetry.erl in :telemetry.span/3 at line 321
  lib/oban/peers/postgres.ex in Oban.Peers.Postgres.handle_info/2 at line 92
  gen_server.erl in :gen_server.try_handle_info/3 at line 1077
  gen_server.erl in :gen_server.handle_msg/6 at line 1165
  proc_lib.erl in :proc_lib.init_p_do_apply/3 at line 241

This shows up in the current Oban code:

{:ok, state} =
Repo.transaction(state.conf, fn ->
state
|> delete_expired_peers()
|> upsert_peer()
end)

Expected Behavior

This transaction result should likely be handled, right?

@sorentwo sorentwo added kind:bug Something isn't working area:oss Related to Oban OSS labels Jan 4, 2024
@sorentwo
Copy link
Owner

sorentwo commented Jan 4, 2024

Thanks for the report. A rollback is unexpected there, and we don't want it to crash the peer.

@whatyouhide
Copy link
Contributor Author

Thanks for the quick fix @sorentwo! 💟

sorentwo added a commit that referenced this issue Jan 11, 2024
Infrequently, the postgres peer election transaction returns `{:error,
:rollback}`. Now that return value is handled to prevent a match error.

The peer maintains its current `leader?` status on rollback—this may
cause inconsistency if the leader encounters an error and multiple
rollbacks happen in sequence. That tradeoff is acceptable because the
situation is unlikely and less of an issue than crashing the peer.

Closes #1007
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:oss Related to Oban OSS kind:bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants