Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(electric): Fix a data race that may occur when a client is sufficiently behind the latest Postgres state #1119

Closed
wants to merge 3 commits into from

Conversation

alco
Copy link
Member

@alco alco commented Apr 2, 2024

This PR addresses the following edge case:

  1. a client connects to the server
  2. the server confirms that the client is in the cached WAL window
  3. a garbage collection cycle occurs and the cache drops old transactions as a result
  4. the client starts replicating from the server but never receives the transactions it was supposed to get

This leads to the client having an inconsistent state with the server.

The way this is addressed is by implementing a reservation system inside the cache to prevent it from dropping reserved transactions even if they are already outside of the cached WAL window size.

Copy link

linear bot commented Apr 2, 2024

@alco alco changed the base branch from main to alco/code-cleanups April 2, 2024 11:46
@alco alco force-pushed the alco/vax-1686-reserve-cached-transactions branch from 9dfdaa9 to 6f3cf33 Compare April 2, 2024 14:14
@alco alco force-pushed the alco/vax-1686-reserve-cached-transactions branch 2 times, most recently from 498d287 to 513ece2 Compare April 4, 2024 11:00
@alco alco changed the title Alco/vax 1686 reserve cached transactions fix(electric): Fix a data race that may occur when a client is sufficiently behind the latest Postgres state Apr 4, 2024
@alco alco force-pushed the alco/vax-1686-reserve-cached-transactions branch from 513ece2 to 29a7a18 Compare April 5, 2024 13:30
@alco alco force-pushed the alco/code-cleanups branch 2 times, most recently from 668518c to 82033ba Compare April 9, 2024 14:10
@alco alco force-pushed the alco/vax-1686-reserve-cached-transactions branch from 29a7a18 to affe5c3 Compare April 9, 2024 14:13
Base automatically changed from alco/code-cleanups to alco/vax-1686-refactor-cached-wal-ets-backed April 10, 2024 16:15
@alco alco force-pushed the alco/vax-1686-refactor-cached-wal-ets-backed branch from 79c68d4 to 8af60bd Compare April 10, 2024 20:22
@alco alco force-pushed the alco/vax-1686-reserve-cached-transactions branch from affe5c3 to f14c8da Compare April 10, 2024 20:53
@alco alco changed the base branch from alco/vax-1686-refactor-cached-wal-ets-backed to alco/origin-based-cleanup April 10, 2024 20:53
@alco alco force-pushed the alco/origin-based-cleanup branch 2 times, most recently from 34a0181 to da7d367 Compare April 18, 2024 14:11
Base automatically changed from alco/origin-based-cleanup to main April 18, 2024 14:12
@alco alco force-pushed the alco/vax-1686-reserve-cached-transactions branch 3 times, most recently from e72f2b4 to ef20224 Compare April 22, 2024 21:59
@alco alco force-pushed the alco/vax-1686-reserve-cached-transactions branch from 54e4bdb to 5b9352a Compare April 24, 2024 10:52
@alco alco force-pushed the alco/vax-1686-reserve-cached-transactions branch from 5b9352a to 4aaf792 Compare April 24, 2024 11:21
@alco
Copy link
Member Author

alco commented Apr 25, 2024

Moved to a different branch - #1199.

@alco alco closed this Apr 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant