Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
[Jepsen] Snapshot Isolation (SI) violation under huge clock skew even if it is bounded by max_clock_skew_usec flag #975
@ttyusupov caught an issue with bank account test - total sum diverges under huge clock skew nemesis even if clock skew is bounded by
on a GCP cluster - 5 separate debian jessie instances.
This test introduces random clock bumps up to 262 seconds in both directions and random clock strobes from 4 ms to 262 seconds, with a period of 1 ms to 1 second, for a duration of 0-32 seconds (every period ms, adjusts the clock forward by delta ms, or, alternatively, back by delta ms. Does this for duration seconds).
If we reduce delta for both bumps and strobes to 131 seconds - test passes.
I was able to reproduce bank account test failure under high clock skew and tracked down to a transaction which broke the total sum along with its read_time and other info, both conflicting transactions were applied.
Transaction 1aa7ccd8-fd49-4803-8b9b-9d6fc1a68d43 is using
Some related logs:
Finally found a bug in
So, in case
failed status won't be saved into