Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Inserts can be lost during a predicate move #2338
With server-side ordering, @upsert schemas, no crashes or network faults, roughly 10 inserts/sec, and no updates or deletes, Dgraph can occasionally (once every five hours or so) lose successfully inserted records: 20180412T161038.000-0500.zip
The lost records occur during a predicate move, which suggests this issue might be related to #2321. This occurs with
and can be reproduced with Jepsen 23329ead4c4e3d8352234658026d09792f15c406 via
You can now reproduce this problem significantly faster in Jepsen eb796cfcc204c592545965968bd28ad1e6b2eff0 by using the move-tablet nemesis, which shuffles tablets around every 15 seconds or so.
With predicate moves, we can get dgraph to lose 99% of acknowledged inserts in 60 seconds: