Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

internal error when inserting/deleting many jobs with binlog #38

Closed
kr opened this Issue · 2 comments

1 participant

@kr
Owner
kr commented

Add a new invariant to reserved space.

A DELETE record can be written at any time, including times when we are
unable to shuffle reserved bytes around. So, to make sure that we don't
lose any reserved bytes, the current binlog must always have reserved
space for some number of complete DELETE records. This lets us fill up
the current binlog exactly with no lost reserved bytes. The only time we
are able to shuffle things around to maintain this property is when
reserving bytes.

We used to try to accomplish this by checking the reserved size of the
current_binlog after each reservation, but that isn't sufficient. The
next binlog can become current and run out of space without any
reservations taking place. This would force us to lose bytes.

Now we check that the current_binlog and all future binlog reservation
sizes are suitable to become the current binlog and fill up exactly.

Closed by 4d56a66.

@kr
Owner
kr commented

Just to clarify, this bug would only get triggered in a rare situation: the client must issue enough contiguous delete commands to fill up an entire binlog segment, from beginning to end. If any commands that allocate binlog space are interspersed, the bug is not triggered.

@dustin dustin referenced this issue from a commit
@kr Add a new invariant to reserved space.
A DELETE record can be written at any time, including times when we are
unable to shuffle reserved bytes around. So, to make sure that we don't
lose any reserved bytes, the current binlog must always have reserved
space for some number of complete DELETE records. This lets us fill up
the current binlog exactly with no lost reserved bytes. The only time we
are able to shuffle things around to maintain this property is when
reserving bytes.

We used to try to accomplish this by checking the reserved size of the
current_binlog after each reservation, but that isn't sufficient. The
next binlog can become current and run out of space without any
reservations taking place. This would force us to lose bytes.

Now we check that the current_binlog and *all future binlog* reservation
sizes are suitable to become the current binlog and fill up exactly.

Closes gh-38.
4d56a66
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.