Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scp timeout short circuit v blocking fix #1935

Merged
merged 7 commits into from
Feb 5, 2019
151 changes: 74 additions & 77 deletions src/scp/BallotProtocol.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1334,99 +1334,96 @@ BallotProtocol::setAcceptCommit(SCPBallot const& c, SCPBallot const& h)
return didWork;
}

static uint32
statementBallotCounter(SCPStatement const& st)
{
switch (st.pledges.type())
{
case SCP_ST_PREPARE:
return st.pledges.prepare().ballot.counter;
case SCP_ST_CONFIRM:
return st.pledges.confirm().ballot.counter;
case SCP_ST_EXTERNALIZE:
return UINT32_MAX;
default:
// Should never be called with SCP_ST_NOMINATE.
abort();
}
}

static bool
hasVBlockingSubsetStrictlyAheadOf(std::shared_ptr<LocalNode> localNode,
std::map<NodeID, SCPEnvelope> const& map,
uint32_t n)
{
return LocalNode::isVBlocking(
localNode->getQuorumSet(), map,
[&](SCPStatement const& st) { return statementBallotCounter(st) > n; });
}

// Step 9 from the paper (Feb 2016):
//
// If ∃ S ⊆ M such that the set of senders {v_m | m ∈ S} is v-blocking
// and ∀m ∈ S, b_m.n > b_v.n, then set b <- <n, z> where n is the lowest
// counter for which no such S exists.
//
// a.k.a 4th rule for setting ballot.counter in the internet-draft (v03):
//
// If nodes forming a blocking threshold all have ballot.counter values
// greater than the local ballot.counter, then the local node immediately
// cancels any pending timer, increases ballot.counter to the lowest
// value such that this is no longer the case, and if appropriate
// according to the rules above arms a new timer. Note that the blocking
// threshold may include ballots from SCPCommit messages as well as
// SCPExternalize messages, which implicitly have an infinite ballot
// counter.

bool
BallotProtocol::attemptBump()
{
if (mPhase == SCP_PHASE_PREPARE || mPhase == SCP_PHASE_CONFIRM)
{
uint32 targetCounter = mCurrentBallot ? mCurrentBallot->counter : 0;

// find all counters greater than targetCounter
// First check to see if this condition applies at all. If there
// is no v-blocking set ahead of the local node, there's nothing
// to do, return early.
auto localNode = getLocalNode();
uint32 localCounter = mCurrentBallot ? mCurrentBallot->counter : 0;
if (!hasVBlockingSubsetStrictlyAheadOf(localNode, mLatestEnvelopes,
localCounter))
{
return false;
}

// Collect all possible counters we might need to advance to.
std::set<uint32> allCounters;
for (auto const& e : mLatestEnvelopes)
{
auto const& st = e.second.statement;
uint32 eCounter;
switch (st.pledges.type())
{
case SCP_ST_PREPARE:
{
auto const& p = st.pledges.prepare();
eCounter = p.ballot.counter;
}
break;
case SCP_ST_CONFIRM:
{
auto const& c = st.pledges.confirm();
eCounter = c.ballot.counter;
}
break;
case SCP_ST_EXTERNALIZE:
{
eCounter = UINT32_MAX;
}
break;
default:
abort();
};
if (eCounter >= targetCounter)
{
allCounters.insert(eCounter);
}
uint32_t c = statementBallotCounter(e.second.statement);
if (c >= localCounter)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be strict inequality. By the time this line is reached, we have already determined (line 1392) that hasVBlockingSubsetStrictlyAheadOf(localNode, mLatestEnvelopes, localCounter) == true. Therefore the condition on line 1413 cannot be satisfied for n = localCounter, so it may as well be excluded from allCounters.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes agree, fixed

allCounters.insert(c);
}

// uses 0 as a way to track if a v-blocking set is at a higher counter
// if so, we move to that smallest counter
allCounters.insert(targetCounter);

// go through the counters, find the smallest not v-blocking
for (auto it = allCounters.begin(); it != allCounters.end(); it++)
// If we got to here, implicitly there _was_ a v-blocking subset
// with counters above the local counter; we just need to find a
// minimal n at which that's no longer true. So check them in
// order, starting from the smallest.
for (uint32_t n : allCounters)
{
uint32 n = *it;

bool vBlocking = LocalNode::isVBlocking(
getLocalNode()->getQuorumSet(), mLatestEnvelopes,
[&](SCPStatement const& st) {
bool res;
auto const& pl = st.pledges;
if (pl.type() == SCP_ST_PREPARE)
{
auto const& p = pl.prepare();
res = n < p.ballot.counter;
}
else
{
if (pl.type() == SCP_ST_CONFIRM)
{
res = n < pl.confirm().ballot.counter;
}
else
{
res = n != UINT32_MAX;
}
}
return res;
});

if (n == targetCounter)
if (!hasVBlockingSubsetStrictlyAheadOf(localNode, mLatestEnvelopes,
n))
{
// if current counter is not behind, don't do anything
if (!vBlocking)
{
break;
}
}
else
{
if (!vBlocking)
{
// move to n
return abandonBallot(n);
}
// Move to n.
return abandonBallot(n);
}
}
}

// We should only have got here if allCounters was empty; otherwise
// there should at worst have been some _maximal_ counter in the set to
// advance to, that by definition no v-blocking subset was "strictly
// ahead of".
assert(allCounters.empty());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is actually reachable. Suppose that there exists a v-blocking set strictly ahead of localCounter. Then there exists at least one element in mLatestEnvelopes with counter c > localCounter, because the empty set is not a v-blocking set. Therefore allCounters contains a counter c > localCounter; specifically allCounters is not empty. Now fix the highest counter C in allCounters. If there does not exist a v-blocking set strictly ahead of C, then this line cannot be reached and we are done. Suppose instead that there does exist a v-blocking set strictly ahead of C. Then, as before, there exists at least one element in mLatestEnvelopes with counter c > C, because the empty set is not a v-blocking set. But this implies that allCounters includes c > C, which contradicts that C is the highest counter in allCounters.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, this comment and assert are confusing, I am removing them. I could have put an assert(false) here but this doesn't seem to meet the bar for crashing consensus

}
return false;
}

Expand Down