Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use ConditionalWork for history replay in catchup #2194

Merged

Conversation

marta-lokhova
Copy link
Contributor

@marta-lokhova marta-lokhova commented Jul 11, 2019

Resolves #1951

@marta-lokhova marta-lokhova force-pushed the ledger_replay_conditional_work branch from 7800db1 to e1cd566 Compare July 11, 2019 16:56
@MonsieurNicolas MonsieurNicolas added this to In progress in v11.4.0 via automation Jul 12, 2019
if (mWorkStarted && !mConditionedWork->isDone())
{
mConditionedWork->crankWork();
return false;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does that mean that aborting started ConditionalWork is not possible?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what you mean: onAbort returns whether Work is done aborting or not, so that BasicWork can proceed appropriately. ConditionalWork will keep getting scheduled to run onAbort until mConditionedWork is done aborting. At that point, ConditionalWork is safe to abort itself.

// Conditioned work was aborted
REQUIRE(conditionedWork->getState() == BasicWork::State::WORK_ABORTED);
// Dependent work was never started (internally, in PENDING state,
// enforced by its destructor)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you mean constructor here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I mean destructor, which asserts that BasicWork is in proper state.

{
clock.crank();
}
CHECK(conditionedWork->getState() == BasicWork::State::WORK_WAITING);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That looks redundant.

mHdrIn.close();
mTxIn.close();
mFilesOpen = false;
}

void
ApplyLedgerChainWork::openCurrentInputFiles()
ApplyCheckpointWork::openCurrentInputFiles()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be renamed to openInputFiles. If this only works on one checkpoint there is only one set of files, so "current" is redundant.

@@ -45,7 +45,6 @@ BatchWork::doWork()
}

addMoreWorkIfNeeded();
mApp.getCatchupManager().logAndUpdateCatchupStatus(true);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

src/work/ConditionalWork.cpp Show resolved Hide resolved
LedgerRange const mRange;
uint32_t mCurrSeq;
uint32_t mCheckpoint;
// Apply checkpoint until given ledger. Otherwise, apply checkpoint in full
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pick a name that clarifies that this is a ledger number

LedgerHeaderHistoryEntry& lastApplied);
~ApplyLedgerChainWork() = default;
ApplyCheckpointWork(Application& app, TmpDir const& downloadDir,
uint32_t checkpoint, uint32_t boundary = 0);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like the use of 0 like this in a low level class as it's quite error prone.

I think it's probably better to keep a LedgerRange here, and force the caller to make a decision on what the upper range is.

This PR then just needs to enforce that that LedgerRange has the following properties:

  1. low is a checkpoint
  2. doesn't span multiple checkpoints

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed that it's safer, updated.

@marta-lokhova marta-lokhova force-pushed the ledger_replay_conditional_work branch 2 times, most recently from 20e3508 to 2063642 Compare August 15, 2019 21:06
Copy link
Contributor

@MonsieurNicolas MonsieurNicolas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's move all calls to logAndUpdateCatchupStatus from the hierarchy into CatchupWork::doWork

{
return State::WORK_SUCCESS;
mApp.getCatchupManager().logAndUpdateCatchupStatus(true);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd call logAndUpdateCatchupStatus regardless (before the if) - it seems arbitrary to only call it if result is true.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would also move the done case to its own it (probably done first) as to make the code easier to read (else will be deciding between running or failure)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

@MonsieurNicolas
Copy link
Contributor

r+ ad674d6

latobarita added a commit that referenced this pull request Aug 16, 2019
…_work

Use ConditionalWork for history replay in catchup

Reviewed-by: MonsieurNicolas
@latobarita latobarita merged commit ad674d6 into stellar:master Aug 16, 2019
v11.4.0 automation moved this from In progress to Done Aug 16, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
v11.4.0
  
Done
Development

Successfully merging this pull request may close these issues.

catchup: applying transactions should be done in the same work unit than downloading
4 participants