Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

INDY-2053: Implement new CATCHUP_REQ distribution logic #1173

Merged
merged 39 commits into from
Apr 26, 2019

Conversation

skhoroshavin
Copy link
Member

No description provided.

Sergey Khoroshavin added 18 commits April 17, 2019 16:12
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
plenum/server/catchup/catchup_rep_service.py Outdated Show resolved Hide resolved
final_size=cons_proof.seqNoEnd,
final_hash=cons_proof.newMerkleRoot) if cons_proof else None

nodes_ledger_sizes = {frm: proof.seqNoEnd
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't we just filter nodes according to the following simple rule:
get all nodes we have ConsistencyProof from with a seqNoEndnot less thanseqNoEnd we choice for catchup?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we do such filtering then most probably we'll filter out all nodes that have less transactions than majority of pool, severely limiting number of nodes participating in catch up. With strategy implemented we actually give chance to all nodes that have new transactions to participate.

plenum/server/catchup/node_leecher_service.py Show resolved Hide resolved
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
plenum/server/catchup/node_leecher_service.py Outdated Show resolved Hide resolved
plenum/server/catchup/catchup_rep_service.py Outdated Show resolved Hide resolved
plenum/server/catchup/catchup_rep_service.py Outdated Show resolved Hide resolved
logger.info("{} creating catchup request {} to {} till {}".format(self, s, e, end))
cReqs.append(req)
return missing
def send_reqs_for_missing(frm, to):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add unit tests for this

plenum/server/catchup/catchup_rep_service.py Show resolved Hide resolved
Andrew Nikitin and others added 9 commits April 18, 2019 15:34
Signed-off-by: Andrew Nikitin <andrew.nikitin@dsr-corporation.com>
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
Sergey Khoroshavin added 2 commits April 25, 2019 14:26
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
ashcherbakov
ashcherbakov previously approved these changes Apr 25, 2019
Copy link
Contributor

@ashcherbakov ashcherbakov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm comfortable with the changes, but it would be great to write the mentioned unit and integration tests.

Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
skhoroshavin and others added 8 commits April 26, 2019 12:19
[INDY-2053] tests for CATHUP_REP from lagged_nodes
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
Signed-off-by: Sergey Khoroshavin <sergey.khoroshavin@dsr-corporation.com>
@skhoroshavin skhoroshavin merged commit 50c03ce into hyperledger:master Apr 26, 2019
@skhoroshavin skhoroshavin deleted the indy-2053 branch April 26, 2019 14:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants