already have block - causing bandwidth to be wasted - vicious circle? #1120

Closed
rebroad opened this Issue Apr 17, 2012 · 12 comments

Comments

Projects
None yet
3 participants
@rebroad
Contributor

rebroad commented Apr 17, 2012

It occurs to me that this happens as the node asks several other nodes for the same blocks when it does not see the block appear within a certain time. Due to the speed of the nodes providing blocks, they both respond with the blocks requested, but one will send the block first and usually stay ahead. This isn't a major issue, but it does mean that bandwidth is wasted, and since it's probably the bandwidth which is causing the problem in the first place, I expect this will continue to get worse due to being a vicious circle. My proposed solution probably requires a BIP, which would allow a node to request to another node to stop sending blocks (i.e. to the node sending the blocks which it already has). It could do this after one duplicate block, or perhaps after several.

Sample case:

04/17/12 22:08:49 askfor block 000000000000096fd80c 0
04/17/12 22:08:50 sending getdata: block 000000000000096fd80c
04/17/12 22:14:12 askfor block 000000000000096fd80c 1334700530000000
04/17/12 22:14:12 sending getdata: block 000000000000096fd80c
04/17/12 22:16:39 received block 000000000000096fd80c
04/17/12 22:16:40 SetBestChain: new best=000000000000096fd80c height=176035 work=29615794588444859740
04/17/12 22:20:07 received block 00000000000009856606
04/17/12 22:20:07 ERROR: ProcessBlock() : already have block 176020 00000000000009856606
04/17/12 22:20:56 received block 00000000000008862df7
04/17/12 22:20:56 ERROR: ProcessBlock() : already have block 176021 00000000000008862df7
04/17/12 22:21:36 received block 000000000000035a84ac
04/17/12 22:21:36 ERROR: ProcessBlock() : already have block 176022 000000000000035a84ac
04/17/12 22:21:59 received block 0000000000000a881c26
04/17/12 22:21:59 ERROR: ProcessBlock() : already have block 176023 0000000000000a881c26
04/17/12 22:22:18 received block 00000000000004ed81e6
04/17/12 22:22:18 ERROR: ProcessBlock() : already have block 176024 00000000000004ed81e6
04/17/12 22:22:52 received block 000000000000068bbe48
04/17/12 22:22:52 ERROR: ProcessBlock() : already have block 176025 000000000000068bbe48
04/17/12 22:23:12 received block 00000000000003425f71
04/17/12 22:23:12 ERROR: ProcessBlock() : already have block 176026 00000000000003425f71
04/17/12 22:23:39 received block 00000000000000f89962
04/17/12 22:23:39 ERROR: ProcessBlock() : already have block 176027 00000000000000f89962
04/17/12 22:23:48 received block 000000000000083ec278
04/17/12 22:23:48 ERROR: ProcessBlock() : already have block 176028 000000000000083ec278
04/17/12 22:24:06 received block 000000000000088984c6
04/17/12 22:24:06 ERROR: ProcessBlock() : already have block 176029 000000000000088984c6
04/17/12 22:24:29 received block 00000000000002857e8b
04/17/12 22:24:29 ERROR: ProcessBlock() : already have block 176030 00000000000002857e8b
04/17/12 22:24:45 received block 000000000000075834ed
04/17/12 22:24:45 ERROR: ProcessBlock() : already have block 176031 000000000000075834ed
04/17/12 22:25:10 received block 000000000000033b7b3e
04/17/12 22:25:10 ERROR: ProcessBlock() : already have block 176032 000000000000033b7b3e
04/17/12 22:25:56 received block 00000000000001f32a95
04/17/12 22:25:56 ERROR: ProcessBlock() : already have block 176033 00000000000001f32a95
04/17/12 22:26:20 received block 00000000000004dffd6d
04/17/12 22:26:20 ERROR: ProcessBlock() : already have block 176034 00000000000004dffd6d
04/17/12 22:26:20 received block 000000000000096fd80c
04/17/12 22:26:20 ERROR: ProcessBlock() : already have block 176035 000000000000096fd80c

Much of this output has been cut, but if I grep my debug.log for "already have", I can see that blocks 174591 to 176061 (1470 blocks and counting) were being sent by two nodes.

@sipa

This comment has been minimized.

Show comment Hide comment
@sipa

sipa Apr 17, 2012

Member

Is this unmodified source code?

Member

sipa commented Apr 17, 2012

Is this unmodified source code?

@rebroad

This comment has been minimized.

Show comment Hide comment
@rebroad

rebroad Apr 17, 2012

Contributor

@sipa, yes, stock 0.6.0.6-beta, running on Windows 7, using tor.

Contributor

rebroad commented Apr 17, 2012

@sipa, yes, stock 0.6.0.6-beta, running on Windows 7, using tor.

@rebroad

This comment has been minimized.

Show comment Hide comment
@rebroad

rebroad Apr 28, 2012

Contributor

My proposal for solving this would be to introduce a "stop sending me blocks" command, or a "I already had those blocks" command, which would cause the sending node to stop sending. For backwards compatibility, old nodes would ignore these commands, so nodes would give them a chance to stop, and if they continued to send duplicate blocks (perhaps 6 already haves in a row) they would simply disconnect from the node in question.

Contributor

rebroad commented Apr 28, 2012

My proposal for solving this would be to introduce a "stop sending me blocks" command, or a "I already had those blocks" command, which would cause the sending node to stop sending. For backwards compatibility, old nodes would ignore these commands, so nodes would give them a chance to stop, and if they continued to send duplicate blocks (perhaps 6 already haves in a row) they would simply disconnect from the node in question.

@rebroad

This comment has been minimized.

Show comment Hide comment
@rebroad

rebroad Apr 28, 2012

Contributor

I can also confirm that this problem exists in a recent version of litecoin.

Contributor

rebroad commented Apr 28, 2012

I can also confirm that this problem exists in a recent version of litecoin.

@rebroad

This comment has been minimized.

Show comment Hide comment
@rebroad

rebroad Apr 28, 2012

Contributor

Is it possible for ProcessMessages() to determine which block is being sent before it's finished downloading it? Perhaps get advance notification of its hash rather than wait for the block to completely download before realising it's wasted time and bandwidth receiving it?

Contributor

rebroad commented Apr 28, 2012

Is it possible for ProcessMessages() to determine which block is being sent before it's finished downloading it? Perhaps get advance notification of its hash rather than wait for the block to completely download before realising it's wasted time and bandwidth receiving it?

@sipa

This comment has been minimized.

Show comment Hide comment
@sipa

sipa Apr 28, 2012

Member

That would require some structural changes to the code, and I'm not sure it's necessary. I'd much rather like to know why your node requests blocks twice.

Member

sipa commented Apr 28, 2012

That would require some structural changes to the code, and I'm not sure it's necessary. I'd much rather like to know why your node requests blocks twice.

@rebroad

This comment has been minimized.

Show comment Hide comment
@rebroad

rebroad Apr 28, 2012

Contributor

Isn't it standard behaviour that it requests the same block more than once if the nodes requested do not respond within a certain time period?

Contributor

rebroad commented Apr 28, 2012

Isn't it standard behaviour that it requests the same block more than once if the nodes requested do not respond within a certain time period?

@rebroad

This comment has been minimized.

Show comment Hide comment
@rebroad

rebroad Apr 29, 2012

Contributor

I've added some debugging code to work out what's going on, and I've noticed that in ProcessMessages() that the blocks seem to get stuck downloading. The vRecv.size() sometimes never reaches nMessageSize. i.e. every time ProcessMessages() is called, the vRecv.size() is still the same as last time for the same node. I'm pretty sure my added code for debugging isn't the cause of this.

I'll upload this code as soon as I obtain sufficient intelligence to navigate git....

Contributor

rebroad commented Apr 29, 2012

I've added some debugging code to work out what's going on, and I've noticed that in ProcessMessages() that the blocks seem to get stuck downloading. The vRecv.size() sometimes never reaches nMessageSize. i.e. every time ProcessMessages() is called, the vRecv.size() is still the same as last time for the same node. I'm pretty sure my added code for debugging isn't the cause of this.

I'll upload this code as soon as I obtain sufficient intelligence to navigate git....

@rebroad

This comment has been minimized.

Show comment Hide comment
@rebroad

rebroad May 7, 2012

Contributor

One solution to this problem is to add the ability to include a checksum/hash of the block in the block header, so the node can reject it without downloading it if it already has it. Only one other solution I can think of, other than putting it with receiving blocks that the node already has, but which took so long to arrive that the node ended up requesting it elsewhere.

The other solution I can think of is to enable ProcessBlock to be threaded, as it may be due to this causing nodes to take a long time to respond to messages that's causing them to time-out and ask elsewhere.

Contributor

rebroad commented May 7, 2012

One solution to this problem is to add the ability to include a checksum/hash of the block in the block header, so the node can reject it without downloading it if it already has it. Only one other solution I can think of, other than putting it with receiving blocks that the node already has, but which took so long to arrive that the node ended up requesting it elsewhere.

The other solution I can think of is to enable ProcessBlock to be threaded, as it may be due to this causing nodes to take a long time to respond to messages that's causing them to time-out and ask elsewhere.

@rebroad

This comment has been minimized.

Show comment Hide comment
@rebroad

rebroad May 12, 2013

Contributor

I'd like to understand exactly why pull request #1382 was closed. The issue was that nodes were being disconnected when they were simply doing as they were told - but what is the alternative to this? How do we avoid receiving duplicate blocks? The only way I see this is to NEVER request the same block from another node unless the nodes previously requested from have disconnected. Surely there can be circumstances in which this could take a very long time for these nodes to disconnect, during which time only orphan blocks can be requested, and the blockchain would appear stuck during this time. Or am I missing something?

Contributor

rebroad commented May 12, 2013

I'd like to understand exactly why pull request #1382 was closed. The issue was that nodes were being disconnected when they were simply doing as they were told - but what is the alternative to this? How do we avoid receiving duplicate blocks? The only way I see this is to NEVER request the same block from another node unless the nodes previously requested from have disconnected. Surely there can be circumstances in which this could take a very long time for these nodes to disconnect, during which time only orphan blocks can be requested, and the blockchain would appear stuck during this time. Or am I missing something?

@sipa

This comment has been minimized.

Show comment Hide comment
@sipa

sipa May 12, 2013

Member

@rebroad If a node is slow in responding, you need to switch to another peer for downloading. If a node is really slow, you can choose to disconnect him. If a node doesn't respond at all, he'll timeout after one minute anyway. But without logic to at least prevent unintentional double requests, disconnecting a peer simply because they do what you asked seems pretty much cutting in your own flesh.

Member

sipa commented May 12, 2013

@rebroad If a node is slow in responding, you need to switch to another peer for downloading. If a node is really slow, you can choose to disconnect him. If a node doesn't respond at all, he'll timeout after one minute anyway. But without logic to at least prevent unintentional double requests, disconnecting a peer simply because they do what you asked seems pretty much cutting in your own flesh.

@laanwj

This comment has been minimized.

Show comment Hide comment
@laanwj

laanwj Oct 26, 2013

Member

Closing, as it "works as intended". Pull requests to speed up block downloading are on the way and could use testing, see #2964.

Member

laanwj commented Oct 26, 2013

Closing, as it "works as intended". Pull requests to speed up block downloading are on the way and could use testing, see #2964.

@laanwj laanwj closed this Oct 26, 2013

suprnurd pushed a commit to chaincoin-legacy/chaincoin that referenced this issue Dec 5, 2017

refactoring PS, cleanup, more log output in debug mode (#1121)
* refactoring PS, cleanup, more log output in debug mode (part of #1120)

* fix dsq rate limit calculations on new dsq
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment