Prevent peer flooding inv request queue (redux) (redux) #7079

gmaxwell · 2015-11-23T02:13:30Z

Currently it's possible for peers to send duplicate invs of the same object and push the inv request time far back.

PR #4547 sought to address this, but was closed with the expectation that it would be replaced by the more comprehensive #4831 but the latter hasn't yet reached escape velocity. I agree a comprehensive rework is better, but I think we should not let a simple improvement wait for that.

PR #6306 tried a simpler version, but by reusing the setInventoryKnown it gets the logic somewhat wrong, e.g. not allowing duplicate INVs even for objects we'd mempool retired but now would be accept again.

Here I bring back #4547 and add size limiting and make an important change to the logic (which I believe was originally flawed, though no one seems to have commented on that in the original PR):

The earlier PR would remove the suppression as soon as the getdata was requested, rather than removing it when response was returned. This one removes only when we decide to not getdata (because we already have) or when the getdata returns. This means that a peer can only have one in-flight INV for a particular tx at once; which is what we want.

mapAlreadyAskedFor does not keep track of which peer has a request queued for a particular tx. As a result, a peer can blind a node to a tx indefinitely by sending many invs for the same tx, and then never replying to getdatas for it. Each inv received will be placed 2 minutes farther back in mapAlreadyAskedFor, so a short message containing 10 invs would render that tx unavailable for 20 minutes. This is fixed by disallowing a peer from having more than one entry for a particular inv in mapAlreadyAskedFor at a time.

…rns. The setAskFor duplicate elimination was too eager and removed entries when we still had no getdata response, allowing the peer to keep INVing and not responding.

laanwj · 2015-11-24T08:22:39Z

concept ACK

dgenr8 · 2015-11-25T02:16:38Z

e.g. not allowing duplicate INVs even for objects we'd mempool retired but now would be accept again.

That's a bug in CTxMemPool::Expire. It should clean the tx out of all setInventoryKnowns anyway, else we won't try to re-inv it to peers when it comes in again.

gmaxwell · 2015-11-25T05:05:19Z

@dgenr8 I don't think it is, or at least it's debatable... we won't reinv to peers we already INVed it to-- they may well still have it-- we should still reinv to anyone we haven't.

dcousens · 2015-11-25T09:09:59Z

concept ACK

dgenr8 · 2015-11-25T22:02:34Z

@gmaxwell Ah well, as long as the attack is mitigated. It's also possible to drop the timeout to 20s (from 2m) and only miss a response 1% of the time. .4% at 30s.

gmaxwell · 2015-11-25T22:35:14Z

The timeouts must be set very conservatively or otherwise a congested node (e.g. one under dos attack) can exhaust itself with repeated transmissions. You cannot count on the average case measured on a node with good connectivity.

I'm also not particularly interested if someone-- with a bunch of work-- can delay a transaction by a couple minutes; it's not a very interesting attack compared to self congestion collapse (which can happen absent any attacker at all). 2 minutes is the appropriate scaling from the 20 minute value used elsewhere, with respect to a the maximum size standard transaction being 1/10th the block size.

sipa · 2015-11-26T21:15:37Z

Untested ACK

laanwj · 2015-11-30T11:57:05Z

utACK

ebb25f4 Limit setAskFor and retire requested entries only when a getdata returns. (Gregory Maxwell) 5029698 prevent peer flooding request queue for an inv (kazcw)

mapAlreadyAskedFor does not keep track of which peer has a request queued for a particular tx. As a result, a peer can blind a node to a tx indefinitely by sending many invs for the same tx, and then never replying to getdatas for it. Each inv received will be placed 2 minutes farther back in mapAlreadyAskedFor, so a short message containing 10 invs would render that tx unavailable for 20 minutes. This is fixed by disallowing a peer from having more than one entry for a particular inv in mapAlreadyAskedFor at a time. Github-Pull: bitcoin#7079 Rebased-From: 5029698

…rns. The setAskFor duplicate elimination was too eager and removed entries when we still had no getdata response, allowing the peer to keep INVing and not responding. Github-Pull: bitcoin#7079 Rebased-From: ebb25f4

mapAlreadyAskedFor does not keep track of which peer has a request queued for a particular tx. As a result, a peer can blind a node to a tx indefinitely by sending many invs for the same tx, and then never replying to getdatas for it. Each inv received will be placed 2 minutes farther back in mapAlreadyAskedFor, so a short message containing 10 invs would render that tx unavailable for 20 minutes. This is fixed by disallowing a peer from having more than one entry for a particular inv in mapAlreadyAskedFor at a time. Github-Pull: bitcoin#7079 Rebased-From: 5029698

…rns. The setAskFor duplicate elimination was too eager and removed entries when we still had no getdata response, allowing the peer to keep INVing and not responding. Github-Pull: bitcoin#7079 Rebased-From: ebb25f4

Upstream patch: Prevent peer flooding inv request queue bitcoin/bitcoin#7079 5029698 ebb25f4

Merge some upstream commits that add better handling of the header/block download logic. Includes: Reduce download timeouts as blocks arrive bitcoin/bitcoin#5976 Prevent peer flooding inv request queue bitcoin/bitcoin#7079

fcdf87a Populate services in GetLocalAddress (Alex Morcos) a0a079f Return early in IsBanned. (Gregory Maxwell) 7772138 Remove redundant nullptr checks before deallocation (practicalswift) 5390862 net: remove unimplemented functions. (furszy) 74d0482 Limit setAskFor and retire requested entries only when a getdata returns. (Gregory Maxwell) 3b3bf63 prevent peer flooding request queue for an inv (kazcw) c814967 Fix subscript[0] potential bugs in key.cpp (Jeremy Rubin) 8e2e79e Remove unnecessary branches in utilstrencodings string constructors. (Jeremy Rubin) 25a16b3 Fix subscript[0] in utilstrencodings.cpp (Jeremy Rubin) df0584e Fix subscript[0] in streams.h (Jeremy Rubin) b7e64a5 Fix subscript[0] in validation.cpp (Jeremy Rubin) 2f7b73b Fix subscript[0] in torcontrol (Jeremy Rubin) 3b883eb Fix subscript[0] in netaddress.cpp (Jeremy Rubin) e5f5401 Fix subscript[0] in base58.cpp (Jeremy Rubin) 7d4ec87 Cleanup (safe, it was checked) subscript[0] in MurmurHash3 (and cleanup MurmurHash3 to be more clear). (Jeremy Rubin) 5e14f54 Fix subscript[0] in compressor.cpp (Jeremy Rubin) 48a622d Fix subscript[0] bug in net.cpp if GetGroup returns a 0-sized vector (Jeremy Rubin) af4cba3 net: use an internal address for fixed seeds (Cory Fields) 36368c5 net: switch to dummy internal ip for dns seed source (Cory Fields) 8f31295 net: do not allow resolving to an internal address (Cory Fields) 8ba3a40 net: add an internal subnet for representing unresolved hostnames (Cory Fields) c60b875 fix uninitialized read when stringifying an addrLocal (Kaz Wesley) 3d36540 add test demonstrating addrLocal UB (Kaz Wesley) Pull request description: Grouped some net updates (plus a miscellaneous one) coming from upstream. Part of another deep rabbit hole that i'm doing in parallel of the tier two work. PRs List: * bitcoin#7079. * bitcoin#9804. * bitcoin#10424 * bitcoin#10446. * bitcoin#10564. * bitcoin#14728. ACKs for top commit: random-zebra: Code ACK fcdf87a Tree-SHA512: ee81c834641aa6fdb9ca4396657457358a4b32f7862d60716e914dcfc2d355970937bd0fb4e164faaa0f4ea26d263ca8a2af4d8d5d6615b2db5bf89ec70d15f3

kazcw and others added 2 commits November 23, 2015 01:18

Limit setAskFor and retire requested entries only when a getdata retu…

ebb25f4

…rns. The setAskFor duplicate elimination was too eager and removed entries when we still had no getdata response, allowing the peer to keep INVing and not responding.

laanwj added the P2P label Nov 24, 2015

laanwj added this to the 0.12.0 milestone Nov 24, 2015

laanwj mentioned this pull request Nov 24, 2015

Prevent peer flooding inv request queue (redux) #6306

Closed

laanwj merged commit ebb25f4 into bitcoin:master Dec 1, 2015

laanwj added a commit that referenced this pull request Dec 1, 2015

Merge pull request #7079

1b5118b

ebb25f4 Limit setAskFor and retire requested entries only when a getdata returns. (Gregory Maxwell) 5029698 prevent peer flooding request queue for an inv (kazcw)

defuse mentioned this pull request Aug 16, 2016

Merge upstream anti DoS patches zcash/zcash#1251

Open

bitcartel mentioned this pull request Sep 16, 2016

Upstream patch: Prevent peer flooding inv request queue zcash/zcash#1407

Merged

zkbot pushed a commit to zcash/zcash that referenced this pull request Sep 20, 2016

Auto merge of #1407 - bitcartel:master_bitcoin_7079, r=daira

5ef7fec

Upstream patch: Prevent peer flooding inv request queue bitcoin/bitcoin#7079 5029698 ebb25f4

This was referenced Dec 27, 2016

large orphan transactions are left in askfor queue even when rejected #3774

Closed

requesting txs that have already been added to the pool #4461

Closed

adamjonas mentioned this pull request Oct 22, 2019

potential problem with nodes ignoring getdata requests #1157

Closed

furszy mentioned this pull request May 3, 2021

Net code updates [Step 2] PIVX-Project/PIVX#2359

Merged

bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Prevent peer flooding inv request queue (redux) (redux) #7079

Prevent peer flooding inv request queue (redux) (redux) #7079

Uh oh!

gmaxwell commented Nov 23, 2015

Uh oh!

laanwj commented Nov 24, 2015

Uh oh!

dgenr8 commented Nov 25, 2015

Uh oh!

gmaxwell commented Nov 25, 2015

Uh oh!

dcousens commented Nov 25, 2015

Uh oh!

dgenr8 commented Nov 25, 2015

Uh oh!

gmaxwell commented Nov 25, 2015

Uh oh!

sipa commented Nov 26, 2015

Uh oh!

laanwj commented Nov 30, 2015

Uh oh!

Uh oh!

Prevent peer flooding inv request queue (redux) (redux) #7079

Prevent peer flooding inv request queue (redux) (redux) #7079

Uh oh!

Conversation

gmaxwell commented Nov 23, 2015

Uh oh!

laanwj commented Nov 24, 2015

Uh oh!

dgenr8 commented Nov 25, 2015

Uh oh!

gmaxwell commented Nov 25, 2015

Uh oh!

dcousens commented Nov 25, 2015

Uh oh!

dgenr8 commented Nov 25, 2015

Uh oh!

gmaxwell commented Nov 25, 2015

Uh oh!

sipa commented Nov 26, 2015

Uh oh!

laanwj commented Nov 30, 2015

Uh oh!

Uh oh!