Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[net] Cleanup connection types- followups #19724

Merged
merged 10 commits into from
Sep 3, 2020

Conversation

amitiuttarwar
Copy link
Contributor

@amitiuttarwar amitiuttarwar commented Aug 14, 2020

This PR addresses outstanding review comments from #19316. It further simplifies net.cpp complexity and adds documentation about design goals about different connection types.

@amitiuttarwar
Copy link
Contributor Author

amitiuttarwar commented Aug 14, 2020

I'm not 100% convinced about replacing IsAddrRelayPeer() with RelayAddrsWithConn() 9345f9c.

Pros:

  • It is more direct to check the connection type instead of a (missing) data structure based on the connection type.
  • Easy to update logic in the future
  • Less mental overhead, less code touch points to glean intention.

Cons:

  • We are now checking the same conditional in two places- [1] & [2], and need to manually ensure they do not fall out of sync.

Curious to hear what other reviewers think is preferable.

@amitiuttarwar amitiuttarwar changed the title [net] Cleanup connection type- followups [net] Cleanup connection types- followups Aug 14, 2020
@amitiuttarwar amitiuttarwar force-pushed the 2020-08-conn-refactor-followups branch from 899f3b7 to b672f8d Compare August 14, 2020 23:43
@sipa
Copy link
Member

sipa commented Aug 14, 2020

@amitiuttarwar I think it makes sense to make the addr-relay decision depend on the connection type. That should be the most authoritative information about the connection. Sure, it's duplication, but if we accidentally forget to create the necessary data structures, test will fail; if we accidentally create them if they're not needed, at worst we've wasted a bit of memory we're already ok with for other nodes.

@maflcko maflcko removed the Tests label Aug 15, 2020
src/net.cpp Outdated Show resolved Hide resolved
@@ -2433,9 +2433,8 @@ void ProcessMessage(
UpdatePreferredDownload(pfrom, State(pfrom.GetId()));
}

if (!pfrom.IsInboundConn() && pfrom.IsAddrRelayPeer())
if (pfrom.AdvertiseAddressToConn())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're cleaning this up, we should really clean it up and move the connman.MarkAddressGood() out of this if statement, since it's not related to address advertising at all. Having it in the same conditional is a trap for developers to fall into.

(specifically, a developer might think "we don't need to advertise our address to FEELER connections, since we disconnect before we actually send the ADDR or GETADDR messages, so we can remove FEELER from AdvertiseAddressToConn", but doing that breaks our FEELER connection logic, because MarkAddressGood() needs to be called to mark the address as good)

cc @sdaftuar

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was looking at this more and I believe we should leave the MarkAddressGood call in the if-block, though it needs an explanation— the reason (I think) is that we don’t want block-relay connections to leave a trace in the addrman that could be leaked by an attacker sniffing our addr responses.

So this just needs a comment explaining why it’s in here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain this a bit? I think you're saying that if we connect out to a peer that we use for block-relay-only we don't want to mark it as good (essentially move it from New to Tried and update the last successful connect time) because then another peer might be able to tell that we've connected to them?

My understanding of how we respond to getaddr messages is that we'll respond with (up to) 1000 records taken at random from all records, regardless of whether they're in New or Tried, so marking a peer as good won't change whether we include it in addr responses.

Copy link
Member

@sdaftuar sdaftuar Aug 16, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have a great understanding of the details of how addrman works or how we interact with it -- so possibly the fear was unfounded. It used to be the case that you could trivially download a listening node's addrman though, and it also used to be the case (perhaps only long ago?) that we'd leak information about current connections based on time stamps stored in our addrman, so when I worked on #15759 I tried to leave the addrman state unchanged when making block-relay-only connections.

I'm not sure that I fully succeeded in not leaking information, or that the concern is correct to begin with given how addrman already works. So it's possible that with some analysis we could make improvements here. Maybe not appropriate for this pr though...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe there are three ways that a peer's record is updated in addrman when we're connected to them:

  1. when we receive a version message from them, we'll mark their address good. That moves the address from the new table to the tried table, and updates the nLastSuccess/nLastTry/nAttempts fields. We only do this if the peer isn't inbound and isn't block-relay-only. I don't believe this change can easily be detected by a peer sending us getaddr messages.
  2. if a peer self-advertises to us, we'll call AddAddressKnown(), which could update the nTime, which is included in the address record that we send to a peer in response to a getaddr request.
  3. when we disconnect from the peer, we'll update the nTime, as long as the peer was marked as fCurrentlyConnected. That happens when we receive a verack from a peer which is not inbound. This doesn't include feeler peers, since we disconnect from them as soon as we receive a version message from them. The nTime is only updated during disconnection so spies can't find out who we're currently connected to ( Reduce fingerprinting through timestamps in 'addr' messages. #5860 (comment)).

In summary, I think block-relay-only peers should have their address marked good when we connect to them (1). If we don't, then we'll update the nLastTry and nAttempts fields when we attempt to connect to them, but not update the nLastSuccess.

The reason it's important in this PR is that placing MarkAddressGood() behind a conditional if (AdvertiseAddressToConn()) is very confusing for anyone reading the code, and makes it easy to introduce subtle bugs.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we're taking the wrong approach in the PR as it is right now. At least that condition should be split in 2 blocks:

I think the current code change here is not useful, but only distracts:
I) The comment Advertise our address is removed for no reason
II) The comment (unrelated to advertising our address) is very confusing. I managed to understand it only after reading this discussion
III) The comment Moves address from New to Tried table in Addrman is not sufficient
IV) The change of IsAddrRelayPeer to IsBlockOnlyConn here is very confusing, the two blocks should be split into 2 very separate blocks according to the following reasoning:

  1. We self-announce to every peer that supports self-announcement AND it's a good idea:
    1a) not useful to self-announce to feelers as we won't actually send ADDR to them (i think?)
    1b) bad to self-announce to block-relay-only (maybe? wrt privacy)
  2. We MarkAddressGood according to the logic we have

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

w.r.t block-relay-only peers and AddrMan, I agree it's a separate issue, so i'd prefer to preserve the approach status quo here.

The current PR currently satisfies it, but I'm still unhappy with the code changes in this particular area (see the previous comment).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@naumenkogs Would it be enough to just add some more comments to this block? I think it's fine to leave as one block until we actually need to split it, so I'd propose something like this:

diff --git a/src/net_processing.cpp b/src/net_processing.cpp
index 7cd4f7c4cb5..4285e8ba898 100644
--- a/src/net_processing.cpp
+++ b/src/net_processing.cpp
@@ -2427,6 +2427,21 @@ void PeerLogicValidation::ProcessMessage(CNode& pfrom, const std::string& msg_ty
         }
 
         if (!pfrom.IsInboundConn() && !pfrom.IsBlockOnlyConn()) {
+            // For outbound peers, we try to relay our address (so that other
+            // nodes can try to find us more quickly, as we have no guarantee
+            // that an outbound peer is even aware of how to reach us) and do a
+            // one-time address fetch (to help populate/update our addrman). If
+            // we're starting up for the first time, our addrman may be pretty
+            // empty and no one will know who we are.
+            //
+            // We also will update the addrman to record connection success for
+            // these peers (which include OUTBOUND_FULL_RELAY peers and FEELER
+            // connections) so that addrman will have an up-to-date notion of
+            // which peers are online and available.
+            //
+            // We skip these operations for BLOCK_RELAY peers to avoid
+            // potentially leaking information about our BLOCK_RELAY
+            // connections via the addrman or address relay.
             if (fListen && !::ChainstateActive().IsInitialBlockDownload())
             {
                 CAddress addr = GetLocalAddress(&pfrom.addr, pfrom.GetLocalServices());
@@ -2446,8 +2461,8 @@ void PeerLogicValidation::ProcessMessage(CNode& pfrom, const std::string& msg_ty
             m_connman.PushMessage(&pfrom, CNetMsgMaker(nSendVersion).Make(NetMsgType::GETADDR));
             pfrom.fGetAddr = true;
 
-            // Moves address from New to Tried table in Addrman
-            // (unrelated to advertising our address)
+            // Moves address from New to Tried table in Addrman, resolves
+            // tried-table collisions, etc.
             m_connman.MarkAddressGood(pfrom.addr);
         }

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd suggest making If we're starting up for the first time, our addrman may be pretty empty and no one will know who we are. less ambiguous:
If we're starting up for the first time, it would improve our peering chances. Otherwise, our addrman may be pretty empty, and also no one will know who we are.

But generally, I think this comment:

  • makes it much more clear
  • actually convinces me that it's an improvement over the existent code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

incorporated these comments into the changes,

I agree this should be further improved, but I'm hoping this is sufficient for this PR. @naumenkogs are you comfortable with the current state?

@jnewbery
Copy link
Contributor

Strong strong concept ACK.

I think it'd be good to extract everything in the while (!interruptNet) in ThreadOpenConnections() into its own function since deeply nested while/for/if blocks obscure control flow and are very often the sources of bugs. Doing that might be scope creep for this PR though.

Copy link
Contributor

@jnewbery jnewbery left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK everything except the AdvertiseAddressToConn() change. I'd much rather leave the conditional as

if (!pfrom.IsInboundConn() && !pfrom.IsBlockOnlyConn())

and add a large comment to above the connman.MarkAddressGood(pfrom.addr); to say that this isn't anything to do with sending an addr or getaddr message.

src/net.h Outdated Show resolved Hide resolved
src/net.h Outdated Show resolved Hide resolved
src/net.h Outdated Show resolved Hide resolved
@DrahtBot
Copy link
Contributor

DrahtBot commented Aug 20, 2020

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@amitiuttarwar
Copy link
Contributor Author

amitiuttarwar commented Aug 21, 2020

thanks for the feedback! all comments should be addressed.

@sipa
I agree with this:

make the addr-relay decision depend on the connection type. That should be the most authoritative information about the connection.

But unfortunately I don't think this is true:

but if we accidentally forget to create the necessary data structures, test will fail;

I updated RelayAddrsWithConn to always return true, recompiled & ran test_runner.py & unit tests. Nothing failed. My concern would be addressed if it did :) I think with #19315, we should be able to add functional tests that would fail if a discrepancy occurs?

Currently I think the downside is probably acceptable, but want to highlight since I don't think we have any automatic protection in place. (unless I'm missing something?)


@jnewbery, RE:

I think it'd be good to extract everything in the while (!interruptNet) in ThreadOpenConnections() into its own function since deeply nested while/for/if blocks obscure control flow and are very often the sources of bugs. Doing that might be scope creep for this PR though.

I 100% agree. I spent so many hours trying to interpret the exact logic around what types of connections are open, and have gotten very confused by the deeply nested logic flows & nuances of things like where a break / continue that's 4 levels deep will pop you back out to. I've taken a first pass at extracting the outer while (!interruptNet) loop here (yes, there are two levels nested): amitiuttarwar@eecc587. However, it needs to be done with great care and I would need to spend a lot more time / build a lot more confidence with these changes before I'd feel comfortable proposing them. Feel free to take a look or further the branch if you're interested :)

@jnewbery
Copy link
Contributor

utACK c99b260

have gotten very confused by the deeply nested logic flows & nuances of things like where a break / continue that's 4 levels deep will pop you back out to.

Yes, deeply nested if/while/switch statements hide bugs. If it's possible to factor them out to clarify control flow, we should aim to do that.

I've taken a first pass at extracting the outer while (!interruptNet) loop here

I've had a very quick skim of that branch and it looks like the right direction to me.

I would need to spend a lot more time / build a lot more confidence with these changes before I'd feel comfortable proposing them. Feel free to take a look or further the branch if you're interested :)

I think you're very well positioned to open this PR if you want to. You've already spent hours getting to grips with the logic here 🙂 . I've already got my quota of refactor PRs open, but I can definitely commit to reviewing a PR if you open it. (Equally, we don't need to fix this all at once. We can always leave this for another time).

src/net_processing.cpp Show resolved Hide resolved
src/net.cpp Outdated Show resolved Hide resolved
src/net.cpp Outdated Show resolved Hide resolved
src/net.cpp Show resolved Hide resolved
src/net.h Show resolved Hide resolved
src/net.h Outdated
* open MAX_BLOCK_RELAY_ONLY_CONNECTIONS to help prevent against partition
* attacks. By not relaying transactions or addresses, these connections
* are harder to detect by a third party, thus helping obfuscate the
* network topology. Addresses are selected from AddrMan.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can mark that addresses for {BLOCK_RELAY,OUTBOUND_FULL_RELAY} are drawn from tried table.

Also block-relay network topology.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are they always? what happens if your tried table isn't populated yet ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah sorry, in fact they are drawn from both new/tried tables at anytime. Surely being confused by SelectTriedCollision in net.cpp path!

src/net.h Show resolved Hide resolved
src/net.h Show resolved Hide resolved
src/net.h Show resolved Hide resolved
src/net_processing.cpp Outdated Show resolved Hide resolved
@jonatack
Copy link
Member

Concept ACK on these changes. Late to the party but will try to review soon-ish.

@amitiuttarwar amitiuttarwar force-pushed the 2020-08-conn-refactor-followups branch from c99b260 to 4c66cc0 Compare August 27, 2020 19:48
@amitiuttarwar
Copy link
Contributor Author

thanks for the review @jnewbery & @ariard ! all review comments are addressed

rebased & made some small changes to docs

-BEGIN VERIFY SCRIPT-
sed -i 's/OUTBOUND, /OUTBOUND_FULL_RELAY, /g' src/net.h
sed -i 's/ConnectionType::OUTBOUND/ConnectionType::OUTBOUND_FULL_RELAY/g' src/test/net_tests.cpp src/test/fuzz/process_message.cpp src/test/fuzz/process_messages.cpp src/net.cpp src/test/denialofservice_tests.cpp src/net.h src/test/fuzz/net.cpp
-END VERIFY SCRIPT-
amitiuttarwar and others added 8 commits September 2, 2020 17:18
We previously identified if we relay addresses to the connection by checking
for the existence of the m_addr_known data structure. With this commit, we
answer this question based on the connection type.

IsAddrRelayPeer() checked for the existence of the m_addr_known
Co-authored-by: Suhas Daftuar <sdaftuar@gmail.com>
Previously we deduced it was a block-relay-only based on presence of the
m_tx_relay structure. Now we have the ability to identify it directly via a
connection type accessor function.
Consolidate the logic to determine connection type into one conditional to
clarify how they are chosen.
@amitiuttarwar
Copy link
Contributor Author

amitiuttarwar commented Sep 3, 2020

thank you for the reviews!

good catch on the silent merge conflict @sdaftuar, I’ve rebased to address. (turns out, there was more than one)

I've addressed all review comments. They were getting pretty web-like, so I've tried my best to resolve and focus on any outstanding conversations.

It looks like all of the remaining review comments are on style/documentation. Perhaps we should aim to merge this and address those in a follow-up?

Most of the changes in this commit are style/documentation. It's not like it's a big logic-changing PR with a lot of ACKs will have to invalidate :)

I'd like to get this PR merged sooner rather than later for a couple reasons:

  • I’d like to get the renames on master to avoid possibility of more silent merge conflicts
  • there are now two PRs that build on this (Periodically make block-relay connections and sync headers #19858 & p2p: Improve diversification of new connections #19860). further improvements can be addressed in parallel rather than holding up the whole sequence.
  • I think its important not to discourage review / lose momentum. @jnewbery has ACKed the tip twice already & I’ve had to invalidate because of needing rebase. with 10 commits deep, it doesn't seem trivial to reACK, so I'd rather use that review energy on smaller isolated changes that can be handled in another PR.

just trying to express that if I do get ACKs, I'm going to try to be cautious about invalidating, so might opt for starting another branch/PR if more non-essential changes are requested.

@amitiuttarwar
Copy link
Contributor Author

there's one failing build, but I'm confused by it. nothing in the logs appears to have failed??
https://travis-ci.org/github/bitcoin/bitcoin/jobs/723629947

@RandyMcMillan
Copy link
Contributor

@amitiuttarwar - This PR is based on parent commits that aren't passing - the fail may have nothing to do with your change. I recommend rebasing your commit on a recent passing commit.

@naumenkogs
Copy link
Member

ACK eb1c5d0

@laanwj
Copy link
Member

laanwj commented Sep 3, 2020

Code review ACK eb1c5d0

@ariard
Copy link
Contributor

ariard commented Sep 3, 2020

post-merge Code Review ACK eb1c5d0

Fabcien pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this pull request Jul 13, 2021
…BOUND_FULL_RELAY

Summary:
```
sed -i 's/OUTBOUND,/OUTBOUND_FULL_RELAY,/g' src/net.h
sed -i 's/ConnectionType::OUTBOUND/ConnectionType::OUTBOUND_FULL_RELAY/g' src/test/net_tests.cpp src/test/fuzz/process_message.cpp src/test/fuzz/process_messages.cpp src/net.cpp src/test/denialofservice_tests.cpp src/net.h src/test/fuzz/net.cpp src/avalanche/test/processor_tests.cpp
arc lint
```

This is a backport of [[bitcoin/bitcoin#19724 | core#19724]] [1/9]
bitcoin/bitcoin@8d6ff46

Test Plan: `ninja all check-all`

Reviewers: #bitcoin_abc, majcosta

Reviewed By: #bitcoin_abc, majcosta

Differential Revision: https://reviews.bitcoinabc.org/D9770
Fabcien pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this pull request Jul 13, 2021
…onnection

Summary:
This is a backport of [[bitcoin/bitcoin#19724 | core#19724]] [2/9]
bitcoin/bitcoin@a6ab1e8

Depends on D9770

Test Plan: `ninja all check-all`

Reviewers: #bitcoin_abc, majcosta

Reviewed By: #bitcoin_abc, majcosta

Differential Revision: https://reviews.bitcoinabc.org/D9771
Fabcien pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this pull request Jul 13, 2021
Summary:
We previously identified if we relay addresses to the connection by checking
for the existence of the m_addr_known data structure. With this commit, we
answer this question based on the connection type.

IsAddrRelayPeer() checked for the existence of the m_addr_known

This is a backport of [[bitcoin/bitcoin#19724 | core#19724]] [3/9]
bitcoin/bitcoin@dff16b1

Depends on D9771

Test Plan: `ninja all check-all`

Reviewers: #bitcoin_abc, majcosta

Reviewed By: #bitcoin_abc, majcosta

Subscribers: majcosta

Differential Revision: https://reviews.bitcoinabc.org/D9772
Fabcien pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this pull request Jul 13, 2021
Summary:
Co-authored-by: Suhas Daftuar <sdaftuar@gmail.com>
This is a backport of [[bitcoin/bitcoin#19724 | core#19724]] [4/9]
bitcoin/bitcoin@ff6b908

Depends on D9772

Test Plan: Proof-reading new comments

Reviewers: #bitcoin_abc, majcosta

Reviewed By: #bitcoin_abc, majcosta

Differential Revision: https://reviews.bitcoinabc.org/D9773
Fabcien pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this pull request Jul 13, 2021
…et correctly

Summary:
This is a backport of [[bitcoin/bitcoin#19724 | core#19724]] [5/9]
bitcoin/bitcoin@da3a0be

Depends on D9773

Test Plan: `ninja check`

Reviewers: #bitcoin_abc, majcosta

Reviewed By: #bitcoin_abc, majcosta

Differential Revision: https://reviews.bitcoinabc.org/D9774
Fabcien pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this pull request Jul 13, 2021
…tion.

Summary:
Previously we deduced it was a block-relay-only based on presence of the
m_tx_relay structure. Now we have the ability to identify it directly via a
connection type accessor function.

This is a backport of [[bitcoin/bitcoin#19724 | core#19724]] [6/9]
bitcoin/bitcoin@1e563ae

Depends on D9774

Test Plan: `ninja all check-all`

Reviewers: #bitcoin_abc, majcosta, Fabien

Reviewed By: #bitcoin_abc, majcosta, Fabien

Subscribers: majcosta

Differential Revision: https://reviews.bitcoinabc.org/D9775
Fabcien pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this pull request Jul 13, 2021
…enConnections

Summary:
Consolidate the logic to determine connection type into one conditional to
clarify how they are chosen.

This is a backport of [[bitcoin/bitcoin#19724 | core#19724]] [7/9]
bitcoin/bitcoin@4829b6f

Depends on D9775

Test Plan: `ninja all check-all`

Reviewers: #bitcoin_abc, majcosta

Reviewed By: #bitcoin_abc, majcosta

Differential Revision: https://reviews.bitcoinabc.org/D9776
Fabcien pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this pull request Jul 13, 2021
Summary:
This is a backport of [[bitcoin/bitcoin#19724 | core#19724]] [8/9]
bitcoin/bitcoin@d5a57ce

Depends on D9776

Test Plan: Proof reading

Reviewers: #bitcoin_abc, majcosta

Reviewed By: #bitcoin_abc, majcosta

Subscribers: majcosta

Differential Revision: https://reviews.bitcoinabc.org/D9777
deadalnix pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this pull request Jul 13, 2021
…ng default.

Summary:
This concludes backport of [[bitcoin/bitcoin#19724 | core#19724]] [9/9]
bitcoin/bitcoin@eb1c5d0

Depends on D9777

Test Plan: Proof-reading

Reviewers: #bitcoin_abc, majcosta

Reviewed By: #bitcoin_abc, majcosta

Differential Revision: https://reviews.bitcoinabc.org/D9778
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Feb 15, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.