Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Separating tor switch context from normal for better privacy #4

Open
wants to merge 6 commits into
base: stable
Choose a base branch
from

Conversation

ujscale
Copy link

@ujscale ujscale commented Oct 4, 2023

This PR aims at minimizing the overlap of torswitch with the context of normal switch by separating peerpool, connection handling, queues, discoveryloop and attestation net finding.

#5

@ujscale
Copy link
Author

ujscale commented Oct 4, 2023

Succuessful run log: __ is a tor switch
The torpeerpool has greater number of available peers, but .publish seem to succeed on less than the number available.
You can also note that torswitch will avoid connecting with peers that are connected on normal and viceversa.

`connect successful, Switch type:(peerId: 16Uiu2HAky12PxRUE2w75yvp6CAAV6KqGLLYSZ8ivGYrHYAX4xwgt, addrs: @[/ip4/54.90.199.254/tcp/9000])
Peer pool10
Avoiding Overlap of tor with normal's (peerId: 16Uiu2HAm9t5zjrL37rJgfQJKxEBBCDHZnXcZPr5d4ukpFdg82FB4, addrs: @[/ip4/189.59.171.91/tcp/9001])
Avoiding Overlap of tor with normal's (peerId: 16Uiu2HAmCTReFja7ry2qRhiXNU6NKhMpjq7U1cTw4BceF2nFP2jM, addrs: @[/ip4/15.235.53.186/tcp/9000])
Connect successful, Switch type:
(peerId: 16Uiu2HAmLQsU2StpTcVRqmZuznEu63FAKEBUfR1dLCbPLJWWYX88, addrs: @[/ip4/3.26.243.80/tcp/9000])
Peer pool10
peers: 61 ❯ finalized: f7b5deb1:208056 ❯ head: c0fd2a82:208058:28 ❯ time: 208058:28 (6657884) ❯ sync: synced ETH: 32.012874914
Connect successful, Switch type:(peerId: 16Uiu2HAmCi4vQSRMET6HL4GCCm42cfGjyVghd1iVK2JwPWTcsViX, addrs: @[/ip4/20.228.188.128/tcp/9000])
Peer pool9
INF 2023-10-04 06:56:58.022+00:00 Slot end topics="beacnde" slot=6657884 nextActionWait=1s977ms210us877ns nextAttestationSlot=6657885 nextProposalSlot=-1 syncCommitteeDu
ties=none head=c0fd2a82:6657884
peers: 60 ❯ finalized: f7b5deb1:208056 ❯ head: c0fd2a82:208058:28 ❯ time: 208058:28 (6657884) ❯ sync: synced ETH: 32.012874914
Connect successful, Switch type:
(peerId: 16Uiu2HAmQw9Zb2EuxBFv3Dfbadak8WtxkZZvXmN4SGxQ1hLN19UC, addrs: @[/ip4/99.117.96.254/tcp/9505])
Peer pool8
INF 2023-10-04 06:57:00.000+00:00 Slot start topics="beacnde" slot=6657885 epoch=208058 sync=synced peers=60 head=c0fd2a82:6657884 finalized=208056:f7b5deb1 delay=700us378
ns
peers: 60 ❯ finalized: f7b5deb1:208056 ❯ head: 929f6434:208058:29 ❯ time: 208058:29 (6657885) ❯ sync: synced ETH: 32.012874914
about to send attestion
Peers from publish3
INF 2023-10-04 06:57:02.674+00:00 Attestation sent attestation="(aggregation_bits: 0b00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000, data: (slot: 6657885, index: 18, beac
on_block_root: "929f6434", source: "208057:c24e638c", target: "208058:7eee55c0"), signature: "b661fbee")" delay=-1s325ms456us874ns subnet_id=18
INF 2023-10-04 06:57:10.016+00:00 Slot end topics="beacnde" slot=6657885 nextActionWait=4m1s983ms722us906ns nextAttestationSlot=6657906 nextProposalSlot=-1 syncCommittee
Duties=none head=929f6434:6657885

true
# check if the normal switch is already using the same connection
# avoid any overlap with normal peer use to prevent leakage or correlation
if node.torpeerPool.hasPeer(peerId) :
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure this is a good idea, this could actually cause a correlation

Copy link
Author

@ujscale ujscale Oct 5, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The point is that both normal switch and tor switch are looking for attestation topic subnet peers! So we should have a way to avoid overlap as the attacker can build a correlation of active normal connections from peer with the concurrent connections through tor within the same time period! So that's why torswitch will check if normal is already connected with a peer that it should avoid making connection to and viceversa.

Ah, there could be a better way. Why not just keep attesstion_net peers specifc for torswitchh only. So the normal switch should not find attestation subnet peers at all or make any connections to them !

@Menduist
Copy link

Menduist commented Oct 4, 2023

The code duplication isn't great here, but that should work for a PoC

@ujscale ujscale self-assigned this Oct 5, 2023
@ujscale
Copy link
Author

ujscale commented Oct 5, 2023

The code duplication isn't great here, but that should work for a PoC

Well, there is a better way either to inherit from Eth2Node with a subclass TorEth2Node subclass that reuses all these functions or through template that use node switch typevariable. This refactoring could be done after we get new behaviours tested.

@ujscale
Copy link
Author

ujscale commented Oct 12, 2023

@tersec adding tersec

@ujscale ujscale requested review from Menduist and Terrytobin and removed request for Terrytobin October 12, 2023 05:42
@ujscale ujscale marked this pull request as ready for review October 27, 2023 04:03
@ujscale
Copy link
Author

ujscale commented Oct 27, 2023

See if its possible to add tersec and zahary to this.

@@ -71,19 +71,24 @@ type
wantedPeers*: int
hardMaxPeers*: int
peerPool*: PeerPool[Peer, PeerId]
torpeerPool*:PeerPool[Peer, PeerId]
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generic, as it were, comment relating to all of these foo and torfoo pairs: they create code duplication, which can be avoided by having an entire Tor switch, so it's be torSwitch.peerPool, etc.

As usual, it depends what the goals here are, how they're balanced between research and deployability.

But more or less all the remaining hundreds of lines of code changes here are duplications of functions resulting from this design decision.

proc torcheckPeer(node: Eth2Node, peerAddr: PeerAddr): bool =
  logScope: peer = peerAddr.peerId
  let peerId = peerAddr.peerId
  if node.torpeerPool.hasPeer(peerId):
    trace "Already connected over tor"
    false
  else:
    if node.istorSeen(peerId):
      trace "Recently connected over tor"
      false
    else:
      # check if the normal switch is already using the same connection
      # avoid any overlap with normal peer use to prevent leakage or correlation
      if node.peerPool.hasPeer(peerId) : 
        echo "Avoiding Overlap of tor with normal's ", peerAddr
        false
      else:
        true

isn't meaningfully different from checkPeer except that it refers to torpeerPool, rather than peerPool, and calls istorSeen rather than isSeen.

There's some notion of that it has to know about the peerPool too, but in general, there are more precise ways to satisfy most of these constraints.

So this is the question: as research code, sure, why not.

I'd push against incorporating this design into actual-nimbus-eth2, if that were proposed.

declareGauge nbc_torgossipsub_healthy_fanout,
"numbers of topics over tor with dHigh fanout"


Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extra newline? Doesn't really matter, maybe intentional (group declareGauges, etc), but wasn't there before in that capacity.

Trivial nit, sure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants