-
Notifications
You must be signed in to change notification settings - Fork 186
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Filter for nodes that we gossip to #443
Comments
@raulk may I start working on a PR for this? This is a very useful feature that we would like to use soon :) |
@vyzo Can you help me transfer this issue to the pubsub repo? |
done; also, you don't need raul's permission to start working on this :) |
@vyzo I was also wondering if you had any thoughts on how this could be implemented cleanly? Originally I intended to do something in a similar vein to the topic validator, but all of the topic validation logic actually occurs in For the selective message filtering however, it will need to happen inside The way the code is currently structured, it would probably need to go somewhere around here. I think this is not ideal because we need to implement it separately for each Instead, I was thinking of adding the logic in What do you think about this? The one catch with this approach is that I would need some way to get the Also, the above still does not entirely solve the problem for my specific use case. I want to be able to prevent all information about a certain message's existence from passing on to certain peers, including IHAVE messages, and in particular my criteria for choosing these messages is based on the message's original sender. In other words, I make the decision based on who the message came from, and who the information would be going to. However, IHAVEs are stored as message ID's in the RPC, so just by looking at an RPC alone I cannot tell which IHAVE's I need to remove. |
I dont think putting it in handleSendingMessages is correct, the right
place to put it in the peer selection.
Simplest way is to have a filter of the form `func(topic, peer) bool` which
we apply inside the peer selection for gossip.
For publishing messages, a similar (or the same) filter can be applied, in
peer selection for mesh construction and flood publishing selection.
The default would be a func that returns true unconditionally.
Note: its very easy to get the peer for a stream, `s.Conn().RemotePeer()`
…On Fri, Aug 13, 2021, 12:11 Simon Zhu ***@***.***> wrote:
@vyzo <https://github.com/vyzo> I was also wondering if you had any
thoughts on how this could be implemented cleanly? Originally I intended to
do something in a similar vein to the topic validator, but all of the topic
validation logic actually occurs in pubsub.go (and validation.go), before
Publish
<https://github.com/libp2p/go-libp2p-pubsub/blob/master/gossipsub.go#L943>
is called.
For the selective message filtering however, it will need to happen inside
Publish, because we will need to run the filter per-peer.
The way the code is currently structured, it would probably need to go
somewhere around here
<https://github.com/libp2p/go-libp2p-pubsub/blob/master/gossipsub.go#L1008>.
I think this is not ideal because we need to implement it separately for
each Router implementation, even though the logic should basically be
identitcal.
Instead, I was thinking of adding the logic in handleSendingMessages
<https://github.com/libp2p/go-libp2p-pubsub/blob/master/comm.go#L155>,
which removes all the information from the message that we care about
before sending it out. This could potentially be made to be even more
general purpose than my usecase here, ie a function that allows you to
modify each message that is sent out.
What do you think about this? The one catch with this approach is that I
would need some way to get the peer.ID associated with a Stream
<https://github.com/libp2p/go-libp2p-swarm>, which I don't think is
possible today. So maybe I'd have to work on an issue for that as well.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#443 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAI4SREAPOTFZLBQ4RPZNDT4TOTVANCNFSM5CDAY44A>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email>
.
|
Note that the filter would be in the pubsub object, initialzed with a
pubsub option, and thus available to each router.
We want the router to decide how to use it, there is no one size fits all.
…On Fri, Aug 13, 2021, 13:08 Dimitris Vyzovitis ***@***.***> wrote:
I dont think putting it in handleSendingMessages is correct, the right
place to put it in the peer selection.
Simplest way is to have a filter of the form `func(topic, peer) bool`
which we apply inside the peer selection for gossip.
For publishing messages, a similar (or the same) filter can be applied, in
peer selection for mesh construction and flood publishing selection.
The default would be a func that returns true unconditionally.
Note: its very easy to get the peer for a stream, `s.Conn().RemotePeer()`
On Fri, Aug 13, 2021, 12:11 Simon Zhu ***@***.***> wrote:
> @vyzo <https://github.com/vyzo> I was also wondering if you had any
> thoughts on how this could be implemented cleanly? Originally I intended to
> do something in a similar vein to the topic validator, but all of the topic
> validation logic actually occurs in pubsub.go (and validation.go),
> before Publish
> <https://github.com/libp2p/go-libp2p-pubsub/blob/master/gossipsub.go#L943>
> is called.
>
> For the selective message filtering however, it will need to happen
> inside Publish, because we will need to run the filter per-peer.
>
> The way the code is currently structured, it would probably need to go
> somewhere around here
> <https://github.com/libp2p/go-libp2p-pubsub/blob/master/gossipsub.go#L1008>.
> I think this is not ideal because we need to implement it separately for
> each Router implementation, even though the logic should basically be
> identitcal.
>
> Instead, I was thinking of adding the logic in handleSendingMessages
> <https://github.com/libp2p/go-libp2p-pubsub/blob/master/comm.go#L155>,
> which removes all the information from the message that we care about
> before sending it out. This could potentially be made to be even more
> general purpose than my usecase here, ie a function that allows you to
> modify each message that is sent out.
>
> What do you think about this? The one catch with this approach is that I
> would need some way to get the peer.ID associated with a Stream
> <https://github.com/libp2p/go-libp2p-swarm>, which I don't think is
> possible today. So maybe I'd have to work on an issue for that as well.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <#443 (comment)>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAAI4SREAPOTFZLBQ4RPZNDT4TOTVANCNFSM5CDAY44A>
> .
> Triage notifications on the go with GitHub Mobile for iOS
> <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
> or Android
> <https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email>
> .
>
|
@vyzo Cool, my original concern was that I didn't think we wanted each router to decide how to use it, and that there should be a common way to handle this. But I see now that may not actually be the case, so I have a good idea how to get this done in that case. Thanks! |
I noticed that topic validators are not initialized with a pubsub option, but rather via I have no current use case for unregistering my peer selection filter, but do you want me to support this? |
Actually, I'm still not entirely sure that this is resolved? Implementing things at the peer selection level will allow me to filter out peers that I forward a particular message to, but it would not prevent me from later gossipping about that message to a filtered out peer or responding to an IWANT for that message from the peer, right? |
Context: https://discuss.libp2p.io/t/filtering-nodes-that-we-gossip-to/1012
We need a mechanism that allows us to provide a validator function that, given a message, filters out nodes that we don’t want to forward or gossip about the message to.
In other words, we want to implement selective gossip / message forwarding.
The text was updated successfully, but these errors were encountered: