Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Filter for nodes that we gossip to #443

Open
synzhu opened this issue Aug 12, 2021 · 9 comments
Open

Filter for nodes that we gossip to #443

synzhu opened this issue Aug 12, 2021 · 9 comments

Comments

@synzhu
Copy link
Contributor

synzhu commented Aug 12, 2021

Context: https://discuss.libp2p.io/t/filtering-nodes-that-we-gossip-to/1012

We need a mechanism that allows us to provide a validator function that, given a message, filters out nodes that we don’t want to forward or gossip about the message to.

In other words, we want to implement selective gossip / message forwarding.

@synzhu
Copy link
Contributor Author

synzhu commented Aug 13, 2021

@raulk may I start working on a PR for this? This is a very useful feature that we would like to use soon :)

@synzhu
Copy link
Contributor Author

synzhu commented Aug 13, 2021

@vyzo Can you help me transfer this issue to the pubsub repo?

@vyzo vyzo transferred this issue from libp2p/go-libp2p Aug 13, 2021
@vyzo
Copy link
Collaborator

vyzo commented Aug 13, 2021

done; also, you don't need raul's permission to start working on this :)

@synzhu
Copy link
Contributor Author

synzhu commented Aug 13, 2021

@vyzo I was also wondering if you had any thoughts on how this could be implemented cleanly? Originally I intended to do something in a similar vein to the topic validator, but all of the topic validation logic actually occurs in pubsub.go (and validation.go), before Publish is called.

For the selective message filtering however, it will need to happen inside Publish, because we will need to run the filter per-peer.

The way the code is currently structured, it would probably need to go somewhere around here. I think this is not ideal because we need to implement it separately for each Router implementation, even though the logic should basically be identitcal.

Instead, I was thinking of adding the logic in handleSendingMessages, which allows us to removes all the information from the message that we care about before sending it out. This could potentially be made to be even more general purpose than my usecase here, ie a function that allows you to modify each message right before it is sent out.

What do you think about this? The one catch with this approach is that I would need some way to get the peer.ID associated with a Stream so that it can be passed to the filter function, which I don't think is possible today. So maybe I'd have to work on an issue for that as well. Either way, this is actually a functionality we need for a different usecase anyways.

Also, the above still does not entirely solve the problem for my specific use case. I want to be able to prevent all information about a certain message's existence from passing on to certain peers, including IHAVE messages, and in particular my criteria for choosing these messages is based on the message's original sender.

In other words, I make the decision based on who the message came from, and who the information would be going to.

However, IHAVEs are stored as message ID's in the RPC, so just by looking at an RPC alone I cannot tell which IHAVE's I need to remove.
If I have something like the above implemented, I could probably get away with remembering this data at the application level and looking at it when my filter function is called, but this is a bit hacky.

@vyzo
Copy link
Collaborator

vyzo commented Aug 13, 2021 via email

@vyzo
Copy link
Collaborator

vyzo commented Aug 13, 2021 via email

@synzhu
Copy link
Contributor Author

synzhu commented Aug 13, 2021

@vyzo Cool, my original concern was that I didn't think we wanted each router to decide how to use it, and that there should be a common way to handle this. But I see now that may not actually be the case, so I have a good idea how to get this done in that case. Thanks!

@synzhu
Copy link
Contributor Author

synzhu commented Aug 13, 2021

Note that the filter would be in the pubsub object, initialzed with a pubsub option, and thus available to each router.

I noticed that topic validators are not initialized with a pubsub option, but rather via RegisterTopicValidator. I assume this is in order to support dynamic register and unregister.

I have no current use case for unregistering my peer selection filter, but do you want me to support this?

@synzhu
Copy link
Contributor Author

synzhu commented Aug 13, 2021

Also, the above still does not entirely solve the problem for my specific use case. I want to be able to prevent all information about a certain message's existence from passing on to certain peers, including IHAVE messages, and in particular my criteria for choosing these messages is based on the message's original sender.

In other words, I make the decision based on who the message came from, and who the information would be going to.

However, IHAVEs are stored as message ID's in the RPC, so just by looking at an RPC alone I cannot tell which IHAVE's I need to remove.
If I have something like the above implemented, I could probably get away with remembering this data at the application level and looking at it when my filter function is called, but this is a bit hacky.

Actually, I'm still not entirely sure that this is resolved? Implementing things at the peer selection level will allow me to filter out peers that I forward a particular message to, but it would not prevent me from later gossipping about that message to a filtered out peer or responding to an IWANT for that message from the peer, right?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants