feat(iroh)!: Configurable path selection#4232
Conversation
|
Documentation for this PR has been generated and is available at: https://n0-computer.github.io/iroh/pr/4232/docs/iroh/ Last updated: 2026-05-22T15:17:32Z |
|
@mcginty could you take a look at this? The current path selection is a bit limited especially once you have custom transports, but really also if you have more complex scenarios involving just ip and relay transports. This is my attempt to make the selection API more flexible. |
0a79c3c to
7f35601
Compare
| /// has chosen. When iroh wants to make this configurable per non-relay transport, this | ||
| /// will move to a method on [`PathSelector`] with a default body that matches the rule | ||
| /// here. | ||
| fn path_status_for(addr: &transports::Addr) -> noq_proto::PathStatus { |
There was a problem hiding this comment.
Yeah, we can merge this one after yours. This just has to live somewhere to make it self-contained for now.
| /// | ||
| /// The same address may appear more than once when it is a path on multiple | ||
| /// connections to the remote. Selectors that care should aggregate as appropriate. | ||
| pub fn paths(&self) -> impl Iterator<Item = PathSelectionData<'a>> + '_ { |
There was a problem hiding this comment.
So this iterates over all connections, then over all paths, and yield PathSelectionData for each. This is a bit weird, you can get the same path multiple times with different stats but no further details. Maybe we should just deduplicate and take the min RTT among all path stats for a transport addr?
There was a problem hiding this comment.
That is how it worked before, but then you make the assumption of min RTT is best before giving the items to the filter/selector, and the whole point of this mechanism is to not make such assumptions.
- rename add to set (for now) - don't use full path for BiasedRttPathSelector - rename addr to remote_addr - change how the pub feature gating works - log the path and rtt during selection
That way the way the selected path is identified is an implementation detail.
…4234) ## Description This is the part of #4232 that removes something from the non feature gated public API. If (as seems likely) we won't get #4232 in for this release, this would prevent having to break the stable (non feature gated) public API when adding #4232 in the next release. ## Breaking Changes Removes some public methods on the endpoint builder that are too opinionated if you want to make path selection fully generic: endpoint::Builder::transport_bias: removed from the public api endpoint::transports::TransportBias: removed from the public api ## Notes & open questions Note: I did a quick check in all notable custom transports I am aware of: n0-computer/iroh-tor-transport n0-computer/iroh-nym-transport mcginty/iroh-ble-transport None of them does use transport_bias, not even in examples. Most of them just remove the ip transport. So this won't break any downstream crate I am aware of. ## Change checklist <!-- Remove any that are not relevant. --> - [ ] Self-review. - [ ] Documentation updates following the [style guide](https://rust-lang.github.io/rfcs/1574-more-api-documentation-conventions.html#appendix-a-full-conventions-text), if relevant. - [ ] Tests if relevant. - [ ] All breaking changes documented. - [ ] List all breaking changes in the above "Breaking Changes" section. - [ ] Open an issue or PR on any number0 repos that are affected by this breaking change. Give guidance on how the updates should be handled or do the actual updates themselves. The major ones are: - [ ] [`quic-rpc`](https://github.com/n0-computer/quic-rpc) - [ ] [`iroh-gossip`](https://github.com/n0-computer/iroh-gossip) - [ ] [`iroh-blobs`](https://github.com/n0-computer/iroh-blobs) - [ ] [`dumbpipe`](https://github.com/n0-computer/dumbpipe) - [ ] [`sendme`](https://github.com/n0-computer/sendme)
|
ci and merge conflicts are very sad |
|
I added a bit of code to write tests for a selector easy without a full endpoint, just to test the selection logic in isolation: PathSource::Test / StatsSource::Test Not entirely happy with this rube goldberg machine, but I don't see a way to avoid it that isn't either involving dynamic dispatch and/or boxing. And it is not in the public API, so maybe it's fine? Without this you would be unable to test a path selector in isolation. |
|
One more thing: we could simplify the BiasedRttPathSelector now that it is no longer supposed to be generic, e.g. hardcore the IPV6 bias again. But if we do it it has time for a subsequent PR. |
Use a new `transport::FourTuple` throughout the send path. Adds a src argument to CustomSender::poll_send. Replaces the recently added `TransportFourTuple` with the new `transports::FourTuple`. Motivations: * We noted that it would be nicer if the TransportFourTuple introduced in #4273 would be an enum, so that we cannot have mismatches between the transport kinds of local and remote addresses. I agree. What prevented it was that we had to convert to `transports::Addr` many times, which is not free. * I noticed that custom transports don't get a `src: Option<CustomAddr>` in their `poll_send`. I think it should be there, because custom transports *can* set a CustomAddr on which incoming packets were received, so to mirror the features of the IP transport they should be able to pin to an interface when replying as well, if the transport has a notion of multiple interfaces This PR fixes both by replacing the `remote_state::TransportFourTuple` with a `transports::FourTuple` that is used throughout the send path, including custom transports. * `iroh::endpoint::transports::CustomSender::poll_send` now has an additional arg `src: Option<&CustomAddr>` <!-- Any notes, remarks or open questions you have to make about the PR. --> <!-- Remove any that are not relevant. --> - [x] Self-review. - [ ] Documentation updates following the [style guide](https://rust-lang.github.io/rfcs/1574-more-api-documentation-conventions.html#appendix-a-full-conventions-text), if relevant. - [x] Tests if relevant. - [x] All breaking changes documented.
|
This is the output diff between pub struct iroh::endpoint::transports::PathSelection
impl iroh::endpoint::transports::PathSelection
pub fn iroh::endpoint::transports::PathSelection::none() -> Self
pub fn iroh::endpoint::transports::PathSelection::set(&mut self, path: &iroh::endpoint::transports::PathSelectionData<'_>)
pub struct iroh::endpoint::transports::PathSelectionContext<'a>
impl<'a> iroh::endpoint::transports::PathSelectionContext<'a>
pub fn iroh::endpoint::transports::PathSelectionContext<'a>::current(&self) -> core::option::Option<&iroh::socket::transports::FourTuple>
pub fn iroh::endpoint::transports::PathSelectionContext<'a>::paths(&self) -> alloc::boxed::Box<(dyn core::iter::traits::iterator::Iterator<Item = iroh::endpoint::transports::PathSelectionData<'a>> + '_)>
pub struct iroh::endpoint::transports::PathSelectionData<'a>
impl<'a> iroh::endpoint::transports::PathSelectionData<'a>
pub fn iroh::endpoint::transports::PathSelectionData<'a>::network_path(&self) -> &iroh::socket::transports::FourTuple
pub fn iroh::endpoint::transports::PathSelectionData<'a>::stats(&self) -> core::option::Option<noq_proto::connection::stats::PathStats>
pub trait iroh::endpoint::transports::PathSelector: core::marker::Send + core::marker::Sync + core::fmt::Debug + 'static
pub fn iroh::endpoint::transports::PathSelector::select(&self, ctx: &iroh::endpoint::transports::PathSelectionContext<'_>) -> iroh::endpoint::transports::PathSelection
pub fn iroh::endpoint::Builder::path_selector(self, selector: alloc::sync::Arc<dyn iroh::endpoint::transports::PathSelector>) -> Self |
Description
An attempt to make path selection more flexible for complex custom transport use cases.
Path selection is now via a dynable trait
PathSelector. The only fn select takes aPathSelectionContextthat has the current path as well as a way to iterate over a flat list ofPathSelectionDatastructs.PathSelectionDatacurrently just has the address and the stats, but in the future could be extended to contain more detailed information such as the congestion controller metricsnoq_proto::ControllerMetricsetc. I think cc metrics would be quite helpful to make good decisions about path selection.selectreturns a PathSelection struct that is at this time a newtype over an Option. add is first writer wins, all subsequent calls will ignore the addr and emit a warning. But we can change this in the future to allow multiple addrs.The entire mechanism is only pub under the
unstable-custom-transportsflag, so we can change it in the future. But there are some additive changes we could do without breakage. For example we could allow PathSelection to also keep track of paths to be closed.Breaking Changes
Removes some public methods on the endpoint builder that are too opinionated if you want to make path selection fully generic:
endpoint::Builder::transport_bias(kind, bias): removedendpoint::transports::TransportBias: removedAnd some changes involving the unstable_custom_transport API.
Notes & open questions
Note: the new signature tries to anticipate that in the future we might want to select multiple transports, but as of now it will just ignore these.
Change checklist
quic-rpciroh-gossipiroh-blobsdumbpipesendme