Skip to content

feat(iroh)!: Configurable path selection#4232

Merged
matheus23 merged 27 commits into
mainfrom
configurable-path-selection
May 22, 2026
Merged

feat(iroh)!: Configurable path selection#4232
matheus23 merged 27 commits into
mainfrom
configurable-path-selection

Conversation

@rklaehn
Copy link
Copy Markdown
Contributor

@rklaehn rklaehn commented May 5, 2026

Description

An attempt to make path selection more flexible for complex custom transport use cases.

Path selection is now via a dynable trait PathSelector. The only fn select takes a PathSelectionContext that has the current path as well as a way to iterate over a flat list of PathSelectionData structs.

PathSelectionData currently just has the address and the stats, but in the future could be extended to contain more detailed information such as the congestion controller metrics noq_proto::ControllerMetrics etc. I think cc metrics would be quite helpful to make good decisions about path selection.

select returns a PathSelection struct that is at this time a newtype over an Option. add is first writer wins, all subsequent calls will ignore the addr and emit a warning. But we can change this in the future to allow multiple addrs.

The entire mechanism is only pub under the unstable-custom-transports flag, so we can change it in the future. But there are some additive changes we could do without breakage. For example we could allow PathSelection to also keep track of paths to be closed.

Breaking Changes

Removes some public methods on the endpoint builder that are too opinionated if you want to make path selection fully generic:

endpoint::Builder::transport_bias(kind, bias): removed
endpoint::transports::TransportBias: removed

And some changes involving the unstable_custom_transport API.

Notes & open questions

Note: the new signature tries to anticipate that in the future we might want to select multiple transports, but as of now it will just ignore these.

Change checklist

  • Self-review.
  • Documentation updates following the style guide, if relevant.
  • Tests if relevant.
  • All breaking changes documented.
    • List all breaking changes in the above "Breaking Changes" section.
    • Open an issue or PR on any number0 repos that are affected by this breaking change. Give guidance on how the updates should be handled or do the actual updates themselves. The major ones are:

@n0bot n0bot Bot added this to iroh May 5, 2026
@github-project-automation github-project-automation Bot moved this to 🚑 Needs Triage in iroh May 5, 2026
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 6, 2026

Documentation for this PR has been generated and is available at: https://n0-computer.github.io/iroh/pr/4232/docs/iroh/

Last updated: 2026-05-22T15:17:32Z

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 6, 2026

Netsim report & logs for this PR have been generated and is available at: LOGS
This report will remain available for 3 days.

Last updated for commit: bebf141

@rklaehn rklaehn requested a review from Frando May 6, 2026 09:18
@rklaehn
Copy link
Copy Markdown
Contributor Author

rklaehn commented May 6, 2026

@mcginty could you take a look at this? The current path selection is a bit limited especially once you have custom transports, but really also if you have more complex scenarios involving just ip and relay transports.

This is my attempt to make the selection API more flexible.

@rklaehn rklaehn changed the title feat(iroh)! Configurable path selection (WIP) feat(iroh)! Configurable path selection May 6, 2026
@rklaehn rklaehn force-pushed the configurable-path-selection branch from 0a79c3c to 7f35601 Compare May 6, 2026 09:49
@rklaehn rklaehn changed the title feat(iroh)! Configurable path selection feat(iroh)!: Configurable path selection May 6, 2026
Comment thread iroh/src/socket/remote_map/remote_state.rs Outdated
/// has chosen. When iroh wants to make this configurable per non-relay transport, this
/// will move to a method on [`PathSelector`] with a default body that matches the rule
/// here.
fn path_status_for(addr: &transports::Addr) -> noq_proto::PathStatus {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think this is the wrong logic, see #4233

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, we can merge this one after yours. This just has to live somewhere to make it self-contained for now.

Comment thread iroh/src/socket/remote_map/remote_state.rs Outdated
Comment thread iroh/src/socket/remote_map/remote_state.rs Outdated
///
/// The same address may appear more than once when it is a path on multiple
/// connections to the remote. Selectors that care should aggregate as appropriate.
pub fn paths(&self) -> impl Iterator<Item = PathSelectionData<'a>> + '_ {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this iterates over all connections, then over all paths, and yield PathSelectionData for each. This is a bit weird, you can get the same path multiple times with different stats but no further details. Maybe we should just deduplicate and take the min RTT among all path stats for a transport addr?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is how it worked before, but then you make the assumption of min RTT is best before giving the items to the filter/selector, and the whole point of this mechanism is to not make such assumptions.

Comment thread iroh/src/socket/remote_map/remote_state.rs Outdated
Comment thread iroh/src/socket/remote_map/remote_state.rs Outdated
Comment thread iroh/src/socket/remote_map/remote_state.rs Outdated
Comment thread iroh/src/socket/remote_map/remote_state.rs Outdated
Comment thread iroh/src/socket/remote_map/remote_state.rs Outdated
Comment thread iroh/src/socket/remote_map/remote_state.rs Outdated
rklaehn added 4 commits May 6, 2026 17:04
- rename add to set (for now)
- don't use full path for BiasedRttPathSelector
- rename addr to remote_addr
- change how the pub feature gating works
- log the path and rtt during selection
That way the way the selected path is identified is an implementation detail.
rklaehn added a commit that referenced this pull request May 6, 2026
…4234)

## Description

This is the part of #4232 that
removes something from the non feature gated public API.

If (as seems likely) we won't get
#4232 in for this release, this
would prevent having to break the stable (non feature gated) public API
when adding #4232 in the next
release.

## Breaking Changes

Removes some public methods on the endpoint builder that are too
opinionated if you want to make path selection fully generic:

endpoint::Builder::transport_bias: removed from the public api
endpoint::transports::TransportBias: removed from the public api

## Notes & open questions

Note: I did a quick check in all notable custom transports I am aware
of:

n0-computer/iroh-tor-transport
n0-computer/iroh-nym-transport
mcginty/iroh-ble-transport

None of them does use transport_bias, not even in examples. Most of them
just remove the ip transport. So this won't break any downstream crate I
am aware of.

## Change checklist
<!-- Remove any that are not relevant. -->
- [ ] Self-review.
- [ ] Documentation updates following the [style
guide](https://rust-lang.github.io/rfcs/1574-more-api-documentation-conventions.html#appendix-a-full-conventions-text),
if relevant.
- [ ] Tests if relevant.
- [ ] All breaking changes documented.
- [ ] List all breaking changes in the above "Breaking Changes" section.
- [ ] Open an issue or PR on any number0 repos that are affected by this
breaking change. Give guidance on how the updates should be handled or
do the actual updates themselves. The major ones are:
    - [ ] [`quic-rpc`](https://github.com/n0-computer/quic-rpc)
    - [ ] [`iroh-gossip`](https://github.com/n0-computer/iroh-gossip)
    - [ ] [`iroh-blobs`](https://github.com/n0-computer/iroh-blobs)
    - [ ] [`dumbpipe`](https://github.com/n0-computer/dumbpipe)
    - [ ] [`sendme`](https://github.com/n0-computer/sendme)
@dignifiedquire
Copy link
Copy Markdown
Contributor

ci and merge conflicts are very sad

@dignifiedquire dignifiedquire moved this from 🚑 Needs Triage to 🏗 In progress in iroh May 7, 2026
@dignifiedquire dignifiedquire added this to the v1.0.0-rc.1 milestone May 7, 2026
@rklaehn rklaehn moved this from 🏗 In progress to 👀 In review in iroh May 19, 2026
@rklaehn
Copy link
Copy Markdown
Contributor Author

rklaehn commented May 19, 2026

I added a bit of code to write tests for a selector easy without a full endpoint, just to test the selection logic in isolation:

PathSource::Test / StatsSource::Test

Not entirely happy with this rube goldberg machine, but I don't see a way to avoid it that isn't either involving dynamic dispatch and/or boxing. And it is not in the public API, so maybe it's fine?

Without this you would be unable to test a path selector in isolation.

@rklaehn
Copy link
Copy Markdown
Contributor Author

rklaehn commented May 19, 2026

One more thing: we could simplify the BiasedRttPathSelector now that it is no longer supposed to be generic, e.g. hardcore the IPV6 bias again. But if we do it it has time for a subsequent PR.

Comment thread iroh/src/socket/biased_rtt_path_selector.rs Outdated
Comment thread iroh/src/socket/biased_rtt_path_selector.rs
Comment thread iroh/src/socket/remote_map/remote_state.rs Outdated
Frando added 3 commits May 22, 2026 16:18
Use a new `transport::FourTuple` throughout the send path. Adds a src
argument to CustomSender::poll_send. Replaces the recently added
`TransportFourTuple` with the new `transports::FourTuple`.

Motivations:
* We noted that it would be nicer if the TransportFourTuple introduced
in #4273 would be an enum, so that we cannot have mismatches between the
transport kinds of local and remote addresses. I agree. What prevented
it was that we had to convert to `transports::Addr` many times, which is
not free.
* I noticed that custom transports don't get a `src: Option<CustomAddr>`
in their `poll_send`. I think it should be there, because custom
transports *can* set a CustomAddr on which incoming packets were
received, so to mirror the features of the IP transport they should be
able to pin to an interface when replying as well, if the transport has
a notion of multiple interfaces

This PR fixes both by replacing the `remote_state::TransportFourTuple`
with a `transports::FourTuple` that is used throughout the send path,
including custom transports.

* `iroh::endpoint::transports::CustomSender::poll_send` now has an
additional arg `src: Option<&CustomAddr>`

<!-- Any notes, remarks or open questions you have to make about the PR.
-->

<!-- Remove any that are not relevant. -->
- [x] Self-review.
- [ ] Documentation updates following the [style
guide](https://rust-lang.github.io/rfcs/1574-more-api-documentation-conventions.html#appendix-a-full-conventions-text),
if relevant.
- [x] Tests if relevant.
- [x] All breaking changes documented.
@matheus23
Copy link
Copy Markdown
Member

matheus23 commented May 22, 2026

This is the output diff between cargo public-api -p iroh --features unstable-custom-transports -sss runs I can see between main and this PR (the diff is purely added lines)

pub struct iroh::endpoint::transports::PathSelection
impl iroh::endpoint::transports::PathSelection
pub fn iroh::endpoint::transports::PathSelection::none() -> Self
pub fn iroh::endpoint::transports::PathSelection::set(&mut self, path: &iroh::endpoint::transports::PathSelectionData<'_>)
pub struct iroh::endpoint::transports::PathSelectionContext<'a>
impl<'a> iroh::endpoint::transports::PathSelectionContext<'a>
pub fn iroh::endpoint::transports::PathSelectionContext<'a>::current(&self) -> core::option::Option<&iroh::socket::transports::FourTuple>
pub fn iroh::endpoint::transports::PathSelectionContext<'a>::paths(&self) -> alloc::boxed::Box<(dyn core::iter::traits::iterator::Iterator<Item = iroh::endpoint::transports::PathSelectionData<'a>> + '_)>
pub struct iroh::endpoint::transports::PathSelectionData<'a>
impl<'a> iroh::endpoint::transports::PathSelectionData<'a>
pub fn iroh::endpoint::transports::PathSelectionData<'a>::network_path(&self) -> &iroh::socket::transports::FourTuple
pub fn iroh::endpoint::transports::PathSelectionData<'a>::stats(&self) -> core::option::Option<noq_proto::connection::stats::PathStats>
pub trait iroh::endpoint::transports::PathSelector: core::marker::Send + core::marker::Sync + core::fmt::Debug + 'static
pub fn iroh::endpoint::transports::PathSelector::select(&self, ctx: &iroh::endpoint::transports::PathSelectionContext<'_>) -> iroh::endpoint::transports::PathSelection
pub fn iroh::endpoint::Builder::path_selector(self, selector: alloc::sync::Arc<dyn iroh::endpoint::transports::PathSelector>) -> Self

@matheus23 matheus23 merged commit 70751de into main May 22, 2026
36 of 37 checks passed
@github-project-automation github-project-automation Bot moved this from 👀 In review to ✅ Done in iroh May 22, 2026
@matheus23 matheus23 deleted the configurable-path-selection branch May 22, 2026 15:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

5 participants