Reduce allocations by using netip.AddrPort - #943
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #943 +/- ##
==========================================
+ Coverage 88.26% 88.35% +0.08%
==========================================
Files 46 46
Lines 6137 6277 +140
==========================================
+ Hits 5417 5546 +129
- Misses 494 498 +4
- Partials 226 233 +7
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
@kcaffrey Thank you so much, I reviewed this over the previous week and i don't see any issues. but I would request to break this into 4 PRs or at least 4 commits in this PR so they are easier to check the blame and for a cleaner history. |
Remote candidates were matched to inbound traffic by comparing the candidate's SDP address string against the stringified source IP of the packet. For an mDNS candidate the SDP address remains the unresolved ".local" hostname, so a resolved mDNS candidate could never match and inbound traffic instead fell back to creating a duplicate peer-reflexive candidate for the same remote. Match on the candidate's resolved transport address instead, which is more in line with RFC 8445 (section 7.3.1.3, with the definition of transport address in section 4 being "address and port"). Part 1/4 of pion#943
UniversalUDPMuxDefault indexed cached XOR-mapped addresses by the string form of the STUN server's net.Addr. Key the map by a canonicalized netip.AddrPort instead, so the same transport address produces one cache entry regardless of the IPv4/IPv4-in-IPv6 form or custom net.Addr implementation it arrives in. One behavior change: callers of GetXORMappedAddr with a custom net.Addr whose string representation is not "ip:port" now get an immediate error instead of sending a request that can never match and timing out. Part 2/4 of pion#943
Refactor the agent, candidates, selectors, and UDP mux internals to represent transport addresses as netip.AddrPort instead of net.Addr where possible, in preparation for allocation-free reads and writes. - bindingRequest tracks its destination as netip.AddrPort plus the NetworkType of the transport the request was sent over; the NAT response symmetry check now compares network type explicitly since netip.AddrPort does not carry one. - Peer reflexive candidates determine network type from the local candidate, which by definition shares the remote's transport. - Candidates keep parallel resolved net.Addr and netip.AddrPort forms so write calls can reuse the cached net.Addr without allocating. - The UDPMux connection maps drop the custom ipPort type and key by a canonicalized netip.AddrPort, preserving zones only for link-local IPv6 and unmapping IPv4-in-IPv6 to IPv4. Part 3/4 of pion#943
Reading from a net.PacketConn allocates a net.Addr per packet. Go 1.18 added ReadFromUDPAddrPort and WriteToUDPAddrPort to *net.UDPConn for allocation-free reads and writes using netip.AddrPort. Add an exported AddrPortReaderWriter interface that lets PacketConn implementations opt in to allocation-free reads and writes, and plumb it through the candidate receive/write loops, UDPMuxDefault, and UniversalUDPMuxDefault. *net.UDPConn is adapted automatically, but only as the exact concrete type so wrappers embedding *net.UDPConn that override ReadFrom/WriteTo are not bypassed. Connections that do not implement the interface keep the prior behavior. This takes a UDP read/write round trip from 2 allocs/op to 0 for connections supporting AddrPort I/O; a test pins it at zero. Part 4/4 of pion#943
|
@JoTurk thanks for the review, and for the good suggestion on how to split the commits. I have force pushed with this PR split into four commits roughly matching your suggestion, and agree it's better this way. The diff from my previous single commit to the sum of these four commits is empty, and each commit passes the tests and lints. This seems easier to merge as a single PR (with rebase instead of squash) but if you want me to open this as four separate PRs I can do so. |
Remote candidates were matched to inbound traffic by comparing the candidate's SDP address string against the stringified source IP of the packet. For an mDNS candidate the SDP address remains the unresolved ".local" hostname, so a resolved mDNS candidate could never match and inbound traffic instead fell back to creating a duplicate peer-reflexive candidate for the same remote. Match on the candidate's resolved transport address instead, which is more in line with RFC 8445 (section 7.3.1.3, with the definition of transport address in section 4 being "address and port"). Part 1/4 of pion#943
UniversalUDPMuxDefault indexed cached XOR-mapped addresses by the string form of the STUN server's net.Addr. Key the map by a canonicalized netip.AddrPort instead, so the same transport address produces one cache entry regardless of the IPv4/IPv4-in-IPv6 form or custom net.Addr implementation it arrives in. One behavior change: callers of GetXORMappedAddr with a custom net.Addr whose string representation is not "ip:port" now get an immediate error instead of sending a request that can never match and timing out. Part 2/4 of pion#943
Refactor the agent, candidates, selectors, and UDP mux internals to represent transport addresses as netip.AddrPort instead of net.Addr where possible, in preparation for allocation-free reads and writes. - bindingRequest tracks its destination as netip.AddrPort plus the NetworkType of the transport the request was sent over; the NAT response symmetry check now compares network type explicitly since netip.AddrPort does not carry one. - Peer reflexive candidates determine network type from the local candidate, which by definition shares the remote's transport. - Candidates keep parallel resolved net.Addr and netip.AddrPort forms so write calls can reuse the cached net.Addr without allocating. - The UDPMux connection maps drop the custom ipPort type and key by a canonicalized netip.AddrPort, preserving zones only for link-local IPv6 and unmapping IPv4-in-IPv6 to IPv4. Part 3/4 of pion#943
Reading from a net.PacketConn allocates a net.Addr per packet. Go 1.18 added ReadFromUDPAddrPort and WriteToUDPAddrPort to *net.UDPConn for allocation-free reads and writes using netip.AddrPort. Add an exported AddrPortReaderWriter interface that lets PacketConn implementations opt in to allocation-free reads and writes, and plumb it through the candidate receive/write loops, UDPMuxDefault, and UniversalUDPMuxDefault. *net.UDPConn is adapted automatically, but only as the exact concrete type so wrappers embedding *net.UDPConn that override ReadFrom/WriteTo are not bypassed. Connections that do not implement the interface keep the prior behavior. This takes a UDP read/write round trip from 2 allocs/op to 0 for connections supporting AddrPort I/O; a test pins it at zero. Part 4/4 of pion#943
Remote candidates were matched to inbound traffic by comparing the candidate's SDP address string against the stringified source IP of the packet. For an mDNS candidate the SDP address remains the unresolved ".local" hostname, so a resolved mDNS candidate could never match and inbound traffic instead fell back to creating a duplicate peer-reflexive candidate for the same remote. Match on the candidate's resolved transport address instead, which is more in line with RFC 8445 (section 7.3.1.3, with the definition of transport address in section 4 being "address and port"). Part 1/4 of #943
UniversalUDPMuxDefault indexed cached XOR-mapped addresses by the string form of the STUN server's net.Addr. Key the map by a canonicalized netip.AddrPort instead, so the same transport address produces one cache entry regardless of the IPv4/IPv4-in-IPv6 form or custom net.Addr implementation it arrives in. One behavior change: callers of GetXORMappedAddr with a custom net.Addr whose string representation is not "ip:port" now get an immediate error instead of sending a request that can never match and timing out. Part 2/4 of #943
Refactor the agent, candidates, selectors, and UDP mux internals to represent transport addresses as netip.AddrPort instead of net.Addr where possible, in preparation for allocation-free reads and writes. - bindingRequest tracks its destination as netip.AddrPort plus the NetworkType of the transport the request was sent over; the NAT response symmetry check now compares network type explicitly since netip.AddrPort does not carry one. - Peer reflexive candidates determine network type from the local candidate, which by definition shares the remote's transport. - Candidates keep parallel resolved net.Addr and netip.AddrPort forms so write calls can reuse the cached net.Addr without allocating. - The UDPMux connection maps drop the custom ipPort type and key by a canonicalized netip.AddrPort, preserving zones only for link-local IPv6 and unmapping IPv4-in-IPv6 to IPv4. Part 3/4 of #943
The normal methods to read and write from
net.PacketConnreturn and accept anet.Addr, respectively. As an interface, this requires at least one allocation for each read. On the write path, this package already makes an effort to cache the addresses returned from reads to reuse, meaning the current usage ofWriteTo()does not always allocate. The benchmark added in #941 shows 2 allocs/op on a read/write round trip. Profiling indicates both allocations are on the read path, with one allocation for the*net.UDPAddrstruct and a second allocation for the backing IPv4 byte slice.In Go 1.18 a pair of methods were added to
*net.UDPConnto perform allocation-free reads and writes by using thenetip.AddrPortstruct instead of an interface:ReadFromUDPAddrPortandWriteToUDPAddrPort.This change reduces the read/write roundtrip to 0 allocs/op for UDP connections (at least when using the built in UDP mux and a connection that supports allocation-free read/writes). A test is added to pin the allocations to zero.
Other than the two aforementioned allocations for every read, there were additional allocations related to
net.Addrperformed for periodic STUN binding requests (which in many cases happen every several seconds for an established connection). These allocations are reduced a fair amount but not entirely to zero.This change refactors the internals of the agent to use
netip.AddrPortinstead ofnet.Addrwhere possible. A new exported interface,AddrPortReaderWriter, is added that lets users of the library opt into allocation-free read/writes by implementing the interface forPacketConns. The interface is automatically implemented for*net.UDPConnwith an internal adapter. Custom packet connections that don't implement the new interface fall back to the prior behavior.A few implementation notes:
ReadFromUDPAddrPortandWriteToUDPAddrPortdirectly with an unexported interface, because this could be a back-compat breakage for users who wrapped a UDP connection and only overrodeReadFromandWriteTo.net.Addrand newnetip.AddrPorton candidates so that we can reuse thenet.Addrin write calls to avoid adding an allocation.netip.AddrPortdoesn't carry a network type, but I believe this is behavior-preserving and matches the spec.There are two known behavior changes:
The first is that previously mDNS candidates would fail to be matched and thus fall back to peer reflexive. This is because matching compared the candidate's SDP address string (Candidate.Address()), which for an mDNS candidate remains the unresolved somehostname.local name, against the stringified source IP of inbound traffic, so a resolved mDNS candidate could never match and traffic fell back to peer reflexive. This change incidentally changed the matching code to compare canonical IP/port to canonical IP/port, and the code will have already set the IP/port for mDNS candidates by the time it tries to match to a remote candidate. This change seems more in line with RFC 8445 (section 7.3.1.3 in particular, with the definition of transport address in section 4 being "address and port") so I kept the change and added a test.
The second is that consumers who call
GetXORMappedAddr()orGetXORMappedAddrContext()directly onUniversalUDPMuxDefaultwith a customnet.Addrwhose string representation isn't "ip:port" will now immediately get an error returned rather than the previous behavior of sending the request and getting a timeout error. Based on scanning open source projects that depend on pion/ice, I don't think this will come up in actual usage.