Support Client Sharing Local to Different Remotes

[nibanks]

Currently, we don't allow a client to share a local address & port (tuple) unless the remote tuple matches as well; i.e. the UDP 4-tuple must match for sharing to succeed. This is currently a work around for a Windows limitation in the port pool, where the same local port cannot be shared among sockets.

This is a painful limitation because this means an app that wants to create lots of connections and wants to be able to share will only need to track which tuples are already in use and can only share if there is a matching tuple.

We should experiment if Linux has the same restriction. If it does not, then we can remove this limitation at least for Linux.

We should also look into fixing this limitation on Windows.

[timprepscius]

This does not appear to work on osx.
It seems socket is sending to the one and only remote, rather than the other.

A test of something like (pseudo code) fails: (acceptor1 gets all of the connections)

			TestProduct acceptor1Product;
			auto acceptor1 = factory->newAcceptor({});
			acceptor1->open();

			auto acceptor1Delegate = newFunctionPacketDelegate(acceptor1Product);
			acceptor1->setDelegate(acceptor1Delegate);

			TestProduct acceptor2Product;
			auto acceptor2 = factory->newAcceptor({});
			acceptor2->open();

			auto acceptor2Delegate = newFunctionPacketDelegate(acceptor2Product);
			acceptor2->setDelegate(acceptor2Delegate);
			
			// delay just a bit to make sure
			// with other libraries I can flush all events
			factory->cycle();

			// all connections from a connector will originate from the same socket
			auto connector = factory->newConnector({});
			
			// make a connection to the first acceptor
			auto connection1 = connector->connectTo(acceptor1->getLocalAddress());
			factory->cycle();

			// make a connection to the second acceptor
			auto connection2 = connector->connectTo(acceptor2->getLocalAddress());
			factory->cycle();

			REQUIRE(!connection1->isClosed());
			
			REQUIRE(!connection2->isClosed());
			
			Packet p { 0, 1, 2, 3, 4, 5 };
			connection1->send(Protocol::Reliable, Packet(p));
			connection2->send(Protocol::Reliable, Packet(p));
			factory->cycle();
			
// ones of these will fail
			REQUIRE(acceptor1Product.packetsReceived.size() == 1);
			// this fails
			REQUIRE(acceptor2Product.packetsReceived.size() == 1);
			
			connection1 = nullptr;
			connection2 = nullptr;
			connector = nullptr;
			acceptor1 = nullptr;
			acceptor2 = nullptr;

[timprepscius]

Also, in a different scenario (osx):

portA connects to portB
close portA
instantiate new-portA using same portA
connect new-portA to portC

this fails, in messages aren't received on portC, and as well, there becomes a protocol violation which causes a close initiation.

[nibanks]

I did not enable it on OSX, because I didn't have a local system to try it on. It might work there if you enable it.

[timprepscius]

Hmmm, you can run osx in a virtual machine these days I think

[timprepscius]

How do you enable it?

[ThadHouse]

You can remove https://github.com/microsoft/msquic/blob/main/CMakeLists.txt#L340 lines 340 and 341. You'll also have to add

Datapath->Features = CXPLAT_DATAPATH_FEATURE_LOCAL_PORT_SHARING;

to CxPLatDataPathInitialize in datapath_kqueue.c

Note I don't actually think it will work, as it requires sockets to be able to be sharable, which iirc macOS doesn't support. But its something you'd have to try.

[timprepscius]

Ok, will try later today.