WIP Tunnel Game Adhoc Traffic through single port #7268

[adenovan]

Testing compile on other platform , the chat commit help showing people in the room later will be rebased

current status

-Direct UDP Tunnel to tunnel PDP trafiic is working and later we can relay it through public server
-Log spammed on pause. how to stop the tunnel thread on pause? , it seem some while code and recvfrom blocking the thread, need code review.

[hrydgard]

Pretty cool stuff. I noticed that this pull request also contains the chat code you submitted in #9090 - I'd like the two to be merged separately, so if you think the chat code is ready, I'll have another look at that and we'll merge that first, then rebase this on top of master and I'll review this for merge. Or does this have a newer version of the chat code?

[adenovan]

@hrydgard When im starting working on this feature, i accidentally branch it from the chat branch thingking im still on the master so this is happend and i just realized when do the pull request. the chat branch has newer code which im not push into the branch yet.

will do the rebase soon on this feature so the other dev can give a review or suggest me some cool way to tackle the problem, this feature is not ready yet i need to figure out how to tunnel the PTP socket. look like it more complicated than the PDP one. the schema is available on #7268 on @anr2me post. its work great on UDP but i think it cause a lot of overhead in TCP.

here is the problem, lets name the socket to illustrate the problem

Direct Tunneling
( A ) Game port <-> ( B ) TCP Tunneler <-> ( C ) Tcp Tunneler <-> ( D )Game Port

when socket A call ptp connect it require a pool to other end or connect to listening socket which is D we can pool and connect and get the error code direclty when it not relayed.

when the tcp socket relayed A will pool or connect to B earlier than expected from D. because A might think it already connected to D. but it actually connect to B. that's mean we must inform A socket the status of connection between B to C and C connection to D and might also their error info.

when a already conected to B , B socket must tell C socket to create a connection to D ( here is the loopback or local address used). D is listening and waiting A socket to connect on some game port.

this i already give a lot of overhead as A need to know B connection status to C, C connection status to D . before it send the data. on the blocking behavior its event worst. socket A must know all of relay socket pool status. before we can mark it as established and sync the matching context.

@Hydroque make a good argument when he tell the program can actually open any socket and we should not open socket asked by the game. i assume this one partially can be implemented or not working at all. because some game might need the socket FD or do what they want like pooling the socket etc. but it just my assumption i can make another experiment to just only open 3 socket and try sending data between peer with that. 1 matchmaking socket, 1 UDP socket as PDP socket's , and 1 TCP as PTP socket's and mess with PTP and PDP internal object direclty without opening port. (used already established socket or public server if relayed)

the good way is like @anr2me schema but i cant figure out how maintain all of TCP relay state and connection status, and another problem is pooling. the tunnel need to created on runtime. how much tunnel is needed it actually determined on runtime by the game. when it relayed through public server its also need B and C tunnel to connect to relay server first which will make a more lot of overhead.

so basicly if we use the schema on tcp this is happened

A socket, without connect to B tell B hey B connect to C , B tell C hey C can you create me local connection to D , and D im listening your connection from C socket A socket . A tell B might need to be earlier to establish and we need to make sure its established until D before we connect A to B . because when A connected to B its already sending the game data. is this type of relay is possible on tcp? need more expertise on tunneling TCP over TCP or some way to let A socket know the connection status of all its tunneler or relay.

[anr2me]

Yes, managing the socket state can be complex and may need to simulates the whole socket functionality.

An easier way (for PPSSPP atleast) probably you only need to simulate TimedOut Error using ConnectionReset error (ie. by dropping the connection) and pass the error code info using the tunneler wrapper header. And you'll need to treat ConnectionReset the same way with TimedOut (ie. on proAdhoc.h: inline bool isTimedOut(int errcode){ return (errcode == ETIMEDOUT || errcode == ECONNRESET); }
The rest or the socket error that occurred on PPSSPP (ie. EALREADY, EINPROGRESS, ENOTCONN) may not have issue with tunneling.

Here is an example:
1). ( B ) TCP Tunneler and ( C ) TCP Tunneler will need to be connected when both player joined AdhocServer (or when joined to the same group might be better? but i'm worried about connection issue when all players are ready and the mission is started because some games seems to switch group when starting the mission), and connection may need to be maintained (will try to reconnect the tunnels when a player is using a bad internet connection and disconnect often to prevent the game from disconnecting).

2). When a game bind & listen to a port it will be changed to port 0 instead of using the port chosen by the game, resulting binding to any available port, and the resulting port need to be mapped to the port number expected by the game for the game to use (so the game will still think the port they asked is available and doesn't behave strangely, while the actual port which is known by the tunneler is a random port).

3). When ( A ) Game port attempt to connect to ( D )Game Port it will connect to ( B ) TCP Tunneler.

( B ) TCP Tunneler will recognize it as an attempt to "connect" and set an ID number (which is part of the wrapper's header) to identify it as "connect" to tell ( C ) TCP Tunneler that an "connect" attempt is occurred.

( C ) TCP Tunneler will receive the wrapped packet which include the attempt ID (ie. "connect","disconnect",etc), and create a new connection to ( D )Game Port

4a). In a case where ( D )Game Port is not binded/ready yet ( C ) TCP Tunneler will get a TimedOut Error, it will then send a reply of empty packet with an ID number to identify it as "timedout" in the wrapper header.
( B ) TCP Tunneler will recognize it as "timedout" and drop the connection that was opened for ( A ) Game port and then ( A ) Game port will receive ConnectionReset error which should be treated the same as TimedOut inside HLE functions for this to work.

4b). In a case where ( D )Game Port is ready & listening, ( C ) TCP Tunneler will not receive any error and will pass any packet sent/received from/to game port to/from the tunneler (the attempt ID can be set to a default number to identify it as "send"ing normal packet)

5). When ( A ) Game port or ( D )Game Port is disconnecting the tunneler will set the attempt ID number to be identified as "disconnect" so the other side can drop the connection.

[adenovan]

thanks for the detailed information @anr2me , l am forgotting there is a friendfinder thread we can hook to establish the tunneler.

1). after reading it im thingking this way. we just create a new outgoing tunneler socket ( B ) with random port when friendfinder says new friend coming on the OPCODE_CONNECT , store that outgoing tunneler socket with the peer mac address and connect it later to ( C ) on PTPCONNECT, actually this one is im trying to figuring out to simulate PTPOPEN on the tunneler side ( B ). with this schema we make the game to maintain the tunnneler connection by itself in case the game did not establish any connection the tunneler also just opening socket and not doing any connection is not waste of connection. its doesnt matter if the game change group because the game will create a new adhocctl handler when the group switch change happened and the proadhoc server will just tell us to create that group info and give use the peer information to create outgoing tunneler socket connection ( B ) to ( C ). when its come to maintain the tunneler connection we let the game decide to connect the tunneler or not based on the HLECALL. we can connect or disconnect the created outgoing ( B ) socket to ( C ) on PTP Connect & PTP Disconnect. after this connection connected we send packet from HLECALL with ( B ) socket to ( C ) and tell ( C ) to establish the local connection and we wait the connected response aftert that we connect ( A ) to ( B ) and mark it as established PTPCONNECTION. ( is it possible? im worried we must make many of thread like friendfinder to parse the outgoing (B) recv from (C) tunneler. or we can just block the ( B ) recv from ( C ) on PTPCONNECT may behave strangley on nonblocking connect asked by the game i guess).

2). actually i will make this experiment if number 1 scheme failed and just open the 1 TCP tunneler socket to listen and many outgoing TCP socket's asked by the game to connect into the listened tunneler without let the game bind or listen to any other port. its just like port offset things i think but the difference we did not bind or listen or even create the gamesocket. can we simulate it just using struct and fool the game we actually create the socket? in case the game want to transfer the data we just wrap the information on the tunneler like UDPTUNNEL created.

3). we can just make an custom packet with attemp, connect , send opcode right? it actually will be used again if we implement the relay scheme and make reverse tcp tunnel like https://github.com/aploium/shootback scheme to relay the tcp connection.

4). and 5.) this is the most complicated think is there any program to look at and implement this kind relay? send and recv the socket state on other end. will need a lot of experiment.

[adenovan]

@hrydgard what is your suggestion when i want to create a new SceNetAdhoc.cpp module from scracth to impelement the tunnel and relay infrastructure ( number 2 ) on this issue #9865 .

its is about the file structure on HLE folder , should i create 3 different file for each mode and wrap it from hle wrapper or keep it single file already wrapped by hle and use if else on every function for different mode. every mode there will take different approach to emulate the entire SceNetAdhoc function.

if i combine all the code in one current single file the code will looks like a mess and have many indentation. the current implementation is also depend alot on pro adhoc file and need a pro adhoc client on psp side. its already need clean up also.

[anr2me]

Was it not enough to override/wrap sockets API functions (send, recv, etc) which being used by network HLE and handle tunneling (or passthrough to the original socket API) from those functions?

[hrydgard]

@adenovan Feel free to split it as you feel necessary, that's fine by me.

[tilkinsc]

I have made no progress on #9334 ,, still need a PSP and a way to sniff the communication.

[adenovan]

@anr2me actually it can just to use that function. But im thingking on another approach to minimize socket creation and port binding ( hope with this we did not bind the port the game want) and just using one port to handle all the game transport. That need to simulate the whole scenetadhoc function and simulate a fake socket. If it cause a problem and need a binding ,i will back to use our last discussed plan.

@hrydgard ok thanks for the reply

[adenovan]

i already refactor proadhoc server to be a ctl server to be able run multiple instance adhoc server instance (online gaming need a lot more ctl server like ps3 adhoc do ) , interested to merge with ppsspp ? this project can be used as official ppsspp server if you want @hrydgard to host it on the public and be used as a relay server later when this pr is done. or should i host ppsspp server on amultios.net ? i will run a custom build on most of the ctl server because i need to remove some ppsspp feature that hurt the online experience (description on the website)

project description
https://amultios.net (under development)

server code
https://github.com/adenovan/amultios

the proadhoc server is refactored on ctlServer.cpp cause its only do a computational tree logic on psp adhoc. the chat server run separately from all other ctl server and do a virtual grouping to make a chat group room. it can global chat also :D

this is the running server screenshot in multi CTL server instance.

it can be used for many later usefull feature too like compability list, i will make one and mark the game compability by user feedback on the facebook group

[tilkinsc]

Do these builds affect master yet? Do they affect #9334 thereof

[LunaMoo]

^ This isn't a build, but a pull request, PR in short. It was opened and nothing else, so no, it doesn't affect anything.

And assuming I read it correctly - in it's current form this isn't even acceptable for merging since tunneling as well as all the added code complexity should be optional and not replacing existing implementation. Real PSP support matters and at the same time LAN does not need tunneling and related code complexity at all.

Oh and - please don't ask the exact same thing in multiply places, it's pointless and just spams people with repeated message.;]

[adenovan]

its a wip and still a pull request @Hydroque so there is no affect at all, this is actualy just a quick experiment if tunneling is possible and not good enough to merge. if you read back in the future work i ended up implement separate adhoc module just for tunneling to separate it with aemu adhoc. but i need to finish the server first as the relay i will implement will require single external public ip address. ah i forgot to mention like lunamoo said it will be optional and considered additional feature to PPSSPP if later will be merged. the complexity may break some games or possibily add some hack to make the game working well in tunneling mode.

[tilkinsc]

I was just making sure nothing happened, and @LunaMoo it was proper to ask that in multiple places because I need to ensure I can remember when to fork it before all this tunneling stuff.

[adenovan]

after much of experiment on this feature i find more suitable protocol to use rather than building it from scratch using udp and tcp , managing many open socket to relay with tcp or udp is complex and broke so many of games if want to see the result use amultios OBT1.5.4 build its relayed in top of many opened tcp socket to server but games like tekken is too demanding for low latency.

the next experiment is using sub pub protocol for IOT mqtt . looks like we can simulate all socket easily in top of that protocol and bring more benefit than using relay building from scratch with multiple socket opened.

currently i try to add new submodule using mosquitto in PPSSPP project and failed to compile the submodule because of many environment change to use to develop PPSSPP 1.8 (VS 2017). i will report the result here