Feature: play easier together with friends from behind home routers #9017
!!NOTE!! Currently the Game Coordinator is not working with this PR, while I rebuild this PR from the ground up. See https://gist.github.com/TrueBrain/75c016cdb90efa18cb79f3f6bae9af25 for details.
Motivation / Problem
Currently, to host a network server you really need to know a few things about routers and networking. This is way to advanced for the average Joe, and this becomes very clear when looking at Steam Discussions.
So, can we do this? I wrote a gist to explain the process of getting to this PR: https://gist.github.com/TrueBrain/f5685dc0fdbd012aac48ea2a36158198
But in case you are not into reading, let me summarize:
There are now three kind of servers:
See the individual commits and the gist above. Not much more to add.
If you want to help out testing, great!
Notice how you didn't have to setup firewalls, port-forwarding, etc?
Let me know if you find any problems, etc.
Checklist for review
Some things are not automated, and forgotten often. This list is a reminder for the reviewers.
The text was updated successfully, but these errors were encountered:
I was sure I wrote that down somewhere .. guess I did not :D
It is a worst-case estimate: on peak on Steam we had 3.2k people online. Say, with a good multiplayer, they all want to play with their friends, round it up, gives an upper value of ~5k.
Infrastructure wise, being able to handle that, means I can sleep peaceful at night.
You think I am understating? Let me know :)
This is pretty much done and ready for review. I am 100% sure there are bugs, so lets find them and fix them :)
The Game Coordinator is not yet running on AWS, and is not scalable yet. Otherwise it should do its job. So any client-facing bug you can find, please report that.
My suggestion is to get this PR code-reviewed and merged. After that I will work on the Game Coordinator, to make it production-ready. But that also gives time to test the client itself.
There is plenty of future-work to be done, but this can be found back here:
Did some testing with a dedicated server and a client.
I had an issue with a ":3979" server, but I can't reproduce it :(
Edit: I had the IPv6 firewall activated on the router, and it has only 2 modes on or off, no fine tuning. Disabled it to test and connection type became "Public". Build from master was advertising only the IPv4, as IPv6 was blocked by this "better than nothing" firewall.
@TrueBrain You mentioned that you load tested the Game Controller and showed that it can handle at least 10k clients. Do you also plan on load testing the TURN server as well, in order to get a good feel for how many active clients and servers it can support so that the network can scale accordingly?
Yeah, tested that in one go too! Also the STUN server btw. Seems to all work and seems rather stable. Python and asyncio have a very small overhead per connected client (few KB of RAM, that is it). And CPU is the one thing we aren't short in, so not a real concern. So that all seems to work fine during testing. Of course .. real world can be a different story (stalling connections, very slow ones, etc) .. but I think we fixed most of those issues already with the current master-server (the framework used is identical between the two).
The reason for the 10k btw is that my computer couldn't spawn more clients in a useful way :P
…dress This allows future extensions to have different ways of referencing a server, instead of forcing to use IP:port.
In this mode you do register to the Game Coordinator, but your server will not show up in the public server listing. You can give your friends the invite code of the server with which they can join.
NewGRFs cannot be changed in multiplayer games, and consumes a lot of bytes. So only send it once to the Game Coordiantor on registration, and skip the field after that for any update. A new game will restart the registration, so any NewGRF change between games is picked up via the new registration. Clients that look at the NewGRFs a server has, want to know the names of the NewGRFs more than the IDs. So in addition to the ID and MD5, send the full name to the clients too. This solution is picked so the Game Coordinator isn't sending the names of all the NewGRFs to the clients constantly (which would make the server-listing very large), but still allow clinets to see the name of a specific server if he clicks the button.
Slowly we have been merging parts of this PR into master. #9447 is the last piece. Once that has been merged, everything in this PR (and yes, I checked) has been merged into master too.
By splitting it up, there was a huge jump in quality of what has been delivered. It really was the right call to make everything smaller, although it took significantly more work to do so. Either way, the result is that the backend is running on AWS like the rest of our infra, and that the client/server are doing what they should be .. at least for as far as I could reasonably test.
After #9447 is merged, the testing is up to the public .. and I am sure they will find plenty of edge-cases for us to fix :D But we will tackle them if/when they come in.
For now, I am going to close this PR. Tnx all who contributed, and feel free to test the latest nightly for STUN .. and hopefully soon TURN will hit the nightly too :D