Server misreporting gameInfo to clients

[HardlySoftly]

Server is reporting 100s of closed state games with 0 players to the FAF client on startup, causing client side performance issues.

See FAForever/client#348

[HardlySoftly]

The number of bad gameInfo items being sent has grown since the report yesterday (seems like a bad trend)

[Crotalus]

An uneducated guess would be that games gets state ENDED but never marked dirty so they doesn't get cleaned up in report_dirties() function.

Could timeout-games be the culprit?
https://github.com/FAForever/server/blob/develop/server/games/game.py#L142

No dirty marking there

[HardlySoftly]

Can confirm some seriously old games are getting reported to the client (like 3+ weeks old), and its the old ones with an open state that are causing issues.

[aeoncleanse]

Specifically, it was that 97 out of 205 registered games were older than 24 hours.

[Crotalus]

old ones with an open state that are causing issues.

Aren't they all closed, or?

[aeoncleanse]

https://github.com/FAForever/client/blob/develop/src/replays/_replayswidget.py#L353

At fault is that function. Specifically, lines 440 and 441, cutting those dramatically cuts execution time. State has to be 'Playing' to get there.

[HardlySoftly]

Here's a pastebin of the games I got sent by the server: http://pastebin.com/0ETTYnRT

[Crotalus]

Hmm, seems like all games are closed, visibility is None which is a bit weird, AFAIK it should be "public" as default.

Only place I can see this being possible would be here:
https://github.com/FAForever/server/blob/develop/server/lobbyconnection.py#L932

i.e. someone sent some unknown visibility argument

but on the other hand that visibility should never be set on the game object then?

[HardlySoftly]

Old games building up again and causing issues, if we don't have a fix yet it may be time for the classic server restart before people stop being able to log in.

[Crotalus]

Client was monkey patched but server bugfix has been lingering since ~1 month AFAIK

[HardlySoftly]

Client hasn't been patched in ages, and right now I'm counting 111 games from the server that didn't start in the last hour, and 90 which didn't start in the last 24 hours.

The last I heard we knew what the problem was, is there a hold up with deploying a fix?

[HardlySoftly]

People have come to me saying they're getting the Outbound Traffic Blocked errors.

I have told them its a known issue, and I'm sure it will be resolved soon.

[Sheeo]

Yes I'll reboot the server with Crotalus' patch tonight when we've got a lower user count.

[HardlySoftly]

The server is still sending dead games to my client

[Crotalus]

Hmm if fix is applied that means more game state debugging I guess

[HardlySoftly]

Still have this problem after server/client update, but at least now the client won't completely barf on it