Weird redis queue non-emptive behavior in game_waiting_room with my local dev env

[Xiaohong-Deng]

Well, for a long time on my local dev env it is when 3 players in the room then it shows 3 on the page. Until today I spotted that my redis queue didn't have a fresh start with size 0 when I ran rails s.

Today when I started a game as the first user, it showed 2 players in the room. I manually emptied it. then I added 3 players. something that never happened happened. one player jumped to another page. the other 2 stayed in waiting room, showing 0 players in the room.

strange thing is before today, each time I enter the room as a player the number shown was always right, just it didn't jump when the room was full.

[Xiaohong-Deng]

ok. now three players all jump to another page. weird. this implementation might not be robust. but all we can do is test it by hand. not thorough. not much can do at the moment.

[Xiaohong-Deng]

I realized that the redis queue is actually persistent. if waiting room is not ended normally like you interrupt a cucumber test, then next time you use waiting room it is not empty. how to ensure the emptiness?

[Xiaohong-Deng]

actioncable also suffers from this weird behavior. need to seriously consider a solution to database_clean redis queue or whatever.

[Xiaohong-Deng]

I guess redis is a database just like postgresql.

if it is loaded, it loads everything relevant say to your application to memory and operates at the memory level. once the application is terminated it stores everything to disk.

local case: if you kill the server by ctrl-C while you're still in the waiting room, then you are stuck in the waiting room.

remote case: I suspect Heroku kill your app after some idle time. if you're still in the room by the time it is killed, you are stuck.

[Xiaohong-Deng]

also currently waiting room queue uses a gem called redis-unique-queue which operates on the gem redis which provides an API to redis.

so waiting queue is actually persistent. no obvious way to empty the queue if the app is interrupted abnormally.

[Xiaohong-Deng]

I did more digging. in Jacob's implementation, every time the GameWaitingRoom is initialized Redis::Unique::Queue.new("game_waiting_room", Redis.new(url: actioncable_url)) is called. the first argument string is the key, the second Redis.new is a connection to Redis which is running on the given url. Redis is persistent. So basically you get a portal to Redis, not a new queue. the connection is new but Redis is not, unless you change the url every time I guess. if you ask for the object stored as the value to the key, Redis will tell you based on the info stored on disk. in our case, Redis.get("game_waiting_room") returns a persistent object if there is one.

Maybe there is a switch to turn off the persistence property, but I haven't found one. maybe I could put a callback somewhere in the app to clear the queue right before the app is turned off on the server, but I don't know how for the moment.

for now I decide to use a ruby set to replace the Redis queue.

[Xiaohong-Deng]

I reopened this ticket because I suspect there could be some concurrency problem. If ten users click New Game at the same time, will they be ejected to the same game altogether?

at the moment I don't know how to do concurrency testing to reproduce the scenario. the first step should be writing a test to test it, see what happens.

If possible it's better to do TDD model testing in RSpec. it could be accessing in memory object at the same time from multiple threads. what's important is adding things to waiting room at the same time

this particular in memory object we are accessing is a class instance variable, which is often concerned in thread-safe discussions.