Design solution for clients leaving the jam

[JayThomason]

We need to design a solution that removes music from the library and keeps the jam going when a user leaves the jam.

For now it is fine if we don't support migrating the jam if the master goes down, so I am making this issue to focus on how we will deal with clients leaving the jam.

[JayThomason]

The first possible solution that comes to mind is pretty simple.

The clients all periodically send a heartbeat or keep-alive message to the master after joining the jam. They could hit an endpoint like /jam/keepAlive.

For each client the master keeps track of the last time that the client made a heartbeat. The master also has a timed thread that loops every X seconds. In this thread it checks if any of the client's has not sent a message in the last X seconds, if so it assumes that they are dead and it removes them.

Removing a client entails removing all songs from the library and jam which belong to that client and then sending a message to all of the other clients telling them to do the same thing.

[hwray]

This sounds perfect. This is exactly what I was picturing but in the reverse direction (master pinging the clients). This way we could potentially avoid the need for a background thread, as we can assume the client has exited if we get an onFailed() from our AsnycHTTP request in the master (potentially after retrying once or twice).

[hwray]

Actually I suppose we would still need a background thread to handle pinging the clients, and this could potentially place more strain on the master, which is already a bottleneck. Just another approach to consider though.

[JayThomason]

For now we do still need a background thread on the master to check whether any clients have not pinged in the last X seconds.

[JayThomason]

Another issue that I thought of is this: if multiple phones have the same song, but the master only tracks the first one that it sees, then what do we do when that phone leaves the jam?

We need a way to keep track of multiples and fall back to other phones when the primary phone with a song leaves the jam.

[JayThomason]

Clients can now leave the jam and it seems to work well. We will have to do a bit more testing before release time so that we ensure clients can leave and others can join at the same time.
We will also need to keep this in mind moving forward:

if multiple phones have the same song, but the master only tracks the first one that it sees, then what do we do when that phone leaves the jam?