Fix client failing to detect multiple client disconnections

After detecting a client re-connect we didn't queue the listener again which meant the listener would stop working.

Thanks to @zhaoyeming for finding this bug!