pubsub.listen() fails to receive new messages after 5-10 mins of channel inactivity.

[kmorris81]

Version: Redis Server = 6.0.6. | redis-py = 3.5.3
Platform: Python 3.7.0 on Windows 10 1909

Sample code:

import redis
r = redis.Redis(host="<redis_ip>", port=<port>, decode_responses=True, client_name="WinSubFail")
p = r.pubsub(ignore_subscribe_messages=True)
p.subscribe("my.test.channel")
for m in p.listen():
    print(m)

On Windows, after a period 5-10 minutes of inactivity on the "my.test.channel", new/subsequent messages will fail to be received by the listen() function.
On Linux (Ubuntu 18.04), the same code will run successfully forever. Hours/days of inactivity can go by and all new messages will be received and printed.

The Redis server still appears to know about the "WinSubFail" client connection as it exists in the output of CLIENT LIST.
I tried several variations of socket_keepalive with socket_keepalive_options values but did not seem to have any affect.

[kmorris81]

I implemented a hacky workaround by setting socket_timeout=60 on the client initialization.
Then wrapping the listen() function inside a while True + try/except

while True:
    try:
        for m in p.listen():
            print(f"{m}")
    except redis.exceptions.TimeoutError:
        pass
...

[andymccurdy]

It sounds like something is severing the TCP connection between the client and server. Have you tried the health_check_interval setting? It's designed to help in situations like this. You can read more about here (at the bottom of the section): https://github.com/andymccurdy/redis-py#connections

[kmorris81]

I have tried setting health_check_interval to various values and that didn't seem to have any affect either.
If I set health_check_interval=2, does that mean I should see additional TCP traffic from my Redis Client to my Redis Server every 2 seconds? When I run tcpdump on the redis server, I don't see any additional traffic.

Also, per the #connections doc section, I'm somewhat confused as to how I can run pubsub.check_health() or listen() any more frequently since the listen() function itself blocks.

Thanks!

[andymccurdy]

Because of listen()'s blocking nature, it doesn't play very well with the health checks. Use get_message() instead:

p.subscribe("my.test.channel")

# use a timeout to be nice to the cpu. this should not impact application performance
# this value should be lower than the `health_check_interval`
for m in p.get_message(timeout=0.1):
    if m is None:
        # the timeout elapsed and no message was delivered. try again
        continue
    # m is not null, so you have a valid message, do something with it here.

[kmorris81]

For some reason, I falsely assumed that get_message() generated network traffic, and I didn't realize it had a timeout parameter.

Just as a FYI for any future readers of this issue, your above code fails with NoneType errors on the for line, but the logic provided in the #connections doc is good:

while True:
    m = p.get_message(timeout=60)
    if m is not None:
        print(m)

I have set health_check_interval=59 and I see the traffic on tcpdump every 60s.

So I think I have an acceptable solution/workaround at this point.
Thanks again for your assistance, I greatly appreciate it!