Response time out with client connected using unix_socket_path when using python client but not with redis-cli

[narayanacharya6]

Response time out with client connected using unix_socket_path when using python client but not with redis-cli.

I have a redis-server instance running locally accepting connections via unix socket.redisgraph.so was built by compiling the RedisGraph project. redis-server was started as:

redis-server --unixsocket /tmp/redis.sock --loadmodule ./src/redisgraph.so

I create the python client as:

conn = redis.Redis(host=redis_host, port=redis_port, decode_responses=True, unix_socket_path="/tmp/redis.sock")
graph = Graph(db_name, conn)

My queries to construct the nodes and edges of the graph work fine. But when I try to to query for something from the constructed graph, the response never returns (partial stack trace below). When I try to run the same query from redis-cli (run as redis-cli -s /tmp/redis.sock) I do get a response. Leads me to believe something in the python client is not right, maybe.

File "/Users/narayan/Documents/path/to/files/file.py", line 84, in build_frames
    result = graph.query(query["query"])
  File "/Users/narayan/anaconda3/envs/bridge/lib/python3.8/site-packages/redisgraph/graph.py", line 175, in query
    response = self.redis_con.execute_command(*command)
  File "/Users/narayan/anaconda3/envs/bridge/lib/python3.8/site-packages/redis/client.py", line 901, in execute_command
    return self.parse_response(conn, command_name, **options)
  File "/Users/narayan/anaconda3/envs/bridge/lib/python3.8/site-packages/redis/client.py", line 915, in parse_response
    response = connection.read_response()
  File "/Users/narayan/anaconda3/envs/bridge/lib/python3.8/site-packages/redis/connection.py", line 739, in read_response
    response = self._parser.read_response()
  File "/Users/narayan/anaconda3/envs/bridge/lib/python3.8/site-packages/redis/connection.py", line 363, in read_response
    response = [self.read_response() for i in xrange(length)]
  File "/Users/narayan/anaconda3/envs/bridge/lib/python3.8/site-packages/redis/connection.py", line 363, in <listcomp>
    response = [self.read_response() for i in xrange(length)]
  File "/Users/narayan/anaconda3/envs/bridge/lib/python3.8/site-packages/redis/connection.py", line 363, in read_response
    response = [self.read_response() for i in xrange(length)]
  File "/Users/narayan/anaconda3/envs/bridge/lib/python3.8/site-packages/redis/connection.py", line 363, in <listcomp>
    response = [self.read_response() for i in xrange(length)]
  File "/Users/narayan/anaconda3/envs/bridge/lib/python3.8/site-packages/redis/connection.py", line 363, in read_response
    response = [self.read_response() for i in xrange(length)]
  File "/Users/narayan/anaconda3/envs/bridge/lib/python3.8/site-packages/redis/connection.py", line 363, in <listcomp>
    response = [self.read_response() for i in xrange(length)]
  File "/Users/narayan/anaconda3/envs/bridge/lib/python3.8/site-packages/redis/connection.py", line 363, in read_response
    response = [self.read_response() for i in xrange(length)]
  File "/Users/narayan/anaconda3/envs/bridge/lib/python3.8/site-packages/redis/connection.py", line 363, in <listcomp>
    response = [self.read_response() for i in xrange(length)]
  File "/Users/narayan/anaconda3/envs/bridge/lib/python3.8/site-packages/redis/connection.py", line 324, in read_response
    raw = self._buffer.readline()
  File "/Users/narayan/anaconda3/envs/bridge/lib/python3.8/site-packages/redis/connection.py", line 256, in readline
    self._read_from_socket()
  File "/Users/narayan/anaconda3/envs/bridge/lib/python3.8/site-packages/redis/connection.py", line 198, in _read_from_socket
    data = recv(self._sock, socket_read_size)
  File "/Users/narayan/anaconda3/envs/bridge/lib/python3.8/site-packages/redis/_compat.py", line 72, in recv
    return sock.recv(*args, **kwargs)

Other Details:
OS version: macOS Monterey 12.0.1, Intel-based MBP (2019)

pip version of clients used:

$ pip list | grep redis
redis                             3.5.3
redisgraph                        2.4.0

redis-server version: Redis server v=6.2.6 sha=00000000:0 malloc=libc bits=64 build=c6f3693d1aced7d9
redisgraph.so was built from latest master branch.

[narayanacharya6]

Managed to isolate the instance where the above process is getting stuck. It blocks after it reads this from the buffer:

b'+\'South Korea s SK Innovation to file patent infringement lawsuit against LG Chem in U.S. South Korea\'"\'"\'s SK Innovation to file patent infringement lawsuit against LG Chem in U.S. \n'

This is the value printed in the data variable at line https://github.com/redis/redis-py/blob/master/redis/connection.py#L249

Removing the field from my query no longer blocks socket IO. Any ideas why this might be happening?

Also, let me know if I should rather move this issue to the redis-py project.

[swilly22]

Hi @narayanacharya6 thank you for reporting! would you mind sharing a dataset with which we can reproduce this behavior?

[narayanacharya6]

Hey @swilly22, due to data privacy reasons I am unable to share the data/query that led to this. I will try to reproduce this issue with dummy data and report back. Meanwhile, do you mind answering a couple of questions, which I anticipate to be the culprit (but have no proof of at the moment)?

1. Can having \n (like above) or \r or other special characters in my response fields lead to inconsistent behavior with reading from the connection buffer? I see that these characters are used as separators while reading fields (https://github.com/redis/redis-py/blob/master/redis/connection.py#L71) .

2. When connecting over unix sockets, are there limits to how long the requested query can be (mine is upwards of 20k string characters, UNION of many MATCH queries) or how large the response can be? If there are buffer limits and associated configs, can you point me to some documentation?

FYI, these queries work as expected when connecting via TCP.

[narayanacharya6]

Ok, I think its because of \n in the response fields.

Minimal example to reproduce inspired by the getting started example here - https://docs.redis.com/latest/modules/redisgraph/redisgraph-quickstart/ is below. The only change I've introduced in the example is added \n to the name Valentino Rossi.

1. Start your redis-server with redisgraph module and unix socket.
   redis-server --unixsocket /tmp/redis.sock --loadmodule /path/to/redisgraph.so

2. For redis-cli:
   a. Start redis-cli: redis-cli -s /tmp/redis.sock
   b. Create graph: GRAPH.QUERY MotoGP "CREATE (:Rider {name:'Valentino Rossi\n'})-[:rides]->(:Team {name:'Yamaha'}), (:Rider {name:'Dani Pedrosa'})-[:rides]->(:Team {name:'Honda'}), (:Rider {name:'Andrea Dovizioso'})-[:rides]->(:Team {name:'Ducati'})"
   c. Run query: GRAPH.QUERY MotoGP "MATCH (r:Rider)-[:rides]->(t:Team) WHERE t.name = 'Yamaha' RETURN r,t"

You will get a response:

1) 1) r
   2) t
2) 1) 1) 1) 1) "id"
            2) (integer) 0
         2) 1) "labels"
            2) 1) Rider
         3) 1) "properties"
            2) 1) 1) name
                  2) Valentino Rossi

      2) 1) 1) "id"
            2) (integer) 1
         2) 1) "labels"
            2) 1) Team
         3) 1) "properties"
            2) 1) 1) name
                  2) Yamaha
3) 1) "Cached execution: 0"
   2) "Query internal execution time: 0.526000 milliseconds"

3. For redis python client:

import redis
from redisgraph import Graph
redis_conn = redis.Redis(host="localhost", port=6379, decode_responses=True, unix_socket_path="/tmp/redis.sock")
graph = Graph("MotoGP", redis_conn)
graph.query("MATCH (r:Rider)-[:rides]->(t:Team) WHERE t.name = 'Yamaha' RETURN r,t")

The above never returns.

[swilly22]

This is what I'm getting on my dev machine:

BigSur 11.6.1
Python 3.9.5
RedisGraph master branch

pip list | grep redis
hiredis                    2.0.0
redis                      3.5.3
redisgraph                 2.4.1

Test script:

import redis
from redisgraph import Graph

db_name = "MotoGP"

def query_db(connection):
    graph = Graph(db_name, connection)

    # query graph
    q = """MATCH (r:Rider)-[:rides]->(t:Team) WHERE t.name = 'Yamaha' RETURN r,t"""
    result = graph.query(q)
    result.pretty_print()

#-------------------------------------------------------------------------------
# form connections
#-------------------------------------------------------------------------------

# connect to database using TCP
tcp_conn = redis.Redis(host='localhost', port=6379, decode_responses=True)

# connect to database using unix-socket
unix_socket_conn = redis.Redis(host='localhost', port=6379, decode_responses=True, unix_socket_path="/tmp/redis.sock")

#-------------------------------------------------------------------------------
# populate graph
#-------------------------------------------------------------------------------

tcp_conn.flushall()

q = """CREATE (:Rider {name:'Valentino Rossi\n'})-[:rides]->(:Team {name:'Yamaha'})"""

#graph = Graph(db_name, tcp_conn)
graph = Graph(db_name, unix_socket_conn)
result = graph.query(q)

#-------------------------------------------------------------------------------
# query graph
#-------------------------------------------------------------------------------

query_db(tcp_conn)
query_db(unix_socket_conn)

Results:

python3 test.py
+------------------------+-----------------+
|           r            |        t        |
+------------------------+-----------------+
| {name:"Valentino Rossi | {name:"Yamaha"} |
|           "}           |                 |
+------------------------+-----------------+

Cached execution 0.0
internal execution time 0.358
+------------------------+-----------------+
|           r            |        t        |
+------------------------+-----------------+
| {name:"Valentino Rossi | {name:"Yamaha"} |
|           "}           |                 |
+------------------------+-----------------+

Cached execution 1.0
internal execution time 0.149

Both connections produce the same result-set, I'm adding @chayim to this issue maybe he'll have an idea,
@narayanacharya6 would you mind installing hiredis when this python package is available redis-py switches to it in-order to perform redis response parsing.

[narayanacharya6]

@swilly22 Installing hiredis makes the difference. With hiredis, I do get back responses as expected.
Keeping this issue open in case we expect the same behavior without hiredis as well.

[swilly22]

Thank you @narayanacharya6 for validating this, we will look into the none hiredis case
Will you be running with hiredis ?
@chayim do you think \n messes with the python response parsing ?

[narayanacharya6]

@swilly22 For now, I can continue working with hiredis installed. I haven't done rigorous testing yet - but so far seems fine.