gevent seems to block redis forever with socket. getaddrinfo when used in multiprocessing

[lxyu]

I came into a problem recently, after tracing problem down, I got this code

# -*- coding: utf-8 -*-

import gevent.monkey
gevent.monkey.patch_all()

import socket
import redis
import multiprocessing


def block():
    print 'x'
    r = redis.Redis()
    r.get(1)
    print 'o'


socket.getaddrinfo('localhost', 8001)

p = multiprocessing.Process(target=block)
p.start()
p.join()

This code will be blocked forever in r.get(1).

If I comment out socket.getaddrinfo('localhost', 8001), it'll not be blocked.

So any idea on how to solve this?

---

Not clear of whether this is gevent problem or redis-py problem, so I also post this issue to redis:
redis/redis-py#310

[lxyu]

btw, I'm using

gevent==1.0rc2
redis==2.7.2

[lxyu]

ok, after some further search, add GEVENT_RESOLVER=block solve this problem.

[awestendorf]

Confirmed that I'm having this problem too and it's not anything to do with redis specifically, but any socket connection, including localhost. Not a problem on 1.0b1 which we're using, but is on 1.0rc2. Any place to start looking to help fix this bug?

[Anorov]

Is this a problem only when multiprocessing is in use?

[lxyu]

Don't know if other situations will cause it.

In this case, it is.

[awestendorf]

I had been using native threads, not multiprocessing, though the lib was loaded in order to get the cpu count. The code in question is single threaded now and I'll give this a shot again. The place where it blocked was in the main thread on process init.

[jgehrcke]

Naive forking via multiprocessing within a gevent-driven application is known to raise problems. With gipc this code works fine:

import gevent.monkey
gevent.monkey.patch_all()
import gipc
import socket
import redis

def block():
    print 'x'
    r = redis.Redis()
    r.get(1)
    print 'o'

s = socket.getaddrinfo("www.python.org", 80, 0, 0, socket.SOL_TCP)

p = gipc.start_process(target=block)
p.join()

[rayalan]

This problem also occurs when running gevent with uwsgi and the --master flag.

[evgenity]

The similar problem occurs for me when using grequest (gevent+request) and redis together. Any ideas about how to fix it?

[alternativshik]

Same problem, but with httplib2. socket.getaddrinfo() in httplib2 block forever

import os
os.environ['GEVENT_RESOLVER'] = 'ares'
resolve problem

[Anorov]

Perhaps a dumb question, but shouldn't c-ares be gevent's default resolver in the latest versions?