test regression dnspython with python 3.5 only: (assert isinstance(ans, greendns.dns.resolver.Answer)) #267
Comments
anthraxx
changed the title
|
Hey @anthraxx, thanks for the report, different issues/tickets for different things is perfectly fine. This is a funny issue. I managed to nail it down to: 1 There's an import cycle that's wrapped by a # socket.py
greendns = None
if os.environ.get("EVENTLET_NO_GREENDNS", '').lower() != "yes":
try:
from eventlet.support import greendns
except ImportError as ex:
passIf instead of silencing the error we print a stack trace we get something like: 2 The import cycle prevents # tests/greendns_test.py
import tests
# ...
try:
# ...
import dns.resolver
# ...
from eventlet.support import greendns
# ...3 A behaviour change in Python 3.5 makes that Python < 3.5:
Python >= 3.5:
4 The point above seems to expose a bug in our patching system. If # minitest.py
def report():
print('xx id(greendns.dns.resolver.Answer): %s' % (id(greendns.dns.resolver.Answer),))
print('xx greendns.HostsAnswer is a subclass of greendns.dns.resolver.Answer: %s' % (
issubclass(greendns.HostsAnswer, greendns.dns.resolver.Answer),
))
print('xx')
import os
if os.environ.get('DNS_FIRST') == 'yes':
print('xx importing dns.resolver')
import dns.resolver
print('xx importing greendns')
from eventlet.support import greendns
report()
print('xx importing dns.resolver')
import dns.resolver
report()Python 2.7: Python 3.6 (custom build): I don't immediately understand why this happens and what needs to be done to improve it but at least we know something. PS. Another thing is, I'm tempted to remove this block from greendns = None
if os.environ.get("EVENTLET_NO_GREENDNS", '').lower() != "yes":
try:
from eventlet.support import greendns
except ImportError as ex:
pass
if greendns:
gethostbyname = greendns.gethostbyname
getaddrinfo = greendns.getaddrinfo
gethostbyname_ex = greendns.gethostbyname_ex
getnameinfo = greendns.getnameinfo
__patched__ = __patched__ + ['gethostbyname_ex', 'getnameinfo'] |
|
@temoto Any ideas regarding the above? I think we're hitting a similar issue when using httplib2 while testing on Python 3.5 |
|
I think it's easy to check if the module was already imported via |
|
Yes, by testing (above) I mean running a production code in a Python 3 environment, experimentally. I'll try to provide a minimal test case. |
|
A minimal code to reproduce (latest Eventlet and httplib2, Python 3.5.1): import httplib2
import eventlet.green.urllib
http = httplib2.Http()
http.request('http://example.org')The result: The relevant bit of code from if isinstance(info, http.client.HTTPResponse): # <- this condition fails although it should not
for key, value in info.getheaders():
key = key.lower()
prev = self.get(key)
if prev is not None:
value = ', '.join((prev, value))
self[key] = value
self.status = info.status
self['status'] = str(self.status)
self.reason = info.reason
self.version = info.version
elif isinstance(info, email.message.Message):
for key, value in list(info.items()):
self[key.lower()] = value
self.status = int(self['status'])
else:
for key, value in info.items(): # <- the first condition doesn't match so this is executed and fails
self[key.lower()] = value
self.status = int(self.get('status', self.status))I think that's because existing, already imported import http.client
print(['before green import', id(http.client)])
import eventlet.green.urllib
print(['after green import', id(http.client)])The result: |
|
@jstasiak thank you very much for minimal case. I can't right now, will look into this issue later. Maybe it's possible to fix this issue in patcher. |
|
I'll extract my test case into a separate issue as I've found another one I think. |
The green socket module seemed to have only blocking DNS resolution
methods even with dnspython installed which is inconsistent with the
documentation.
This patch has a few consequences:
* an import cycle is eliminated
* if an import cycle reappears here it'll be visible
Note: eliminating the import cycle revealed an issue related to monkey
patching and the way we perform greendns tests (the test failures were
already present on Python 3.5[1] as that version has some import cycle
handling changes). The failures look like this:
======================================================================
FAIL: test_query_ans_types (tests.greendns_test.TestHostsResolver)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/kuba/projects/eventlet/tests/greendns_test.py", line 97, in test_query_ans_types
assert isinstance(ans, greendns.dns.resolver.Answer)
AssertionError
======================================================================
FAIL: test_query_unknown_no_raise (tests.greendns_test.TestHostsResolver)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/kuba/projects/eventlet/tests/greendns_test.py", line 129, in test_query_unknown_no_raise
assert isinstance(ans, greendns.dns.resolver.Answer)
AssertionError
This issue will be addressed in a separate commit.
This patch is contributed by Smarkets Limited.
[1] #267
Hey, me again, sorry for all the noise, I hope creating new issues for different problems is in your interest? Thanks for all your fast feedback.
With an up to date dnspython (1.12.0) two greendns_test.py tests are failing (this happens only with python3.5, python2.7 is working fine):
Is this a regression in greendns_test within eventlet or may this be a dnspython issue?
The text was updated successfully, but these errors were encountered: