New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test_urllib2 fails - urlopen error file not on local host #49875
Comments
[NOTE: applies to 2.x urllib2 and similar code in merged 3.x urllib] test_urllib2 can fail because urllib2.FileHandler assumes incorrectly Both the urllib module and test_urllib2 use Python 2.6.1 (r261:67515, Dec 17 2008, 23:27:50)
[GCC 4.0.1 (Apple Inc. build 5490)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> socket.gethostbyname(socket.gethostname())
'10.52.12.105'
>>> socket.gethostbyname(socket.gethostname())
'10.52.12.105'
>>> socket.gethostbyname(socket.gethostname())
'10.52.12.205'
>>> This leads to predictable test failures when the calls in test_urllib2 test_urllib2
test test_urllib2 failed -- Traceback (most recent call last):
File
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/test/te
st_urllib2.py", line 621, in test_file
r = h.file_open(Request(url))
File
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2
.py", line 1229, in file_open
return self.open_local_file(req)
File
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2
.py", line 1266, in open_local_file
raise URLError('file not on local host')
URLError: <urlopen error file not on local host> The simplest way to avoid the test failure is to modify
urllib2.FileHandler to use socket.gethostbyname_ex which returns all
of the IPv4 addresses associated with a hostname:
>>> socket.gethostbyname_ex(socket.gethostname())
('myhost.net', [], ['10.52.12.205', '10.52.12.105']) Attached patches for 2.x urllib2 and 3.x urllib do that. Note that
|
While you're poking around in urllib2, perhaps I can interest you in |
Thanks for the patch, Ned. Fixed in the trunk revision 77058. |
Merged the fixes in r77059, r77060 and r77061 |
Could you please add this change to test_urllib2.py as well? It has the following line: But urllib2.py has the change related to this bug. (Pdb) socket.gethostbyname_ex(socket.gethostname())[2] |
Zsolt, The change in the urllib2 was at a place where tuple of all local ips
+ localaddr = socket.gethostbyname('localhost') If this is sufficient, this change can be made in the trunk. |
The test which failed was HandlerTests.test_file, and I'm using python 2.7.1. socket.gethostbyname('localhost') returns "127.0.0.1" which is ok, but in the unittest it's already tested (line 671). The problem is that my /etc/hosts file contains a different IP than the DNS (I cannot change this behaviour as I'm not the administrator of the host) and that's the difference between gethostbyname and gethostbyname_ex. The unittest creates an url which is not local (from urllib2 point of view). I'm attaching a patch which has fixed my problem. |
+ localaddr = socket.gethostbyname_ex(socket.gethostname())[2][0] May not be a generic solution, because in another system the other ip |
The order of the IP addresses doesn't matter as urllib2 is flexible enough to handle all local IP addresses as local (that was the original bug - it handled only one IP returned by gethostbyname which returned a random IP if there were more than one). So picking up the first IP is ok I think as the order of the IP addresses doesn't matter - urllib2 will handle all of them as local. The problem is that gethostbyname doesn't guarantee that it returns one IP address from the set returned by gethostbyname_ex as gethostbyname looks up the name in /etc/hosts file first (or as configured in NSS). |
Well, ignore my comment on order of ip addresses. It definitely does not matter in this case for test_urllib2. However, readability does matter again as per my previous explanation, since http://localhost/ was being exercised in the test_file, gethostbyname('localhost') is much better than that return value's [2][0] element. I overlooked one thing in your first message, namely gethostbyname and gethostbyname_ex()[2] returning completely different ips and turning out to be exclusive. This should not be the case. gethostbyname_ex()[2] should include the ip which was returned by gethostbyname. If it were the case, the test would not have failed as well. And btw, both these are supposed have similar behavior (The default action is to query named(8), followed by /etc/hosts) only thing is gethostbyname_ex uses the reentrant c function call and is thread-safe. (You may probably want to identify the problem for the difference in o/p there) And for this bug report, I am still inclined to having 'localhost' for readability purposes or leaving it as such because the problem seems be elsewhere. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: