Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Proxies can be sent as a string #300

Merged
merged 8 commits into from
Jun 13, 2018
Merged

Proxies can be sent as a string #300

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
392ed80
proxies can be sent as a string now
paulefoe Jun 4, 2018
5b4582a
changed self.proxies
paulefoe Jun 6, 2018
ca18e9b
added test for http and https, removed modification of a sche,e insid…
paulefoe Jun 6, 2018
06f60fe
fix
paulefoe Jun 6, 2018
0b180e5
chanded str to string_compare for compatibility
paulefoe Jun 6, 2018
bd18569
test for checking if both schemes are respected
paulefoe Jun 8, 2018
e63a625
separete test for checking both schemes, changed proxies
paulefoe Jun 11, 2018
7da916c
removed print and unnecessary explicit scheme
paulefoe Jun 13, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
5 changes: 5 additions & 0 deletions geopy/geocoders/base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,8 @@ class options(object):
If specified, tunnel requests through the specified proxy.
E.g., ``{"https": "192.0.2.0"}``. For more information, see
documentation on :class:`urllib.request.ProxyHandler`.
Can be sent as a string ``"https://192.0.2.0"``, this will
automatically set the scheme to the ``https``
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest, I don't like it 🤔. But I'm not ready to propose a better phrasing for now as well.

Let's just keep it, but I'll get to this later after merging.


default_scheme
Use ``'https'`` or ``'http'`` as the API URL's scheme.
Expand Down Expand Up @@ -185,6 +187,9 @@ def __init__(
else options.default_ssl_context)

if self.proxies:
if isinstance(self.proxies, string_compare):
self.proxies = {'http': self.proxies, 'https': self.proxies}

opener = build_opener_with_context(
self.ssl_context,
ProxyHandler(self.proxies),
Expand Down
18 changes: 18 additions & 0 deletions test/test_proxy.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,3 +76,21 @@ def test_ssl_context_with_proxy_is_respected(self):
geocoder_dummy.geocode(self.remote_website_https)
self.assertIn('SSL', str(cm.exception))
self.assertEqual(1, len(self.proxy_server.requests))

def test_geocoder_constructor_uses_str_proxy(self):
base_http = urlopen(self.remote_website_http, timeout=self.timeout)
base_html = base_http.read()
geocoder_dummy = DummyGeocoder(proxies=self.proxy_url, scheme='http',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I don't think we need to test two schemes in two almost identical integration tests, which aren't cheap with regards to time.

It would be better to keep the previous single test, and add a new unit test here. Something like this:

g = DummyGeocoder(proxies=self.proxy_url, scheme='http')
self.assertDictEqual(g.proxies, {'http': self.proxy_url, 'https': self.proxy_url})

That should be enough. We already tested that the g.proxies dict is respected in the integration tests, but we didn't check if both schemes are respected – which is exactly what this test does.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if I get this right, but this will assert error:

self.assertDictEqual(g.proxies, {'http': self.proxy_url, 'https': self.proxy_url})

So I added 2 DummyGeocoders to check if both schemes are respected.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you by chance missed this comment? #300 (comment)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also it would've been better if it was in a separate test_* function.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yeah, I did miss this comment, pardon my inattention.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any reason to set the scheme kwarg explicitly? If not, could you please remove that?

timeout=self.timeout)
print(geocoder_dummy.scheme, geocoder_dummy.proxies, 'proxy')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙃

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

whoops 😅

self.assertEqual(0, len(self.proxy_server.requests))
self.assertEqual(
base_html,
geocoder_dummy.geocode(self.remote_website_http)
)
self.assertEqual(1, len(self.proxy_server.requests))

def test_geocoder_constructor_have_both_schemes_proxy(self):
g = DummyGeocoder(proxies=self.proxy_url, scheme='http')
self.assertDictEqual(g.proxies, {'http': self.proxy_url,
'https': self.proxy_url})