Extended conan.conf syntax to permit host-specific proxies

[dawidcha]

Changelog: (Feature): Extend syntax to permit host-specific proxies (backward compatible)
Docs: conan-io/docs#1241

• Fixes [proxies] section in conan.conf does not permit proxies for specific targets #4920
• No related docs found in Conan docs repo; documentation added in default conan.conf script

[CLAassistant]

All committers have signed the CLA.

[dawidcha]

Just noticed I'd left 'import sys' in used for debugging - it can be removed.

[memsharded]

This PR looks useful and nice that it seems backwards compatible, thanks for contributing it.

It also seems that it would be necessary to bump the minimum requirement of requests to 2.8.1, as the one that is currently declared in requirements.txt (2.7.0), won't be enough: http://docs.python-requests.org/en/v2.8.1/user/advanced/

@lasote do you recall some reason why we were requesting 2.7.0? Could it be upgraded?

[lasote]

I cannot recall, probably we are good upgrading requests.

[lasote]

I think we would need a test with more than 1 host line.

[dawidcha]

Added as requested

[jgsogo]

Hi! Thanks for the contribution

One suggestion to consider: IMO, it is easier to understand (and to parse) a plain list of schema[://hostname] = proxy that will be translated directly into a dictionary:

[proxies]
https = http://default-proxy.acme.com
https://conan-repo.cloud.acme.com = http://cloud-proxy.acme.com
http = http://other.proxy

I assume that requests module will handle each url from most specific to most generic one, so the order in the dictionary is not important (no need for an OrderedDict).

[dawidcha]

I'm away from my Dev rig at present, but fwiw, I did run a manual integration test - ran:

conan search Poco* -r conan-center

• Withdefault conan.conf
  (success)
• With

https=http://1.2.3.4

(can't find proxy)

• With

https=
       conan.bintray.com=http://1.2.3.4

(can't find proxy)

• With

https=
       google.com=http://1.2.3.4

(Success)

Not totally conclusive, but it suggests requests is working off the develop branch. Maybe it's an undocumented feature in 2.7.0?

I'll improve the test as suggested this (UK) evening

[dawidcha]

Hi! Thanks for the contribution

One suggestion to consider: IMO, it is easier to understand (and to parse) a plain list of schema[://hostname] = proxy that will be translated directly into a dictionary:

[proxies]
https = http://default-proxy.acme.com
https://conan-repo.cloud.acme.com = http://cloud-proxy.acme.com
http = http://other.proxy

This doesn't work as ConfigParser treats the : in the URL as a name/Val separator, by default. That's configurable, but not in a way that retains 100% backward compatibility

I assume that requests module will handle each url from most specific to most generic one, so the order in the dictionary is not important (no need for an OrderedDict).

Yes, I kinda assumed that also. The docs aren't clear. I can investigate.

[dawidcha]

Note: the proxy search code does indeed search for host-constrained proxies before general proxies

From 2.8.1
https://github.com/kennethreitz/requests/blob/v2.8.1/requests/utils.py : select_proxy(...)

Looks like my install had pulled in 2.21.0 which also worked.

[jgsogo]

btw, proxies are documented here in the reference: https://docs.conan.io/en/latest/reference/config_files/conan.conf.html#proxies

[dawidcha]

btw, proxies are documented here in the reference: https://docs.conan.io/en/latest/reference/config_files/conan.conf.html#proxies

Aha, thanks. I'll update and submit a docs pull request

[dawidcha]

Pull request for docs here
conan-io/docs#1241

[memsharded]

Hi @dawidcha

I have proposed in dawidcha#1 some minor improvements, but also bumping the min version of requests to 2.8.1, I think this PR should do it.

Thanks again for your contribution, great job!

[lasote]

@dawidcha Could you take a look to the @memsharded PR? Thanks!

[lasote]

ping @dawidcha

[lasote]

pong @dawidcha

[memsharded]

Didn't want to have this waiting for so long, as it was good.
Merged with develop, solved conflicts, merged my small refactors and merged.

Will be in Conan 1.16, thanks @dawidcha for the contribution!