botocore/Boto 3 calls not getting proxy settings

[riposte-sec]

I have an Ubuntu and a CentOS instance that has issues with botocore calls.
All boto2 calls are fine.
Botocore calls are not getting the proxy settings.
Examples of the log messages are:
security_monkey-deploy.log:51326:2016-01-15 08:04:11,143 DEBUG: Adding ('iamrole', u'', 'universal') to the exceptions list. Exception was: "Problem Connecting to iamrole//None:\nHTTPSConneconPool(host='iam.amazonaws.com', port=443): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<botocore.awsrequest.AWSHTTPSConnection object at 0x7fa6880f7710>, 'Connection to iam.amazonawcom timed out. (connect timeout=60)'))" [in /usr/local/src/security_monkey/security_monkey/watcher.py:161]

security_monkey-deploy.log:53229:2016-01-15 09:19:15,516 DEBUG: Adding ('elb', u'', u'ap-southeast-2', u'<ELB_ID>') to the exceptions list. Exception was: HTTPSConnectionPool(host='elasticloadbalancing.ap-southeast-2.amazonaws.com', port=443): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<botocore.awsrequest.AWSHTTPSConnection object at 0x7fa689cdf690>, 'Connection to elasticloadbalancing.ap-southeast-2.amazonaws.com timed out. (connect timeout=60)')) [in /usr/local/src/security_monkey/security_monkey/watcher.py:161]

Netstat -an shows that these calls are trying to go straight to the API instead of the proxy:
tcp 0 1 10.x.x.x:48739 54.240.193.98:443 SYN_SENT

The proxies are set as part of the environment profile.

Boto versions are:
boto (2.38.0)
boto3 (1.2.3)
botocore (1.3.17)
python --version
Python 2.7.5

urllib3 version
urllib3 (1.14)

Has anyone else seen this and found a resolution?
I could setup squid and transparently proxy everything, but I'd really prefer to see why the botocore calls are failing so if anyone else has the problem it is documented - as I suspect I am overlooking something simple.

[scriptsrc]

https://github.com/boto/botocore/blob/develop/botocore/endpoint.py#L272

    def _get_proxies(self, url):
        # We could also support getting proxies from a config file,
        # but for now proxy support is taken from the environment.
        return get_environ_proxies(url)

It looks like botocore may be relying on urllib to pull proxy information out of environment variables and does not accept proxy settings from the boto config file.

https://docs.python.org/3/library/urllib.request.html#urllib.request.getproxies

urllib.request.getproxies()
This helper function returns a dictionary of scheme to proxy server URL mappings. It scans the environment for variables named _proxy, in a case insensitive approach,

I would think you could set a https_proxy environment variable and it would work. Hopefully http_proxy wouldn't also be required.

Let us know if this works.

[riposte-sec]

I'd actually already tested that - when I ran the code independently from the python console I had no problem returning the dict (I'd played around with the proxy settings as http_proxy and https_proxy in both lower and upper case.
I should add that I am new to Flask and botocore/boto3 so there is still a huge chance I am doing something stupid.
I also noted over at botocore that they some day are going to add support for a file as per boto but I am not sure how long away that is. I thought I'd raise this here in case wiser heads knew better, but I suspect the problem is more to do with botocore's handling of that dict.

[riposte-sec]

I am going to close this here. I basically had to place the dict, which getproxies did get into botocore/vendored/requests/utils.py's get_envion_proxies function as the return value. - this is by no means a fix to the problem, but I'll raise it over with botocore. So quite odd, getproxies worked when I ran it manually, but something in the way it was called was not returning it correctly to endpoint.py.

Hopefully if anyone has this problem this helps...

[omkarkulkarni]

@finalriposte can you please share that part of the code where you use proxy with boto3. Even I am facing the same issue as you. I am confused on how to set the proxy settings though. Thank you.