Simple test script fails with boto3

[a1exsh]

import boto3
import moto

@moto.mock_ec2
def test_ec2():
  ec2 = boto3.client('ec2', region_name='eu-west-1')
  ec2.describe_instances()

test_ec2()

Gives me this:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/moto/core/models.py", line 71, in wrapper
    result = func(*args, **kwargs)
  File "<stdin>", line 4, in test_ec2
  File "/usr/local/lib/python2.7/dist-packages/botocore/client.py", line 310, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python2.7/dist-packages/botocore/client.py", line 384, in _make_api_call
    operation_model, request_dict)
  File "/usr/local/lib/python2.7/dist-packages/botocore/endpoint.py", line 112, in make_request
    return self._send_request(request_dict, operation_model)
  File "/usr/local/lib/python2.7/dist-packages/botocore/endpoint.py", line 141, in _send_request
    success_response, exception):
  File "/usr/local/lib/python2.7/dist-packages/botocore/endpoint.py", line 214, in _needs_retry
    caught_exception=caught_exception)
  File "/usr/local/lib/python2.7/dist-packages/botocore/hooks.py", line 226, in emit
    return self._emit(event_name, kwargs)
  File "/usr/local/lib/python2.7/dist-packages/botocore/hooks.py", line 209, in _emit
    response = handler(**kwargs)
  File "/usr/local/lib/python2.7/dist-packages/botocore/retryhandler.py", line 183, in __call__
    if self._checker(attempts, response, caught_exception):
  File "/usr/local/lib/python2.7/dist-packages/botocore/retryhandler.py", line 250, in __call__
    caught_exception)
  File "/usr/local/lib/python2.7/dist-packages/botocore/retryhandler.py", line 265, in _should_retry
    return self._checker(attempt_number, response, caught_exception)
  File "/usr/local/lib/python2.7/dist-packages/botocore/retryhandler.py", line 313, in __call__
    caught_exception)
  File "/usr/local/lib/python2.7/dist-packages/botocore/retryhandler.py", line 222, in __call__
    return self._check_caught_exception(attempt_number, caught_exception)
  File "/usr/local/lib/python2.7/dist-packages/botocore/retryhandler.py", line 355, in _check_caught_exception
    raise caught_exception
TypeError: __str__ returned non-string (type SysCallError)

Any ideas? Is moto supposed to mock the ec2 object or does it work by running an HTTP server and mocking responses only?

Thanks.

[a1exsh]

The inner exception seems to be this one:

2015-12-02 12:13:14,499 botocore.endpoint [DEBUG] Sending http request: <PreparedRequest [POST]>
2015-12-02 12:13:14,501 botocore.vendored.requests.packages.urllib3.connectionpool [INFO] Starting new HTTPS connection (1): ec2.eu-west-1.amazonaws.com
2015-12-02 12:13:14,508 botocore.endpoint [DEBUG] Exception received when sending HTTP request.
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/botocore/endpoint.py", line 169, in _get_response
    proxies=self.proxies, timeout=self.timeout)
  File "/usr/local/lib/python2.7/dist-packages/botocore/vendored/requests/sessions.py", line 573, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/botocore/vendored/requests/adapters.py", line 370, in send
    timeout=timeout
  File "/usr/local/lib/python2.7/dist-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 544, in urlopen
    body=body, headers=headers)
  File "/usr/local/lib/python2.7/dist-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 344, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=conn.timeout)
  File "/usr/local/lib/python2.7/dist-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 314, in _raise_timeout
    if 'timed out' in str(err) or 'did not complete (read)' in str(err):  # Python 2.6
TypeError: __str__ returned non-string (type SysCallError)

And the err is: SSLError('bad handshake', SysCallError(32, 'Broken pipe')), where type(err) -> <class 'ssl.SSLError'>.

[a1exsh]

Looks like this HTTPretty issue is related: gabrielfalcao/HTTPretty#242

[a1exsh]

Confirmed that this is a combination of Python2 and pyopenssl urllib3 mucking: this code Just Works(tm) with Python3.

[a1exsh]

The following solves the problem for me:

try:
  from botocore.vendored.requests.packages.urllib3 import connection

  def fake_ssl_wrap_socket(sock, *args, **kwargs):
    return sock
  connection.ssl_wrap_socket = fake_ssl_wrap_socket
except ImportError:
  pass

import boto3
import moto
...

The key thing here is that botocore ships with requests (which in turn ships with urllib3, how weird is that already?), and we need to override ssl_wrap_socket which is replaced within urllib3.contrib.pyopenssl. To make things even more interesting, the replacement happens in requests/__init__.py so while we're importing it, the replacement has already happened. That's why we need to override connection.ssl_wrap_socket after the fact.

[spulec]

I'm not able to recreate the failure using the latest version of moto and httpretty 0.8.10. Can you give me any more details? OS?

[a1exsh]

There you go :)

$ uname -a
Linux ashulgin01 3.19.0-37-generic #42-Ubuntu SMP Fri Nov 20 18:22:05 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 15.04
Release:    15.04
Codename:   vivid

$ pip list | grep -E 'httpretty|boto|moto'
boto (2.38.0)
boto3 (1.2.1)
botocore (1.3.6)
httpretty (0.8.10)
moto (0.4.19)

And it's Python 2.7.9. The problem does not appear with Python 3.4.3.

[spulec]

I think the answer to this is going to be a resolution of gabrielfalcao/HTTPretty#242

[spulec]

This should no longer be an issue on master. Feel free to reopen if you see otherwise.