AttributeError when recording HTTPS request

[larkin]

Not sure if this is a problem with the HTTPS implementation in vcrpy or the boto implementation specifically, but when recording a s3 request made via boto, VCR throws the following error:

Traceback (most recent call last):
  File "repro.py", line 11, in <module>
    k.set_contents_from_string('hello world i am a string')
  File "/python2.7/site-packages/boto/s3/key.py", line 1379, in set_contents_from_string
    encrypt_key=encrypt_key)
  File "/python2.7/site-packages/boto/s3/key.py", line 1246, in set_contents_from_file
    chunked_transfer=chunked_transfer, size=size)
  File "/python2.7/site-packages/boto/s3/key.py", line 725, in send_file
    chunked_transfer=chunked_transfer, size=size)
  File "/python2.7/site-packages/boto/s3/key.py", line 914, in _send_file_internal
    query_args=query_args
  File "/python2.7/site-packages/boto/s3/connection.py", line 633, in make_request
    retry_handler=retry_handler
  File "/python2.7/site-packages/boto/connection.py", line 1040, in make_request
    retry_handler=retry_handler)
  File "/python2.7/site-packages/boto/connection.py", line 913, in _mexe
    request.body, request.headers)
  File "/python2.7/site-packages/boto/s3/key.py", line 815, in sender
    http_conn.send(chunk)
  File "/python2.7/site-packages/vcr/stubs/__init__.py", line 101, in send
    self._vcr_request.body = (self._vcr_request.body or '') + data
AttributeError: VCRHTTPSConnection instance has no attribute '_vcr_request'

Assuming you have a AWS_CREDENTIAL_FILE, you can reproduce it with this code:

from boto.s3.connection import S3Connection
from boto.s3.key import Key
import vcr

s3_conn = S3Connection()
s3_bucket = s3_conn.get_bucket('some-bucket') # a bucket you can access

with vcr.use_cassette('test.yml'):
    k = Key(s3_bucket)
    k.key = 'test.txt'
    k.set_contents_from_string('hello world i am a string')

If I have time later today I will try and narrow this down to something that doesn't require boto, and if I can, then I should be able to make a fix too. Haven't really dug into vcrpy's source yet so any pointers are welcome.

[kevin1024]

Not sure if this is a problem with the HTTPS implementation in vcrpy or the boto implementation specifically

Probably an issue with VCR, it works by monkeypatching out the httplib stuff with its own VCRHTTPSConnection, which might not implement the (bizarre) stdlib API 100% correctly.

VCRHTTPSConnection instance has no attribute '_vcr_request'

This tells me that since the _vcr_request doesn't exist on the stubbed out connection, Boto is probably calling send())) without calling request() first, so the connection object doesn't exist yet.

send() probably needs to check and if self._vcr_request.body doesn't exist yet, maybe initialize it by calling request() first. Except... send() doesn't give us the host it's connecting to. So how does httplib know which host to connect to?

Just of the top of my head, I don't have time to look into this too deeply right now :)

[kevin1024]

After looking into this a bit futher, I think this is caused by Boto's reuse of the existing HTTP connection (keepalive). This breaks some of my assumptions I made when designing the mock.

[kevin1024]

I think there might also be an issue with the signature process and timestamps, might need to mock out the time too. Yikes :)

[kevin1024]

Actually the timestamp thing can be solved with a customer marcher that ignores the Date header.

[kevin1024]

OK, this should be fixed in master, give it a try and let me know if/how it works for you.