Deleting an S3 object fails for now reason

[nikolay]

aws s3 s3://bucket/object yields delete failed: s3://bucket/object 'str' object has no attribute 'text' and return an error code.

[nikolay]

$ aws --version
aws-cli/1.5.1 Python/2.7.8 Darwin/14.0.0

[kyleknap]

Can you add a debug log by appending --debug to command?

[nikolay]

$ aws s3 rm --debug s3://bucketname/cfn/test.json
2014-10-08 15:49:57,460 - MainThread - awscli.clidriver - DEBUG - CLI version: aws-cli/1.5.1 Python/2.7.8 Darwin/14.0.0, botocore version: 0.65.0
2014-10-08 15:49:57,460 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.s3.anonymous: calling handler <function uri_param at 0x10e6a35f0>
2014-10-08 15:49:57,462 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.rm.paths: calling handler <function uri_param at 0x10e6a35f0>
2014-10-08 15:49:57,462 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.rm.dryrun: calling handler <function uri_param at 0x10e6a35f0>
2014-10-08 15:49:57,462 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.custom.rm: calling handler <awscli.argprocess.ParamShorthand object at 0x10e7e4e10>
2014-10-08 15:49:57,462 - MainThread - awscli.argprocess - DEBUG - Detected structure: scalar
2014-10-08 15:49:57,462 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.rm.quiet: calling handler <function uri_param at 0x10e6a35f0>
2014-10-08 15:49:57,462 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.custom.rm: calling handler <awscli.argprocess.ParamShorthand object at 0x10e7e4e10>
2014-10-08 15:49:57,462 - MainThread - awscli.argprocess - DEBUG - Detected structure: scalar
2014-10-08 15:49:57,462 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.rm.anonymous: calling handler <function uri_param at 0x10e6a35f0>
2014-10-08 15:49:57,462 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.rm.anonymous: calling handler <function uri_param at 0x10e6a35f0>
2014-10-08 15:49:57,462 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.rm.only-show-errors: calling handler <function uri_param at 0x10e6a35f0>
2014-10-08 15:49:57,463 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.custom.rm: calling handler <awscli.argprocess.ParamShorthand object at 0x10e7e4e10>
2014-10-08 15:49:57,463 - MainThread - awscli.argprocess - DEBUG - Detected structure: scalar
2014-10-08 15:49:57,463 - MainThread - botocore.service - DEBUG - Creating service object for: s3
2014-10-08 15:49:57,499 - MainThread - botocore.hooks - DEBUG - Event service-data-loaded.s3: calling handler <function signature_overrides at 0x10e405de8>
2014-10-08 15:49:57,499 - MainThread - botocore.hooks - DEBUG - Event service-created: calling handler <function register_retries_for_service at 0x10e405c80>
2014-10-08 15:49:57,502 - MainThread - botocore.handlers - DEBUG - Registering retry handlers for service: Service(s3)
2014-10-08 15:49:57,504 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: env
2014-10-08 15:49:57,504 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: shared-credentials-file
2014-10-08 15:49:57,504 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: config-file
2014-10-08 15:49:57,504 - MainThread - botocore.credentials - INFO - Credentials found in config file: ~/.aws/config
2014-10-08 15:49:57,514 - MainThread - botocore.service - DEBUG - Creating service object for: s3
2014-10-08 15:49:57,514 - MainThread - botocore.hooks - DEBUG - Event service-data-loaded.s3: calling handler <function signature_overrides at 0x10e405de8>
2014-10-08 15:49:57,515 - MainThread - botocore.hooks - DEBUG - Event service-created: calling handler <function register_retries_for_service at 0x10e405c80>
2014-10-08 15:49:57,520 - MainThread - botocore.handlers - DEBUG - Registering retry handlers for service: Service(s3)
2014-10-08 15:49:57,527 - MainThread - botocore.service - DEBUG - Creating operation objects for: Service(s3)
2014-10-08 15:49:57,532 - MainThread - botocore.operation - DEBUG - Operation:HeadObject called with kwargs: {'bucket': u'bucketname', 'key': u'cfn/test.json'}
2014-10-08 15:49:57,534 - MainThread - botocore.hooks - DEBUG - Event before-call.s3.HeadObject: calling handler <function sse_md5 at 0x10e405aa0>
2014-10-08 15:49:57,534 - MainThread - botocore.hooks - DEBUG - Event before-call.s3.HeadObject: calling handler <function add_expect_header at 0x10e405e60>
2014-10-08 15:49:57,534 - MainThread - botocore.endpoint - DEBUG - Making request for <botocore.model.OperationModel object at 0x10f49bb10> (verify_ssl=True) with params: {'query_string': {}, 'headers': {}, 'url_path': u'/bucketname/cfn/test.json', 'body': '', 'method': u'HEAD'}
2014-10-08 15:49:57,534 - MainThread - botocore.hooks - DEBUG - Event before-auth.s3: calling handler <function fix_s3_host at 0x10e405b90>
2014-10-08 15:49:57,535 - MainThread - botocore.handlers - DEBUG - Checking for DNS compatible bucket for: https://s3.amazonaws.com/bucketname/cfn/test.json
2014-10-08 15:49:57,535 - MainThread - botocore.handlers - DEBUG - URI updated to: https://bucketname.s3.amazonaws.com/cfn/test.json
2014-10-08 15:49:57,535 - MainThread - botocore.auth - DEBUG - Calculating signature using hmacv1 auth.
2014-10-08 15:49:57,535 - MainThread - botocore.auth - DEBUG - HTTP request method: HEAD
2014-10-08 15:49:57,535 - MainThread - botocore.auth - DEBUG - StringToSign:
HEAD


Wed, 08 Oct 2014 22:49:57 GMT
/bucketname/cfn/test.json
2014-10-08 15:49:57,543 - MainThread - botocore.endpoint - DEBUG - Sending http request: <PreparedRequest [HEAD]>
2014-10-08 15:49:57,544 - MainThread - botocore.vendored.requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): bucketname.s3.amazonaws.com
2014-10-08 15:49:58,018 - MainThread - botocore.vendored.requests.packages.urllib3.connectionpool - DEBUG - "HEAD /cfn/test.json HTTP/1.1" 200 0
2014-10-08 15:49:58,019 - MainThread - botocore.parsers - DEBUG - Response body:

2014-10-08 15:49:58,022 - MainThread - botocore.hooks - DEBUG - Event needs-retry.s3.HeadObject: calling handler <botocore.retryhandler.RetryHandler object at 0x10e83ec90>
2014-10-08 15:49:58,022 - MainThread - botocore.retryhandler - DEBUG - No retry needed.
2014-10-08 15:49:58,022 - MainThread - botocore.hooks - DEBUG - Event after-call.s3.HeadObject: calling handler <awscli.errorhandler.ErrorHandler object at 0x10e7e4e50>
2014-10-08 15:49:58,022 - MainThread - awscli.errorhandler - DEBUG - HTTP Response Code: 200
2014-10-08 15:49:58,031 - MainThread - botocore.service - DEBUG - Creating service object for: s3
2014-10-08 15:49:58,031 - MainThread - botocore.hooks - DEBUG - Event service-data-loaded.s3: calling handler <function signature_overrides at 0x10e405de8>
2014-10-08 15:49:58,031 - MainThread - botocore.hooks - DEBUG - Event service-created: calling handler <function register_retries_for_service at 0x10e405c80>
2014-10-08 15:49:58,031 - MainThread - botocore.handlers - DEBUG - Registering retry handlers for service: Service(s3)
2014-10-08 15:49:58,032 - MainThread - awscli.customizations.s3.executor - DEBUG - Submitting task: <awscli.customizations.s3.tasks.BasicTask object at 0x10f4a4f10>
2014-10-08 15:49:58,032 - MainThread - awscli.customizations.s3.executor - DEBUG - Queueing end sentinel for worker thread (priority: 11)
2014-10-08 15:49:58,032 - Thread-5 - awscli.customizations.s3.executor - DEBUG - Worker thread invoking task: <awscli.customizations.s3.tasks.BasicTask object at 0x10f4a4f10>
2014-10-08 15:49:58,033 - Thread-5 - botocore.operation - DEBUG - Operation:DeleteObject called with kwargs: {'bucket': u'bucketname', 'key': u'cfn/test.json'}
2014-10-08 15:49:58,033 - Thread-6 - awscli.customizations.s3.executor - DEBUG - Shutdown request received in worker thread, shutting down worker thread.
2014-10-08 15:49:58,033 - MainThread - awscli.customizations.s3.executor - DEBUG - Queueing end sentinel for worker thread (priority: 11)
2014-10-08 15:49:58,034 - Thread-5 - botocore.hooks - DEBUG - Event before-call.s3.DeleteObject: calling handler <function add_expect_header at 0x10e405e60>
2014-10-08 15:49:58,034 - MainThread - awscli.customizations.s3.executor - DEBUG - Queueing end sentinel for worker thread (priority: 11)
2014-10-08 15:49:58,035 - Thread-5 - botocore.endpoint - DEBUG - Making request for <botocore.model.OperationModel object at 0x10f49b290> (verify_ssl=True) with params: {'query_string': {}, 'headers': {}, 'url_path': u'/bucketname/cfn/test.json', 'body': '', 'method': u'DELETE'}
2014-10-08 15:49:58,035 - Thread-5 - botocore.hooks - DEBUG - Event before-auth.s3: calling handler <function fix_s3_host at 0x10e405b90>
2014-10-08 15:49:58,036 - Thread-5 - botocore.handlers - DEBUG - Checking for DNS compatible bucket for: https://s3.amazonaws.com/bucketname/cfn/test.json
2014-10-08 15:49:58,036 - Thread-5 - botocore.handlers - DEBUG - URI updated to: https://bucketname.s3.amazonaws.com/cfn/test.json
2014-10-08 15:49:58,036 - Thread-7 - awscli.customizations.s3.executor - DEBUG - Shutdown request received in worker thread, shutting down worker thread.
2014-10-08 15:49:58,036 - MainThread - awscli.customizations.s3.executor - DEBUG - Queueing end sentinel for worker thread (priority: 11)
2014-10-08 15:49:58,036 - Thread-5 - botocore.auth - DEBUG - Calculating signature using hmacv1 auth.
2014-10-08 15:49:58,037 - Thread-5 - botocore.auth - DEBUG - HTTP request method: DELETE
2014-10-08 15:49:58,037 - Thread-8 - awscli.customizations.s3.executor - DEBUG - Shutdown request received in worker thread, shutting down worker thread.
2014-10-08 15:49:58,038 - Thread-5 - botocore.auth - DEBUG - StringToSign:
DELETE


Wed, 08 Oct 2014 22:49:58 GMT
/bucketname/cfn/test.json
2014-10-08 15:49:58,038 - Thread-5 - botocore.endpoint - DEBUG - Sending http request: <PreparedRequest [DELETE]>
2014-10-08 15:49:58,037 - MainThread - awscli.customizations.s3.executor - DEBUG - Queueing end sentinel for worker thread (priority: 11)
2014-10-08 15:49:58,037 - Thread-9 - awscli.customizations.s3.executor - DEBUG - Shutdown request received in worker thread, shutting down worker thread.
2014-10-08 15:49:58,039 - MainThread - awscli.customizations.s3.executor - DEBUG - Queueing end sentinel for worker thread (priority: 11)
2014-10-08 15:49:58,039 - MainThread - awscli.customizations.s3.executor - DEBUG - Queueing end sentinel for worker thread (priority: 11)
2014-10-08 15:49:58,039 - Thread-10 - awscli.customizations.s3.executor - DEBUG - Shutdown request received in worker thread, shutting down worker thread.
2014-10-08 15:49:58,039 - Thread-11 - awscli.customizations.s3.executor - DEBUG - Shutdown request received in worker thread, shutting down worker thread.
2014-10-08 15:49:58,040 - MainThread - awscli.customizations.s3.executor - DEBUG - Queueing end sentinel for worker thread (priority: 11)
2014-10-08 15:49:58,040 - Thread-12 - awscli.customizations.s3.executor - DEBUG - Shutdown request received in worker thread, shutting down worker thread.
2014-10-08 15:49:58,040 - MainThread - awscli.customizations.s3.executor - DEBUG - Queueing end sentinel for worker thread (priority: 11)
2014-10-08 15:49:58,040 - Thread-13 - awscli.customizations.s3.executor - DEBUG - Shutdown request received in worker thread, shutting down worker thread.
2014-10-08 15:49:58,041 - MainThread - awscli.customizations.s3.executor - DEBUG - Queueing end sentinel for worker thread (priority: 11)
2014-10-08 15:49:58,041 - MainThread - awscli.customizations.s3.executor - DEBUG - Waiting for thread to shutdown: <Worker(Thread-5, started daemon 4564230144)>
2014-10-08 15:49:58,041 - Thread-14 - awscli.customizations.s3.executor - DEBUG - Shutdown request received in worker thread, shutting down worker thread.
2014-10-08 15:49:58,164 - Thread-5 - botocore.vendored.requests.packages.urllib3.connectionpool - DEBUG - "DELETE /cfn/test.json HTTP/1.1" 204 0
2014-10-08 15:49:58,165 - Thread-5 - botocore.parsers - DEBUG - Response body:

2014-10-08 15:49:58,165 - Thread-5 - awscli.customizations.s3.tasks - DEBUG - 'str' object has no attribute 'text'
Traceback (most recent call last):
  File "/Users/username/Projects/projectname/.env/pyenv/versions/virtualenv/lib/python2.7/site-packages/awscli/customizations/s3/tasks.py", line 94, in _execute_task
    getattr(filename, filename.operation_name)(**kwargs)
  File "/Users/username/Projects/projectname/.env/pyenv/versions/virtualenv/lib/python2.7/site-packages/awscli/customizations/s3/fileinfo.py", line 308, in delete
    params)
  File "/Users/username/Projects/projectname/.env/pyenv/versions/virtualenv/lib/python2.7/site-packages/awscli/customizations/s3/utils.py", line 220, in operate
    http_response, response_data = operation.call(**kwargs)
  File "/Users/username/Projects/projectname/.env/pyenv/versions/virtualenv/lib/python2.7/site-packages/botocore/operation.py", line 84, in call
    response = endpoint.make_request(self.model, request_dict)
  File "/Users/username/Projects/projectname/.env/pyenv/versions/virtualenv/lib/python2.7/site-packages/botocore/endpoint.py", line 74, in make_request
    return self._send_request(prepared_request, operation_model)
  File "/Users/username/Projects/projectname/.env/pyenv/versions/virtualenv/lib/python2.7/site-packages/botocore/endpoint.py", line 130, in _send_request
    response, exception = self._get_response(request, operation_model, attempts)
  File "/Users/username/Projects/projectname/.env/pyenv/versions/virtualenv/lib/python2.7/site-packages/botocore/endpoint.py", line 155, in _get_response
    http_response), None)
  File "/Users/username/Projects/projectname/.env/pyenv/versions/virtualenv/lib/python2.7/site-packages/botocore/response.py", line 117, in get_response
    operation_model.output_shape)
  File "/Users/username/Projects/projectname/.env/pyenv/versions/virtualenv/lib/python2.7/site-packages/botocore/parsers.py", line 157, in parse
    return self._do_parse(response, shape)
  File "/Users/username/Projects/projectname/.env/pyenv/versions/virtualenv/lib/python2.7/site-packages/botocore/parsers.py", line 479, in _do_parse
    member_shapes, final_parsed)
  File "/Users/username/Projects/projectname/.env/pyenv/versions/virtualenv/lib/python2.7/site-packages/botocore/parsers.py", line 536, in _parse_non_payload_attrs
    member_shape, headers[header_name])
  File "/Users/username/Projects/projectname/.env/pyenv/versions/virtualenv/lib/python2.7/site-packages/botocore/parsers.py", line 168, in _parse_shape
    return handler(shape, node)
  File "/Users/username/Projects/projectname/.env/pyenv/versions/virtualenv/lib/python2.7/site-packages/botocore/parsers.py", line 288, in _handle_boolean
    if node.text == 'true':
AttributeError: 'str' object has no attribute 'text'
2014-10-08 15:49:58,179 - Thread-5 - awscli.customizations.s3.executor - DEBUG - Shutdown request received in worker thread, shutting down worker thread.
2014-10-08 15:49:58,179 - Thread-1 - awscli.customizations.s3.executor - DEBUG - Received print task: PrintTask(message=u"delete failed: s3://bucketname/cfn/test.json 'str' object has no attribute 'text'", error=True, total_parts=None, warning=None)
delete failed: s3://bucketname/cfn/test.json 'str' object has no attribute 'text'
2014-10-08 15:49:58,223 - MainThread - awscli.customizations.s3.executor - DEBUG - Thread has been shutdown: <Worker(Thread-5, stopped daemon 4564230144)>
2014-10-08 15:49:58,223 - MainThread - awscli.customizations.s3.executor - DEBUG - Waiting for thread to shutdown: <Worker(Thread-6, stopped daemon 4568436736)>
2014-10-08 15:49:58,223 - MainThread - awscli.customizations.s3.executor - DEBUG - Thread has been shutdown: <Worker(Thread-6, stopped daemon 4568436736)>
2014-10-08 15:49:58,223 - MainThread - awscli.customizations.s3.executor - DEBUG - Waiting for thread to shutdown: <Worker(Thread-7, stopped daemon 4572643328)>
2014-10-08 15:49:58,223 - MainThread - awscli.customizations.s3.executor - DEBUG - Thread has been shutdown: <Worker(Thread-7, stopped daemon 4572643328)>
2014-10-08 15:49:58,224 - MainThread - awscli.customizations.s3.executor - DEBUG - Waiting for thread to shutdown: <Worker(Thread-8, stopped daemon 4576849920)>
2014-10-08 15:49:58,224 - MainThread - awscli.customizations.s3.executor - DEBUG - Thread has been shutdown: <Worker(Thread-8, stopped daemon 4576849920)>
2014-10-08 15:49:58,224 - MainThread - awscli.customizations.s3.executor - DEBUG - Waiting for thread to shutdown: <Worker(Thread-9, stopped daemon 4581056512)>
2014-10-08 15:49:58,239 - MainThread - awscli.customizations.s3.executor - DEBUG - Thread has been shutdown: <Worker(Thread-9, stopped daemon 4581056512)>
2014-10-08 15:49:58,239 - MainThread - awscli.customizations.s3.executor - DEBUG - Waiting for thread to shutdown: <Worker(Thread-10, stopped daemon 4585263104)>
2014-10-08 15:49:58,239 - MainThread - awscli.customizations.s3.executor - DEBUG - Thread has been shutdown: <Worker(Thread-10, stopped daemon 4585263104)>
2014-10-08 15:49:58,239 - MainThread - awscli.customizations.s3.executor - DEBUG - Waiting for thread to shutdown: <Worker(Thread-11, stopped daemon 4589469696)>
2014-10-08 15:49:58,239 - MainThread - awscli.customizations.s3.executor - DEBUG - Thread has been shutdown: <Worker(Thread-11, stopped daemon 4589469696)>
2014-10-08 15:49:58,239 - MainThread - awscli.customizations.s3.executor - DEBUG - Waiting for thread to shutdown: <Worker(Thread-12, stopped daemon 4593676288)>
2014-10-08 15:49:58,242 - MainThread - awscli.customizations.s3.executor - DEBUG - Thread has been shutdown: <Worker(Thread-12, stopped daemon 4593676288)>
2014-10-08 15:49:58,242 - MainThread - awscli.customizations.s3.executor - DEBUG - Waiting for thread to shutdown: <Worker(Thread-13, stopped daemon 4597882880)>
2014-10-08 15:49:58,242 - MainThread - awscli.customizations.s3.executor - DEBUG - Thread has been shutdown: <Worker(Thread-13, stopped daemon 4597882880)>
2014-10-08 15:49:58,242 - MainThread - awscli.customizations.s3.executor - DEBUG - Waiting for thread to shutdown: <Worker(Thread-14, stopped daemon 4602089472)>
2014-10-08 15:49:58,242 - MainThread - awscli.customizations.s3.executor - DEBUG - Thread has been shutdown: <Worker(Thread-14, stopped daemon 4602089472)>
2014-10-08 15:49:58,243 - MainThread - awscli.customizations.s3.executor - DEBUG - Queueing end sentinel for result thread.

2014-10-08 15:49:58,243 - MainThread - awscli.customizations.s3.executor - DEBUG - Queueing end sentinel for IO thread.
2014-10-08 15:49:58,243 - MainThread - awscli.customizations.s3.executor - DEBUG - Waiting for result thread to shutdown.
2014-10-08 15:49:58,243 - Thread-1 - awscli.customizations.s3.executor - DEBUG - Shutdown request received in print thread, shutting down print thread.
2014-10-08 15:49:58,243 - Thread-2 - awscli.customizations.s3.executor - DEBUG - Shutdown request received in IO thread, shutting down.
2014-10-08 15:49:58,243 - MainThread - awscli.customizations.s3.executor - DEBUG - Waiting for IO thread to shutdown.
2014-10-08 15:49:58,243 - MainThread - awscli.customizations.s3.executor - DEBUG - All threads have been shutdown.

[nikolay]

Your new version 1.5.1 broke this. The file still gets deleted, but the command fails. It worked yesterday, just not today anymore.

[nikolay]

There was a similar issue before, I reported it, you fixed it, and now it came back to life somehow.

[jamesls]

I can confirm the issue, working on a fix.

[nikolay]

Thanks!