"aws s3api get-bucket-policy" fails when policy exists

[ronan-ft]

If I run the following command on a certain bucket, I get an error message:

aws s3api get-bucket-policy --bucket CENSORED

("CENSORED" is not the real bucket name, of course.)

The error message is:

not well-formed (invalid token): line 1, column 0

If I add the --debug option to the above command, this is the stack
trace I get:

Traceback (most recent call last):
  File "/home/ronan/.local/lib/aws/local/lib/python2.7/site-packages/awscli/clidriver.py", line 188, in main
    return command_table[parsed_args.command](remaining, parsed_args)
  File "/home/ronan/.local/lib/aws/local/lib/python2.7/site-packages/awscli/clidriver.py", line 332, in __call__
    return command_table[parsed_args.operation](remaining, parsed_globals)
  File "/home/ronan/.local/lib/aws/local/lib/python2.7/site-packages/awscli/clidriver.py", line 441, in __call__
    self._operation_object, call_parameters, parsed_globals)
  File "/home/ronan/.local/lib/aws/local/lib/python2.7/site-packages/awscli/clidriver.py", line 532, in invoke
    **parameters)
  File "/home/ronan/.local/lib/aws/local/lib/python2.7/site-packages/botocore/operation.py", line 68, in call
    response = endpoint.make_request(self, params)
  File "/home/ronan/.local/lib/aws/local/lib/python2.7/site-packages/botocore/endpoint.py", line 76, in make_request
    return self._send_request(prepared_request, operation)
  File "/home/ronan/.local/lib/aws/local/lib/python2.7/site-packages/botocore/endpoint.py", line 97, in _send_request
    response, exception = self._get_response(request, operation, attempts)
  File "/home/ronan/.local/lib/aws/local/lib/python2.7/site-packages/botocore/endpoint.py", line 120, in _get_response
    http_response), None)
  File "/home/ronan/.local/lib/aws/local/lib/python2.7/site-packages/botocore/response.py", line 492, in get_response
    xml_response.parse(body, encoding)
  File "/home/ronan/.local/lib/aws/local/lib/python2.7/site-packages/botocore/response.py", line 71, in parse
    parser.feed(s)
ParseError: not well-formed (invalid token): line 1, column 0

Clearly, the program is trying to parse JSON as XML. The bug seems to
be essentially the same as the following, in the AWS JavaScript library:
aws/aws-sdk-js#49.

I know the bucket has a policy. For one thing, before encountering
this problem, I successfully performed aws s3api put-bucket-policy.
Also, I can see the policy in the HTTP response body when I use the
--debug option.

The following is the output of aws --version:

aws-cli/1.3.0 Python/2.7.3 Linux/3.2.0-4-amd64

[jamesls]

Good catch. This is in fact the same bug as the aws-sdk-js as you've mentioned. Should be a straightforward fix.

[quiver]

Should be a straightforward fix.

This may not be elegant but checking for its response content-type is one solution:

$ git diff response.py
diff --git a/botocore/response.py b/botocore/response.py
index b489b2f..084d2cf 100644
--- a/botocore/response.py
+++ b/botocore/response.py
@@ -478,7 +478,8 @@ def get_response(session, operation, http_response):
         "Response Headers:\n%s",
         '\n'.join("%s: %s" % (k, v) for k, v in http_response.headers.items()))
     logger.debug("Response Body:\n%s", body)
-    if operation.service.type in ('json', 'rest-json'):
+    if operation.service.type in ('json', 'rest-json') or \
+        http_response.headers['content-type'] == 'application/json':
         json_response = JSONResponse(session, operation)
         if body:
             json_response.parse(body, encoding)