Skip to content

Update HmacAuthV4Handler to properly populate POST parameters. #1081

Merged
merged 1 commit into from Nov 6, 2012

2 participants

@g2harris

I ran into this problem when digging into why AWS was returning HTTP 505 Version Not Supported responses for autoscaling's CreateLaunchConfiguration action. In past cases it looks like this error has occurred when the query string of GET requests exceed some threshold (AWS likely reads a fixed size and takes the last 5 bytes on the first line as the HTTP version). Although the AWSQueryConnection.get_object call was set to use POST the actual request consisted of a POST with all of the parameters in the query string.

In researching this I figured out the logic to actually map request parameters into the query string (GET requests) or into the request body (POST requests) actually occurs in the add_auth call in _mexe shortly before the request is issued to Amazon. For HmacAuthV4Handler there was no special POST request handling so parameters always came in as query strings with the method 'POST'.

This patch does the following:

1. Moves the query string / request body manipulation on HmacAuthV4Handler to before the canonical_request is calculated so that the request body signature is correctly generated.

2. Updates the canonical_uri to look at req.auth_path instead of req.path.  Since the query string manipulation is occurring before the request is signed now we need to use the cached version of this field that is set aside for authentication already.

3. Modify canonical_query_string to return '' when a POST request is used.  This is because the parameters in a POST request will now be part of the body when calculating the canonical_request to sign.

This appears to do the right thing in the cases I've tested. It does blow away the contents of request.body when their is a query string present so if anything ever attempts to make a POST call with both params and data set bad things could happen. This is just copying the behaviour already present in QuerySignatureHelper when a POST request is processed.

An aside - auth.py seems like the wrong place to be formatting the http_request based on whether or not a POST request is going out. Based on the code in QuerySignatureHelper it looks like this may have been done for convenience in earlier development but I think it would make more sense to move this logic into connection.py when the actual HttpRequest object is built.

@g2harris g2harris Fix HmacAuthV4Handler to treat POST parameters properly.
I ran into this problem when digging into why AWS was returning HTTP 505 Version Not Supported
responses for autoscaling's CreateLaunchConfiguration action.  In past cases it looks like this
error has occurred when the query string of GET requests exceed some threshold (AWS likely reads a
fixed size and takes the last 5 bytes on the first line as the HTTP version).  Although the
AWSQueryConnection.get_object call was set to use POST the actual request consisted of a POST with
all of the parameters in the query string.

In researching this I figured out the logic to actually map request parameters into the query string
(GET requests) or into the request body (POST requests) actually occurs in the add_auth call in
_mexe shortly before the request is issued to Amazon.  For HmacAuthV4Handler there was no special
POST request handling so parameters always came in as query strings with the method 'POST'.

This patch does the following:

    (1) Moves the query string / request body manipulation on HmacAuthV4Handler to before the
canonical_request is calculated so that the request body signature is correctly generated.

    (2) Updates the canonical_uri to look at req.auth_path instead of req.path.  Since the query
string manipulation is occuring before the request is signed now we need to use the cached version
of this field that is set aside for authentication already.

    (3) Modify canonical_query_string to return '' when a POST request is used.  This is because the
parameters in a POST request will now be part of the body when calculating the canonical_request to
sign.

This appears to do the right thing in the cases I've tested.  It does blow away the contents of
request.body when their is a query string present so if anything ever attempts to make a POST call
with both params and data set bad things could happen.  This is just copying the behaviour already
present in QuerySignatureHelper when a POST request is processed.
4fb722f
@jamesls jamesls merged commit 4fb722f into boto:develop Nov 6, 2012
@jamesls
the boto project member
jamesls commented Nov 6, 2012

Thanks for the PR, great description as well.

@romanb romanb added a commit to romanb/aws that referenced this pull request Dec 14, 2014
@romanb romanb Do not send a query string when PostQuery is used.
At present, any API operations that use the PostQuery request method send both, a query string and an application/x-www-form-urlencoded request body. This is not just redundant but can also lead to "505 HTTP Version Not Supported" errors when exceeding the maximum URL length allowed by AWS (the request line will be truncated and thus leads to misinterpretation of the HTTP version).

Related prior issue in boto: boto/boto#1081
bbefdcf
@romanb romanb referenced this pull request in aristidb/aws Dec 14, 2014
Merged

Do not send a query string when PostQuery is used. #145

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.