Skip to content
This repository

Update HmacAuthV4Handler to properly populate POST parameters. #1081

Merged
merged 1 commit into from over 1 year ago

2 participants

g2harris James Saryerwinnie
g2harris

I ran into this problem when digging into why AWS was returning HTTP 505 Version Not Supported responses for autoscaling's CreateLaunchConfiguration action. In past cases it looks like this error has occurred when the query string of GET requests exceed some threshold (AWS likely reads a fixed size and takes the last 5 bytes on the first line as the HTTP version). Although the AWSQueryConnection.get_object call was set to use POST the actual request consisted of a POST with all of the parameters in the query string.

In researching this I figured out the logic to actually map request parameters into the query string (GET requests) or into the request body (POST requests) actually occurs in the add_auth call in _mexe shortly before the request is issued to Amazon. For HmacAuthV4Handler there was no special POST request handling so parameters always came in as query strings with the method 'POST'.

This patch does the following:

1. Moves the query string / request body manipulation on HmacAuthV4Handler to before the canonical_request is calculated so that the request body signature is correctly generated.

2. Updates the canonical_uri to look at req.auth_path instead of req.path.  Since the query string manipulation is occurring before the request is signed now we need to use the cached version of this field that is set aside for authentication already.

3. Modify canonical_query_string to return '' when a POST request is used.  This is because the parameters in a POST request will now be part of the body when calculating the canonical_request to sign.

This appears to do the right thing in the cases I've tested. It does blow away the contents of request.body when their is a query string present so if anything ever attempts to make a POST call with both params and data set bad things could happen. This is just copying the behaviour already present in QuerySignatureHelper when a POST request is processed.

An aside - auth.py seems like the wrong place to be formatting the http_request based on whether or not a POST request is going out. Based on the code in QuerySignatureHelper it looks like this may have been done for convenience in earlier development but I think it would make more sense to move this logic into connection.py when the actual HttpRequest object is built.

g2harris g2harris Fix HmacAuthV4Handler to treat POST parameters properly.
I ran into this problem when digging into why AWS was returning HTTP 505 Version Not Supported
responses for autoscaling's CreateLaunchConfiguration action.  In past cases it looks like this
error has occurred when the query string of GET requests exceed some threshold (AWS likely reads a
fixed size and takes the last 5 bytes on the first line as the HTTP version).  Although the
AWSQueryConnection.get_object call was set to use POST the actual request consisted of a POST with
all of the parameters in the query string.

In researching this I figured out the logic to actually map request parameters into the query string
(GET requests) or into the request body (POST requests) actually occurs in the add_auth call in
_mexe shortly before the request is issued to Amazon.  For HmacAuthV4Handler there was no special
POST request handling so parameters always came in as query strings with the method 'POST'.

This patch does the following:

    (1) Moves the query string / request body manipulation on HmacAuthV4Handler to before the
canonical_request is calculated so that the request body signature is correctly generated.

    (2) Updates the canonical_uri to look at req.auth_path instead of req.path.  Since the query
string manipulation is occuring before the request is signed now we need to use the cached version
of this field that is set aside for authentication already.

    (3) Modify canonical_query_string to return '' when a POST request is used.  This is because the
parameters in a POST request will now be part of the body when calculating the canonical_request to
sign.

This appears to do the right thing in the cases I've tested.  It does blow away the contents of
request.body when their is a query string present so if anything ever attempts to make a POST call
with both params and data set bad things could happen.  This is just copying the behaviour already
present in QuerySignatureHelper when a POST request is processed.
4fb722f
James Saryerwinnie jamesls merged commit 4fb722f into from
James Saryerwinnie
Owner

Thanks for the PR, great description as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 1 unique commit by 1 author.

Oct 24, 2012
g2harris g2harris Fix HmacAuthV4Handler to treat POST parameters properly.
I ran into this problem when digging into why AWS was returning HTTP 505 Version Not Supported
responses for autoscaling's CreateLaunchConfiguration action.  In past cases it looks like this
error has occurred when the query string of GET requests exceed some threshold (AWS likely reads a
fixed size and takes the last 5 bytes on the first line as the HTTP version).  Although the
AWSQueryConnection.get_object call was set to use POST the actual request consisted of a POST with
all of the parameters in the query string.

In researching this I figured out the logic to actually map request parameters into the query string
(GET requests) or into the request body (POST requests) actually occurs in the add_auth call in
_mexe shortly before the request is issued to Amazon.  For HmacAuthV4Handler there was no special
POST request handling so parameters always came in as query strings with the method 'POST'.

This patch does the following:

    (1) Moves the query string / request body manipulation on HmacAuthV4Handler to before the
canonical_request is calculated so that the request body signature is correctly generated.

    (2) Updates the canonical_uri to look at req.auth_path instead of req.path.  Since the query
string manipulation is occuring before the request is signed now we need to use the cached version
of this field that is set aside for authentication already.

    (3) Modify canonical_query_string to return '' when a POST request is used.  This is because the
parameters in a POST request will now be part of the body when calculating the canonical_request to
sign.

This appears to do the right thing in the cases I've tested.  It does blow away the contents of
request.body when their is a query string present so if anything ever attempts to make a POST call
with both params and data set bad things could happen.  This is just copying the behaviour already
present in QuerySignatureHelper when a POST request is processed.
4fb722f
This page is out of date. Refresh to see the latest.

Showing 1 changed file with 18 additions and 6 deletions. Show diff stats Hide diff stats

  1. +18 6 boto/auth.py
24 boto/auth.py
@@ -264,7 +264,7 @@ def string_to_sign(self, http_request):
264 264 headers_to_sign = self.headers_to_sign(http_request)
265 265 canonical_headers = self.canonical_headers(headers_to_sign)
266 266 string_to_sign = '\n'.join([http_request.method,
267   - http_request.path,
  267 + http_request.auth_path,
268 268 '',
269 269 canonical_headers,
270 270 '',
@@ -337,6 +337,10 @@ def query_string(self, http_request):
337 337 return '&'.join(pairs)
338 338
339 339 def canonical_query_string(self, http_request):
  340 + # POST requests pass parameters in through the
  341 + # http_request.body field.
  342 + if http_request.method == 'POST':
  343 + return ""
340 344 l = []
341 345 for param in http_request.params:
342 346 value = str(http_request.params[param])
@@ -363,7 +367,7 @@ def signed_headers(self, headers_to_sign):
363 367 return ';'.join(l)
364 368
365 369 def canonical_uri(self, http_request):
366   - return http_request.path
  370 + return http_request.auth_path
367 371
368 372 def payload(self, http_request):
369 373 body = http_request.body
@@ -443,6 +447,18 @@ def add_auth(self, req, **kwargs):
443 447 req.headers['X-Amz-Date'] = now.strftime('%Y%m%dT%H%M%SZ')
444 448 if self._provider.security_token:
445 449 req.headers['X-Amz-Security-Token'] = self._provider.security_token
  450 + qs = self.query_string(req)
  451 + if qs and req.method == 'POST':
  452 + # Stash request parameters into post body
  453 + # before we generate the signature.
  454 + req.body = qs
  455 + req.headers['Content-Type'] = 'application/x-www-form-urlencoded; charset=UTF-8'
  456 + req.headers['Content-Length'] = str(len(req.body))
  457 + else:
  458 + # Safe to modify req.path here since
  459 + # the signature will use req.auth_path.
  460 + req.path = req.path.split('?')[0]
  461 + req.path = req.path + '?' + qs
446 462 canonical_request = self.canonical_request(req)
447 463 boto.log.debug('CanonicalRequest:\n%s' % canonical_request)
448 464 string_to_sign = self.string_to_sign(req, canonical_request)
@@ -454,10 +470,6 @@ def add_auth(self, req, **kwargs):
454 470 l.append('SignedHeaders=%s' % self.signed_headers(headers_to_sign))
455 471 l.append('Signature=%s' % signature)
456 472 req.headers['Authorization'] = ','.join(l)
457   - qs = self.query_string(req)
458   - if qs:
459   - req.path = req.path.split('?')[0]
460   - req.path = req.path + '?' + qs
461 473
462 474
463 475 class QuerySignatureHelper(HmacKeys):

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.