Skip to content

Fix pagination with string params #689

Merged
merged 4 commits into from Mar 5, 2014

2 participants

@danielgtaylor

Fixes boto/botocore#243 by making sure the max items pagination param uses the same type as the parameter it is replacing.

$ aws route53 list-resource-record-sets --hosted-zone-id /hostedzone/ABCD --max-items 1

A client error (NoSuchHostedZone) occurred when calling the ListResourceRecordSets operation: No hosted zone found with ID: ABCD

@jamesls please review.

@jamesls jamesls and 1 other commented on an outdated diff Mar 4, 2014
awscli/customizations/paginate.py
@@ -66,8 +66,19 @@ def unify_paging_params(argument_table, operation, event_name, **kwargs):
STARTING_TOKEN_HELP,
operation,
parse_type='string')
+ # Try to get the pagination parameter type
+ type_ = 'integer'
+ if 'limit_key' in operation.pagination:
+ for param in operation.params:
+ if param.name == operation.pagination['limit_key']:
@jamesls
Amazon Web Services member
jamesls added a note Mar 4, 2014

I don't think it's a safe assumption that there's always limit_key, based on the code below that does a if 'limit_key' in operation.pagination. It would be worth double checking that this is the case. Either way we should be consistent in this module.

@danielgtaylor
danielgtaylor added a note Mar 4, 2014

Is line 71 not the same as 107? I'm not sure what you mean here.

@jamesls
Amazon Web Services member
jamesls added a note Mar 4, 2014

You're right, sorry I misread the code. Disregard my previous comment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@jamesls jamesls commented on the diff Mar 4, 2014
tests/unit/customizations/test_paginate.py
self.bar_param = mock.Mock()
self.bar_param.cli_name = 'bar'
+ self.bar_param.type = 'string'
@jamesls
Amazon Web Services member
jamesls added a note Mar 4, 2014

Can we add a test for:

1) integer
2) Unknown type (the raising TypeError is not tested).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@jamesls jamesls commented on an outdated diff Mar 4, 2014
awscli/customizations/paginate.py
@@ -66,8 +66,19 @@ def unify_paging_params(argument_table, operation, event_name, **kwargs):
STARTING_TOKEN_HELP,
operation,
parse_type='string')
+ # Try to get the pagination parameter type
+ type_ = 'integer'
+ if 'limit_key' in operation.pagination:
+ for param in operation.params:
+ if param.name == operation.pagination['limit_key']:
+ type_ = param.type
+ break
+
+ if type_ not in PageArgument.type_map:
+ raise TypeError('Unsupported pagination type {0}'.format(type_))
@jamesls
Amazon Web Services member
jamesls added a note Mar 4, 2014

Let's include more context in the error message (the operation/parameter name would be helpful).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@jamesls
Amazon Web Services member
jamesls commented Mar 4, 2014

Please add a test for this specific case you mention in the PR descriptions:

$ aws route53 list-resource-record-sets --hosted-zone-id /hostedzone/ABCD --max-items 1
@danielgtaylor

I believe my latest changes fix all the comments above. Please take another look 😄

@jamesls jamesls commented on an outdated diff Mar 5, 2014
tests/unit/customizations/test_paginate.py
+
+class TestStringLimitKey(TestPaginateBase):
+
+ def setUp(self):
+ super(TestStringLimitKey, self).setUp()
+ self.bar_param.type = 'string'
+
+ def test_integer_limit_key(self):
+ argument_table = {
+ 'foo': mock.Mock(),
+ 'bar': mock.Mock(),
+ }
+ paginate.unify_paging_params(argument_table, self.operation,
+ 'building-argument-table.foo.bar')
+ # Max items should be the same type as bar, which may not be an int
+ self.assertEqual('string', argument_table['max-items']._parse_type)
@jamesls
Amazon Web Services member
jamesls added a note Mar 5, 2014

In general, we should not be testing through any internal APIs in a unittest unless there's really no other way to do so (just a general rule of thumb to keep in mind). You can use the .cli_type_name property to get this information.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@jamesls
Amazon Web Services member
jamesls commented Mar 5, 2014

Also, I'd say just make the s/._parse_type/.cli_type_name/ change and feel free to merge.

:shipit:

@danielgtaylor danielgtaylor merged commit bf5e9b3 into aws:develop Mar 5, 2014
@danielgtaylor danielgtaylor deleted the danielgtaylor:page-string branch Mar 5, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.