changing describe_parameters call to use paginator

[eisjcormier]

SUMMARY

updated create_update_parameter() to use a paginator for the describe_parameter method when updating an existing parameter. I ran into an issue where the parameter I needed to update was not in the first result set of items returned by: client.get_parameter(Name=args['Name'], WithDecryption=True)

See ADDITIONAL INFORMATION for details

ISSUE TYPE

• Bugfix Pull Request

COMPONENT NAME

aws_ssm_parameter_store

ANSIBLE VERSION

ansible 2.6.4
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /bin/ansible
  python version = 2.7.5 (default, Aug  4 2017, 00:39:18) [GCC 4.8.5 20150623 (Red Hat 4.8.5-16)]

ADDITIONAL INFORMATION

I ran into an issue where the parameter I needed to update was not in the first result set of items returned by: client.get_parameter(Name=args['Name'], WithDecryption=True)
I got this error when running my playbook:

Result:
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: root
<127.0.0.1> EXEC /bin/sh -c 'echo ~root && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp/ansible-tmp-1536863695.84-147938161401239 `" && echo ansible-tmp-1536863695.84-147938161401239="` echo /root/.ansible/tmp/ansible-tmp-1536863695.84-147938161401239 `" ) && sleep 0'
Using module file /usr/lib/python2.7/site-packages/ansible/modules/cloud/amazon/aws_ssm_parameter_store.py
<127.0.0.1> PUT /root/.ansible/tmp/ansible-local-30457dmHRd8/tmp_9kYoS TO /root/.ansible/tmp/ansible-tmp-1536863695.84-147938161401239/aws_ssm_parameter_store.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1536863695.84-147938161401239/ /root/.ansible/tmp/ansible-tmp-1536863695.84-147938161401239/aws_ssm_parameter_store.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python2 /root/.ansible/tmp/ansible-tmp-1536863695.84-147938161401239/aws_ssm_parameter_store.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /root/.ansible/tmp/ansible-tmp-1536863695.84-147938161401239/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_0pStDP/ansible_module_aws_ssm_parameter_store.py", line 253, in <module>
    main()
  File "/tmp/ansible_0pStDP/ansible_module_aws_ssm_parameter_store.py", line 248, in main
    (changed, response) = invocations[state](client, module)
  File "/tmp/ansible_0pStDP/ansible_module_aws_ssm_parameter_store.py", line 192, in create_update_parameter
    if describe_existing_parameter['Parameters'][0]['Description'] != args['Description']:
IndexError: list index out of range

fatal: [localhost]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_0pStDP/ansible_module_aws_ssm_parameter_store.py\", line 253, in <module>\n    main()\n  File \"/tmp/ansible_0pStDP/ansible_module_aws_ssm_parameter_store.py\", line 248, in main\n    (changed, response) = invocations[state](client, module)\n  File \"/tmp/ansible_0pStDP/ansible_module_aws_ssm_parameter_store.py\", line 192, in create_update_parameter\n    if describe_existing_parameter['Parameters'][0]['Description'] != args['Description']:\nIndexError: list index out of range\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE",
    "rc": 1
}

The call to client.get_parameter(Name=args['Name'], WithDecryption=True) was actually returning this:

 {u'NextToken': u'AAEAAYo5rAFED3zqXPRdM2barhRoPEy7XKiKyQseIg/NfpRoAAAAAFuai7PyhJdo0bspyIm/QCuI/KQWErCGt+O7i4bwVHyJXsX+Z0jYMdddl9TpkJwG+vxArVfXLlu+0rfdy14DbLbK8audCg4cJiThDTRdFhQLCHWptaOTIiblxoJnJfd5KT8yfXsFig/dVTcx9T8oYOFXKF6SdlJ4hlvDmzA/cdGUxTuLGqCXAj0OBmDXsH3jVRm57qEVqkWeTbXzWpZNJvQ6rsAK+UQQKJG4UlgNu3j/0iIlWjgRWr4fd3LaNjTmadFPXUW6sWw8o+tLR0j324YmVCg0s/rSo+2iXnSdh0GgPOAZfmVaBUvH2VkmYe+7yESYlUsYc7Bgzhd+BhE0Cj5h5atle5ipSiJa2/mdtwBJigg7lQiQg2tOfsCvzf9gyvpdqrNvdND37xktB6dOxu+tpUeaJDZ+joqGqdEWLTKE1jfA1ajLCjKXiP/gkUMNrZnXXx/++nmL3ohaWXdV0OfXDgzR2YxQSLTFCPhTm6TcPlK5ibhFYeaC8IX+TtsRTxJD8LMDB5lhbxwXNJO0gLm7LkSFsrTKbaiJhELgChQZBx20zX7t8XCm8pn/Zq23L7wigvnl4Ce0dIKMCqiNk7Vqls2U+90QnHK+X08ulPOS2tF4nzJ0MlTaLyYypWDlr0/Ktn/rp+kCHl1vrYOiwRJuqt8t2zrK+J5faOkleVV0Fx3g5Z4XPrpvYmt8ZlNPGE2zXQjk5insxzLFt+C7vj14xcvql4O2iAK5duFoTQE/L4/ZCw==', 'ResponseMetadata': {'RetryAttempts': 0, 'HTTPStatusCode': 200, 'RequestId': 'XXXXXXX', 'HTTPHeaders': {'x-amzn-requestid': 'XXXXXXX', 'date': 'Thu, 13 Sep 2018 16:09:22 GMT', 'content-length': '840', 'content-type': 'application/x-amz-json-1.1'}}, u'Parameters': []})

[ansibot]

cc @mikedlr @nathanwebsterdotme
click here for bot help

[ansibot]

Hi @eisjcormier,

Thank you for the pullrequest, just so you are aware we have a dedicated Working Group for aws.
You can find other people interested in this in #ansible-aws on Freenode IRC
For more information about communities, meetings and agendas see https://github.com/ansible/community

click here for bot help

[s-hertel]

It's very strange that the parameters are an empty list for the first page!

[eisjcormier]

Hi Sloane,
According to the boto3 docs: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ssm.html#SSM.Client.describe_parameters

If the service reaches an internal limit while processing the results, it stops the operation and returns the matching values up to that point and a NextToken

So it appears that limit is 50 parameters ( similar to how the UI displays parameters).

[s-hertel]

Ah, I see. Thanks @eisjcormier, curiosity satiated. :-)

[eisjcormier]

Thanks Sloane!

[s-hertel]

Thanks for fixing it! I'm going to open a backport PR so this can be included in 2.7.x as well.