Add support for EC2 dynamic data in ec2_facts

[roadmapper]

ISSUE TYPE

• Feature Pull Request

COMPONENT NAME

module ec2_facts

ANSIBLE VERSION

ansible 2.2.1.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides

SUMMARY

This change allows the ec2_facts module to set facts with the data from the EC2 dynamic data (instance identity document, signatures, etc.). Due to the this, parsing the region out of the availability zone is no longer needed; the hardcoded list of regions and parsing are removed and the old placement region key is left for backwards compatibility.

Minor changes:

• The IAM info/credentials data is also flattened out so that there is a key for each value inside the JSON document
• The IAM role is parsed out separately (handles Add better iam role ec2 facts #21626)
• The security group IDs have commas instead of newlines

Before

"ansible_ec2_iam_info": "{\n  \"Code\" : \"Success\",\n  \"LastUpdated\" : \"2017-02-16T18:28:22Z\",\n  \"InstanceProfileArn\" : \"arn:aws:iam::<AWS account ID>:instance-profile/<role name>\",\n  \"InstanceProfileId\" : \"<instance profile ID>\"\n}",
"ansible_ec2_iam_security_credentials_<role name>": "{\n  \"Code\" : \"Success\",\n  \"LastUpdated\" : \"2017-02-16T18:28:07Z\",\n  \"Type\" : \"AWS-HMAC\",\n  \"AccessKeyId\" : \"<Access Key ID>\",\n  \"SecretAccessKey\" : \"<Secret Access Key>\",\n  \"Token\" : \"<token>\",\n  \"Expiration\" : \"2017-02-17T01:01:22Z\"\n}",
...
"ansible_ec2_network_interfaces_macs_<mac>_security_group_ids": "<sg1>\n<sg2>",

After

"ansible_ec2_fws_instance_monitoring": "enabled",
...
"ansible_ec2_iam_info_code": "Success",
"ansible_ec2_iam_info_instanceprofilearn": "arn:aws:iam::<AWS Account ID>:instance-profile/<role name>",
"ansible_ec2_iam_info_instanceprofileid": "<instance profile ID>",
"ansible_ec2_iam_info_lastupdated": "2017-02-16T18:28:22Z",
"ansible_ec2_iam_instance_profile_role": "<role name>",
"ansible_ec2_iam_security_credentials_<role name>_accesskeyid": "<Access Key ID>",
"ansible_ec2_iam_security_credentials_<role name>_code": "Success",
"ansible_ec2_iam_security_credentials_<role name>_expiration": "2017-02-17T01:01:22Z",
"ansible_ec2_iam_security_credentials_<role name>_lastupdated": "2017-02-16T18:28:07Z",
"ansible_ec2_iam_security_credentials_<role name>_secretaccesskey": "<Secret Access Key>",
"ansible_ec2_iam_security_credentials_<role name>_token": "<token>",
"ansible_ec2_iam_security_credentials_<role name>_type": "AWS-HMAC",
...
"ansible_ec2_instance_identity_document_accountid": "<AWS Account ID>",
"ansible_ec2_instance_identity_document_architecture": "x86_64",
"ansible_ec2_instance_identity_document_availabilityzone": "us-east-1a",
"ansible_ec2_instance_identity_document_billingproducts": null,
"ansible_ec2_instance_identity_document_devpayproductcodes": null,
"ansible_ec2_instance_identity_document_imageid": "<image ID>",
"ansible_ec2_instance_identity_document_instanceid": "<instance ID>",
"ansible_ec2_instance_identity_document_instancetype": "t2.medium",
"ansible_ec2_instance_identity_document_kernelid": null,
"ansible_ec2_instance_identity_document_pendingtime": "2017-02-07T20:24:56Z",
"ansible_ec2_instance_identity_document_privateip": "<private IP>",
"ansible_ec2_instance_identity_document_ramdiskid": null,
"ansible_ec2_instance_identity_document_region": "us-east-1",
"ansible_ec2_instance_identity_document_version": "2010-08-31",
"ansible_ec2_instance_identity_pkcs7": "<PKCS7 signature>",
"ansible_ec2_instance_identity_rsa2048": "<RSA signature>",
"ansible_ec2_instance_identity_signature": "<instance identity signature>",
...
"ansible_ec2_network_interfaces_macs_<mac>_security_group_ids": "<sg1>,<sg2>",

[ansibot]

cc @willthames @silviud
click here for bot help

[roadmapper]

Just checking in if there was any update on this. I don't think there will be any action from @silviud; he doesn't seem to actively checks PRs. Maybe this PR should be community reviewed?

[ansibot]

The test ansible-test sanity --test pep8 failed with the following error:

lib/ansible/modules/cloud/amazon/ec2_facts.py:133:5: E303 too many blank lines (2)

click here for bot help

[mattclay]

Renaming/deprecating a module should be done in a separate PR from actual code changes. There are two reasons for this:

1. Renaming/deprecating is something that can be quickly reviewed and merged (or rejected), since there are no substantial changes required to the module itself.

2. Combining a rename and code changes usually complicates reviewing diffs. Since git doesn't track renames, it must detect them after the fact. Significant code changes usually prevent this detection from occurring, causing diffs to show up as a full delete and add instead of a rename.

[mattclay]

Oh, and a third reason. The bot will ping namespace maintainers on a rename, despite the module not being new. It's better to do this on a PR which will quickly be merged (the rename), instead of pulling in a lot of maintainers to discussion on changes for the module which really only need to involve the module maintainer(s).

[roadmapper]

Unfortunately this PR has sat out for a long time; I'm happy to change it back to the old name and submit a PR if that will speed things along. I kept trying to find guidelines on the module rename process with changes, but I didn't find much guidance, except for @ryansb's proposal suggested by @s-hertel. I would still like to modify the Ansible doc of the old module so that return values are properly documented.

[s-hertel]

@roadmapper If you want help with putting this back the way it was prior to the renaming changes let me know. I agree with mattclay's points but I'm sorry about your inconvenience with the back and forth.

[roadmapper]

Reverted the module rename and tested, ready_for_review

[willthames]

Merged, thanks for your patience and bearing with us!

[willthames]

And apologies for my bad suggestion of doing the deprecation at the same time.