Provide full and filtered RDS Aurora details in ec2 inventory

[aioue]

ISSUE TYPE

• Feature Pull Request

Re-submission of #15609 against latest devel

ANSIBLE VERSION

ansible 2.0.2.0
  config file =
  configured module search path = Default w/o overrides

SUMMARY

Add db_clusters to the ec2 inventory. Show tags. Only show clusters matching tags in the .ini.

Use boto3 to get the clusters.

  "db_clusters": {
    "test-eu-west-1": {
      "AllocatedStorage": 1,
      "AvailabilityZones": [
        "eu-west-1a",
        "eu-west-1b",
        "eu-west-1c"
      ],
      "BackupRetentionPeriod": 3,
      "DBClusterIdentifier": "test-eu-west-1",
      "DBClusterMembers": [
        {
          "DBClusterParameterGroupStatus": "in-sync",
          "DBInstanceIdentifier": "test-eu-west-1-0",
          "IsClusterWriter": true,
          "PromotionTier": 1
        },
        {
          "DBClusterParameterGroupStatus": "in-sync",
          "DBInstanceIdentifier": "test-eu-west-1-1",
          "IsClusterWriter": false,
          "PromotionTier": 1
        }
      ],
      "DBClusterParameterGroup": "aurora-cluster-test",
      "DBSubnetGroup": "dev_test_db_group",
      "Endpoint": "test-eu-west-1.cluster-sdfsdfsd.eu-west-1.rds.amazonaws.com",
      "Engine": "aurora",
      "EngineVersion": "5.6.10a",
      "KmsKeyId": "arn:aws:kms:eu-west-1:345345345345:key/sfsdfsdfsdfsdf",
      "MasterUsername": "sdfsdfsdfs",
      "Port": 33306,
      "PreferredBackupWindow": "05:21-05:51",
      "PreferredMaintenanceWindow": "sat:03:18-sat:03:48",
      "Status": "available",
      "StorageEncrypted": true,
      "Tags": [
        {
          "Key": "env",
          "Value": "test"
        }
      ],
      "VpcSecurityGroups": [
        {
          "Status": "active",
          "VpcSecurityGroupId": "sg-234234234"
        }
      ]
    }
  }

[aioue]

@ryansb unrelated test failing?

[aioue]

@ryansb @nitzmahone all tests passing on resubmit of #15609 against fresh devel. Please consider for inclusion before tower updates the base files again 👍

[nitzmahone]

LGTM, but I have limited experience with RDS, so cc @defionscode @ryansb for a look from people with hopefully more RDS experience...

[juliedavila]

This needs to do a check for boto3, and set a var like HAS_BOTO3 to false when there is an ImportError. When HAS_BOTO3 is false, then the logic that uses it must also be skipped, which means that the ini comment around that setting should also make note of that fact.

[aioue]

@defionscode do you have a 'best practice' example of this to link to?

[ryansb]

@aioue Yep, you'd do something like:

HAS_BOTO3 = False
try:
    import boto3
    HAS_BOTO3 = True
except ImportError:
    pass


# later in the code

if not HAS_BOTO3:
    module.fail_json(message="This module requires boto3 be installed - please install boto3 and try again")

[aioue]

@ryansb updated as per your comment.

[ryansb]

Great - works for me, but the way you connect to AWS doesn't work with the boto-specified profile parameter in the .ini file. To make profiles work, change the client to client = self.connect_to_aws(rds, region)

Once that's fixed, it'll be set to roll

[ryansb]

Please change this to client = self.connect_to_aws(rds, region)

[aioue]

@ryansb done.

[aioue]

@ryansb need some shipits if you're happy 👍

[ryansb]

Hi, I haven't forgotten about this PR, I noticed that there isn't a connection creator for boto3 available in the inventory ec2.py. I'm working on a change that'll add one, and once that's in this will be mergeable.

Right now, you don't need to do anything and this will still be on schedule for 2.2.

[ryansb]

@aioue, @defionscode, and @nitzmahone please see my rebase of this PR, with a module_utils method to use boto3 properly PR#16606.

[ryansb]

The version w/ boto3 checks has been merged (see #16606 )