allow multiple resources to be specified on a policy

[kapilt]

implement and use expand_resources for execution ala expand_regions with --region all. literal in policy should be either string or array. ala

policies:
  - name: collect-all
    resource: [ec2, elb, lambda, rds]

[scotwk]

@kapilt - what's the implications for the output directory with this change?

[kapilt]

good question, one thought was switching resources.json to $resource-type.json and ditto for logs. effectively we'll have the expanded policies sharing the same output directory, but with the same policy name, but different files per resource type. report gets a little wierd across multiple resources, and would likely need a resource specifier to disambiguate if given a multi resource policy from config. another thought that might be easier and less backward compatibility issues would be suffixing/prefixing the policy name with resource type when expanding.

[bulususb]

@kapilt has this feature been added? We are pretty much trying to solve a similar issue:
Problem: Identify all ec2's with a public IP and have ssh port(22) open to 0.0.0.0/0.
Since we can identify a public IP only using an EC2 resource and find out open ports using security groups resource we are trying to see if multiple resources can be combined in the same policy. If not, alternatively, we have to 1. identify and tag all SG's with open permission and 2. identify ec2 that belong to a security group that has the custom tag

[cbarensfeld]

If this has been implemented please let me know. Right now I'm replicated the same tagging policy for each resource type. It would be great if I could use a list of resources and events. thanks

[kapilt]

The easiest way to avoid the duplication in absence of this is to, use yaml anchor and reference syntax to minimize the duplication.

https://confluence.atlassian.com/bitbucket/yaml-anchors-960154027.html
https://gist.github.com/bowsersenior/979804
https://medium.com/@kinghuang/docker-compose-anchors-aliases-extensions-a1e4105d70bd

[mykmuh]

I tried using the anchor and reference syntax but I'm having an issue with override values.

vars:
  common: &common
      - type: tag
        key: "CostCenter"
        value: "1111111"
      - type: tag
        key: "BusinessUnit"
        value: "Bubba"

policies:
  - name: tag-ec2s
    resource: aws.ec2
    actions:
      <<: *common

When I validate, I get back the following error:

:ERROR Error on policy:tag-ec2s resource:aws.ec2
{'type': 'tag', 'key': 'CostCenter', 'value': '1111111'} is not of type 'array'

Failed validating 'type' in schema[18]['allOf'][0]['properties']['actions']:
    {'type': 'array'}

On instance['actions']:
    {'key': 'CostCenter', 'type': 'tag', 'value': '1111111'}
2019-07-09 16:09:27,032: custodian.commands:ERROR tag-ec2s

It doesn't seem to matter how I space the alias. It always errors out. If I run the exact same yaml with a simple * instead of <<: *, everything validates.

[kapilt]

@mykmug thats not correct anchor / ref syntax, if you yaml load that file in python and print it you'll see that its putting a dictionary for actions not an array

{'policies': [{'actions': {'key': 'CostCenter',
                           'type': 'tag',
                           'value': '1111111'},
               'name': 'tag-ec2s',
               'resource': 'aws.ec2'}],
 'vars': {'common': [{'key': 'CostCenter', 'type': 'tag', 'value': '1111111'},                                                                        
                     {'key': 'BusinessUnit', 'type': 'tag', 'value': 'Bubba'}]}}    

[mykmuh]

@kapilt Thanks the suggestion to check the output via Python. I loaded up my repl and played around a bit and ended up with the following. It allows me to override as needed while centralizing the fiddly bits in one location.

vars:
  common:
      - &st [ type: tag, key: "SupportTeam", value: "it@somewhere.com" ]
      - &pr [ type: tag, key: "Product", value: "OurProduct"]
      - &ap [ type: tag, key: "Application", value: "WidgetApi" ]
      - &en [ type: tag, key: "Environment", value: "prod" ]
policies:
  - name: tag-ec2s-prod
    resource: aws.ec2
    filters:
      - type: value
        key: VpcId
        value: "vpc-1111111"
    actions:
      - <<: *st
        value: "bubba@hotep.com"
      - <<: *pr
      - <<: *ap
      - <<: *en
  - name: tag-ec2s-nonprod
    resource: aws.ec2
    filters:
      - type: value
        key: VpcId
        value: "vpc-222222"
    actions:
      - <<: *st
      - <<: *pr
      - <<: *ap
      - <<: *en
        value: "NonProd"

[IngussNeilands]

Any planned progress on resource: [ec2, elb, lambda, rds] ?

[naut-brian]

Are there still plans to implement this feature?

[ntmggr]

I also would like to know if there are any plans implementing this

[nickwenner]

This would be a great feature to have!

[syed-synix]

Do we have any update on this, is this expected to be available?