module for parsing cloudformation files

[sjmiller609]

SUMMARY

New module 'cloudformation_parse' - sometimes a user may want to parse a cloudformation file before deploying it. You can't parse yaml cloudformation with normal yaml parsing because the built-in functions cannot be parsed without special yaml handling (e.g. !Ref). This will also work for json cloudformation.

ISSUE TYPE

• New Module Pull Request

COMPONENT NAME

cloudformation_parse

ANSIBLE VERSION

ansible 2.6.2
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python2.7/dist-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 2.7.15rc1 (default, Apr 15 2018, 21:51:34) [GCC 7.3.0]

ADDITIONAL INFORMATION

cfn-flip is used for parsing the cloudformation files. This python module is supported by amazon https://github.com/awslabs/aws-cfn-template-flip

[ansibot]

The test ansible-test sanity --test ansible-doc --python 2.6 [explain] failed with 1 error:

lib/ansible/modules/cloud/amazon/cloudformation_parse.py:0:0: has a documentation error formatting or is missing documentation.

The test ansible-test sanity --test ansible-doc --python 2.7 [explain] failed with 1 error:

lib/ansible/modules/cloud/amazon/cloudformation_parse.py:0:0: has a documentation error formatting or is missing documentation.

The test ansible-test sanity --test ansible-doc --python 3.5 [explain] failed with 1 error:

lib/ansible/modules/cloud/amazon/cloudformation_parse.py:0:0: has a documentation error formatting or is missing documentation.

The test ansible-test sanity --test ansible-doc --python 3.6 [explain] failed with 1 error:

lib/ansible/modules/cloud/amazon/cloudformation_parse.py:0:0: has a documentation error formatting or is missing documentation.

The test ansible-test sanity --test ansible-doc --python 3.7 [explain] failed with 1 error:

lib/ansible/modules/cloud/amazon/cloudformation_parse.py:0:0: has a documentation error formatting or is missing documentation.

The test ansible-test sanity --test docs-build [explain] failed with the error:

Command "/usr/bin/python test/sanity/code-smell/docs-build.py" returned exit status 1.
>>> Standard Error
Traceback (most recent call last):
  File "test/sanity/code-smell/docs-build.py", line 100, in <module>
    main()
  File "test/sanity/code-smell/docs-build.py", line 17, in main
    raise subprocess.CalledProcessError(sphinx.returncode, cmd, output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['make', 'singlehtmldocs']' returned non-zero exit status 2.

The test ansible-test sanity --test import --python 2.6 [explain] failed with 1 error:

lib/ansible/modules/cloud/amazon/cloudformation_parse.py:74:0: ImportError: No module named cfn_flip

The test ansible-test sanity --test import --python 2.7 [explain] failed with 1 error:

lib/ansible/modules/cloud/amazon/cloudformation_parse.py:74:0: ImportError: No module named cfn_flip

The test ansible-test sanity --test import --python 3.5 [explain] failed with 1 error:

lib/ansible/modules/cloud/amazon/cloudformation_parse.py:74:0: ImportError: No module named 'cfn_flip'

The test ansible-test sanity --test import --python 3.6 [explain] failed with 1 error:

lib/ansible/modules/cloud/amazon/cloudformation_parse.py:74:0: ModuleNotFoundError: No module named 'cfn_flip'

The test ansible-test sanity --test import --python 3.7 [explain] failed with 1 error:

lib/ansible/modules/cloud/amazon/cloudformation_parse.py:74:0: ModuleNotFoundError: No module named 'cfn_flip'

The test ansible-test sanity --test pep8 [explain] failed with 8 errors:

lib/ansible/modules/cloud/amazon/cloudformation_parse.py:76:1: E302 expected 2 blank lines, found 1
lib/ansible/modules/cloud/amazon/cloudformation_parse.py:100:23: E231 missing whitespace after ','
lib/ansible/modules/cloud/amazon/cloudformation_parse.py:109:11: E114 indentation is not a multiple of four (comment)
lib/ansible/modules/cloud/amazon/cloudformation_parse.py:110:11: E111 indentation is not a multiple of four
lib/ansible/modules/cloud/amazon/cloudformation_parse.py:111:15: E111 indentation is not a multiple of four
lib/ansible/modules/cloud/amazon/cloudformation_parse.py:113:93: E226 missing whitespace around arithmetic operator
lib/ansible/modules/cloud/amazon/cloudformation_parse.py:124:1: E302 expected 2 blank lines, found 1
lib/ansible/modules/cloud/amazon/cloudformation_parse.py:127:1: E305 expected 2 blank lines after class or function definition, found 1

The test ansible-test sanity --test validate-modules [explain] failed with 4 errors:

lib/ansible/modules/cloud/amazon/cloudformation_parse.py:0:0: E105 GPLv3 license header not found in the first 20 lines of the module
lib/ansible/modules/cloud/amazon/cloudformation_parse.py:0:0: E319 RETURN.value.returned: required key not provided @ data['returned']. Got None
lib/ansible/modules/cloud/amazon/cloudformation_parse.py:0:0: E321 Exception attempting to import module for argument_spec introspection, 'No module named 'cfn_flip''
lib/ansible/modules/cloud/amazon/cloudformation_parse.py:26:59: E302 DOCUMENTATION is not valid YAML

The test ansible-test sanity --test yamllint [explain] failed with 1 error:

lib/ansible/modules/cloud/amazon/cloudformation_parse.py:26:59: error DOCUMENTATION: syntax error: expected <block end>, but found ','

click here for bot help

[ansibot]

Hi @sjmiller609,

Thank you for the pullrequest, just so you are aware we have a dedicated Working Group for aws.
You can find other people interested in this in #ansible-aws on Freenode IRC
For more information about communities, meetings and agendas see https://github.com/ansible/community

click here for bot help

[ryansb]

I think this might be better suited as a lookup plugin. That way users could do:

set_fact: '{{ lookup("cloudformation_parse", "myfile.yml") }}' 

Then that will return the full dict, and they can pipe it to json_query to grab specific pieces if they need it, or add other values.

[sjmiller609]

@ryansb I agree with your point

[ryansb]

Great, you can look at some of the existing lookups, such as env https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/lookup/env.py to see what to inherit from/how to take args.

[sjmiller609]

@ryansb Thanks Ryan for the example

[sjmiller609]

@ryansb Let me know what you think. Thanks

[sjmiller609]

I'm going to close and re-open to get the ball rolling