New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support to find AWS Elastic Map Reduce #35865

Open
wants to merge 1 commit into
base: devel
from

Conversation

Projects
None yet
4 participants
@wmaramos

wmaramos commented Feb 7, 2018

SUMMARY
  • return a list of Cluster EMR based on name and state
  • results can be sorted
  • It depends on boto3
ISSUE TYPE
  • New Module Pull Request
COMPONENT NAME

emr_cluster_find

ANSIBLE VERSION
ansible 2.5.0 (wr-emr-cluster-find-module 6f06664fb9) last updated 2018/02/07 19:24:58 (GMT -200)
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/wellington.ramos/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /home/wellington.ramos/git/ansible/lib/ansible
  executable location = /home/wellington.ramos/git/ansible/bin/ansible
  python version = 2.7.12 (default, Nov 20 2017, 18:23:56) [GCC 5.4.0 20160609]
@ansibot

This comment has been minimized.

Contributor

ansibot commented Feb 7, 2018

@ansibot

This comment has been minimized.

Contributor

ansibot commented Feb 7, 2018

The test ansible-test sanity --test pylint [?] failed with the following error:

lib/ansible/modules/cloud/amazon/emr_cluster_find.py:151:50: undefined-variable Undefined variable 'traceback'

The test ansible-test sanity --test pep8 [?] failed with the following error:

lib/ansible/modules/cloud/amazon/emr_cluster_find.py:97:1: E302 expected 2 blank lines, found 1

The test ansible-test sanity --test validate-modules [?] failed with the following errors:

lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E105 GPLv3 license header not found in the first 20 lines of the module
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E322 "access_key" is listed in the argument_spec, but not documented in the module
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E322 "access_token" is listed in the argument_spec, but not documented in the module
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E322 "aws_access_key" is listed in the argument_spec, but not documented in the module
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E322 "aws_region" is listed in the argument_spec, but not documented in the module
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E322 "aws_secret_key" is listed in the argument_spec, but not documented in the module
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E322 "ec2_access_key" is listed in the argument_spec, but not documented in the module
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E322 "ec2_region" is listed in the argument_spec, but not documented in the module
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E322 "ec2_secret_key" is listed in the argument_spec, but not documented in the module
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E322 "ec2_url" is listed in the argument_spec, but not documented in the module
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E322 "profile" is listed in the argument_spec, but not documented in the module
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E322 "secret_key" is listed in the argument_spec, but not documented in the module
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E322 "security_token" is listed in the argument_spec, but not documented in the module
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E322 "validate_certs" is listed in the argument_spec, but not documented in the module
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:69:5: E313 RETURN is not valid YAML

The test ansible-test sanity --test yamllint [?] failed with the following error:

lib/ansible/modules/cloud/amazon/emr_cluster_find.py:69:5: error RETURN: syntax error: could not find expected ':'

click here for bot help

@wmaramos wmaramos force-pushed the wmaramos:wr-emr-cluster-find-module branch Feb 8, 2018

@ansibot

This comment has been minimized.

Contributor

ansibot commented Feb 8, 2018

The test ansible-test sanity --test pep8 [?] failed with the following error:

lib/ansible/modules/cloud/amazon/emr_cluster_find.py:67:161: E501 line too long (174 > 160 characters)

The test ansible-test sanity --test validate-modules [?] failed with the following errors:

lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E105 GPLv3 license header not found in the first 20 lines of the module
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E319 RETURN.results.contains: expected dict for dictionary value @ data['contains']. Got None
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E319 RETURN.results.id: extra keys not allowed @ data['id']. Got {'description': 'The Cluster ID from EMR', 'type': 'str', 'sample': 'j-12CK7ZJ37B5YA'}
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E319 RETURN.results.name: extra keys not allowed @ data['name']. Got {'description': 'name of EMR Cluster', 'type': 'str', 'sample': 'emr-test-name'}
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E319 RETURN.results.status: extra keys not allowed @ data['status']. Got {'description': 'details about the current status of the cluster', 'type': 'dict'}
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E322 "access_key" is listed in the argument_spec, but not documented in the module
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E322 "access_token" is listed in the argument_spec, but not documented in the module
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E322 "aws_access_key" is listed in the argument_spec, but not documented in the module
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E322 "aws_region" is listed in the argument_spec, but not documented in the module
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E322 "aws_secret_key" is listed in the argument_spec, but not documented in the module
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E322 "ec2_access_key" is listed in the argument_spec, but not documented in the module
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E322 "ec2_region" is listed in the argument_spec, but not documented in the module
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E322 "ec2_secret_key" is listed in the argument_spec, but not documented in the module
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E322 "ec2_url" is listed in the argument_spec, but not documented in the module
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E322 "profile" is listed in the argument_spec, but not documented in the module
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E322 "secret_key" is listed in the argument_spec, but not documented in the module
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E322 "security_token" is listed in the argument_spec, but not documented in the module
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E322 "validate_certs" is listed in the argument_spec, but not documented in the module

click here for bot help

@mkrizek mkrizek removed the needs_triage label Feb 9, 2018

@wmaramos wmaramos force-pushed the wmaramos:wr-emr-cluster-find-module branch Feb 9, 2018

@ansibot ansibot removed the ci_verified label Feb 9, 2018

@ansibot

This comment has been minimized.

Contributor

ansibot commented Feb 9, 2018

The test ansible-test sanity --test validate-modules [explain] failed with 1 error:

lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E307 version_added should be 2.6. Currently 2.5

click here for bot help

@ansibot ansibot added the ci_verified label Feb 9, 2018

@wmaramos wmaramos changed the title from Add support to find AWS EMR Clusters to Add support to find AWS Elastic Map Reduce Feb 9, 2018

@wmaramos wmaramos force-pushed the wmaramos:wr-emr-cluster-find-module branch Feb 9, 2018

@ansibot ansibot removed the ci_verified label Feb 9, 2018

@ansibot

This comment has been minimized.

Contributor

ansibot commented Feb 9, 2018

The test ansible-test sanity --test validate-modules [explain] failed with 1 error:

lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E307 version_added should be 2.6. Currently 2.5

click here for bot help

@ansibot ansibot added the ci_verified label Feb 9, 2018

@wmaramos wmaramos force-pushed the wmaramos:wr-emr-cluster-find-module branch to f1f3501 Feb 15, 2018

@ansibot ansibot removed the ci_verified label Feb 15, 2018

@ansibot

This comment has been minimized.

Contributor

ansibot commented Feb 15, 2018

@Constantin007 @Constantin07 @Deepakkothandan @Etherdaemon @Java1Guy @Lujeni @Madhura-CSI @MichaelBaydoun @Sodki @adq @akazakov @alachaum @amir343 @anryko @bekelchik @bpennypacker @brandond @carsongee @defunctio @dkhenry @fiunchinho @fivethreeo @garethr @gunzy83 @gurumaia @hsingh @hyperized @iiibrad @infectsoldier @j-carl @jarv @Java1Guy @jimbydamonk @jmenga @joelthompson @jonhadfield @jonmer85 @joshsouza @jsdalton @jsmartin @kaczynskid @leedm777 @linuxdynasty @loia @lwade @MichaelBaydoun @michaeljs1990 @minichate @mjschultz @mmochan @nadirollo @nand0p @naslanidis @nickball @orthanc @piontas @pjodouin @prasadkatti @psykotox @pwnall @raags @rickmendes @roadmapper @ryansydnor @scicoin-project @scottanderson42 @shepdelacreme @silviud @steynovich @tastychutney @tedder @tgerla @timmahoney @tombamford @tsiganenok @viper233 @whiter @willricardo @wilvk @wimnat @zacblazic @zbal @zeekin @zimbatm

As a maintainer of a module in the same namespace this new module has been submitted to, your vote counts for shipits. Please review this module and add shipit if you would like to see it merged.

click here for bot help

)
)
module = AnsibleModule(

This comment has been minimized.

@s-hertel

s-hertel Apr 6, 2018

Contributor

This can be updated to use AnsibleAWSModule from ansible.module_utils.aws.core. Then you will be able to remove the check for HAS_BOTO3, the exception handling around the client, and the use of get_aws_connection_info and boto3_conn (as well as any of the imports for those things) and do client = module.client('emr') instead.

limit = module.params.get('limit')
sort_order = module.params.get('sort_order')
resource = client.list_clusters(ClusterStates=[state])

This comment has been minimized.

@s-hertel

s-hertel Apr 6, 2018

Contributor

This should have exception handling around it. Once you update to AnsibleAWSModule you can do:

except (BotoCoreError, ClientError) as e:
    module.fail_json_aws(e, msg="Unable to list clusters")

and it will use e.response and add the traceback for you.

'''
try:
from botocore.exception import ClientError

This comment has been minimized.

@s-hertel

s-hertel Apr 6, 2018

Contributor

BotoCoreError should also be handled.

region:
description:
- region where this module will find EMR
state:

This comment has been minimized.

@s-hertel

s-hertel Apr 6, 2018

Contributor

Since state is used in Ansible modules as the state ansible should leave a resource in, renaming this to something like status or emr_status may be less confusing.

# For more information see http://boto3.readthedocs.io/en/latest/reference/services/emr.html#EMR.Client.list_clusters
---
results:
description: dictionary containg all cluster settings see link above for more information

This comment has been minimized.

@s-hertel

s-hertel Apr 6, 2018

Contributor

a list of dictionaries

argument_spec = ec2_argument_spec()
argument_spec.update(
dict(
name_regex=dict(required=False),

This comment has been minimized.

@s-hertel

s-hertel Apr 6, 2018

Contributor

required=False is the default, so you can omit that, same for any of the following. name_regex=dict().

@ansibot ansibot added the affects_2.6 label May 24, 2018

@wmaramos wmaramos closed this Oct 4, 2018

@wmaramos wmaramos reopened this Oct 5, 2018

@ansibot

This comment has been minimized.

Contributor

ansibot commented Oct 5, 2018

The test ansible-test sanity --test pep8 [explain] failed with 1 error:

lib/ansible/modules/cloud/amazon/emr_cluster_find.py:154:1: E305 expected 2 blank lines after class or function definition, found 1

The test ansible-test sanity --test validate-modules [explain] failed with 3 errors:

lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E307 version_added should be 2.8. Currently 2.6
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E324 Value for "default" from the argument_spec ('ascending') for "sort_order" does not match the documentation (None)
lib/ansible/modules/cloud/amazon/emr_cluster_find.py:0:0: E326 Value for "choices" from the argument_spec (['ascending', 'descending']) for "sort_order" does not match the documentation ([])

click here for bot help

@ansibot ansibot added ci_verified and removed stale_ci labels Oct 5, 2018

@ansibot ansibot added the stale_ci label Oct 14, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment