New module = elb_application_lb #19491

wimnat · 2016-12-18T17:07:52Z

ISSUE TYPE

New Module Pull Request

COMPONENT NAME

elb_application_lb

ANSIBLE VERSION

2.3

SUMMARY

New module for AWS application ELBs. I have broken with naming convention compared to the classic elb module ec2_elb_lb because ELB is actually a separate part of boto3 and not part of ec2. I suggest we rename ec2_elb_lb to elb_classic_lb.

I have also not attempted to merge classic and application load balancers in to one module because they are again separate in boto3 API and operate different enough to warrant two modules.

The module is currently unfinished. I know there are bugs and modification of listeners doesn't work properly at the moment either but i have created the PR to get some feedback from the community as I imagine this module is probably quite sought after.

wimnat · 2016-12-18T17:07:52Z

Migrated from ansible/ansible-modules-extras#2946

gundalow

needs_revision

gundalow · 2016-12-19T15:47:26Z

lib/ansible/modules/cloud/amazon/elb_application_lb.py

+    choices: [ 'yes', 'no' ]
+  idle_timeout:
+    description:
+      - The number of seconds to wait before an idle connection is closed


Missing full stop. descriptions: must be sentences.

wimnat · 2016-12-19T17:52:55Z

For those asking about a name change here are my reasons:

The older "classic" elb module name of ec2_elb_lb i think is not particularly clear and I think it should be renamed to elb_classic_lb. I have already put this forward in the proposals repo.
Although the ELBs turn up in the EC2 section of the AWS console, they are not controlled by the ec2 API, they are controlled by a separate ELB API. Ansible dictates that we should follow boto3 naming convention where possible so i think starting the module name with elb_ rather than ec2_ would be proper
Someone suggested a name of ec2_alb - aside from the reason above, i think this name is ambiguous and also for the same reasons mentioned here - http://softwareengineering.stackexchange.com/questions/33349/scientific-evidence-that-supports-using-long-variable-names-instead-of-abbreviat

ryansb

We can definitely add an elb_classic_lb alias that will point at ec2_elb_lb, the renaming process takes multiple cycles because we must have one release (at least) for each phase (numbers assume starting now):

Allow the new name (release the alias) in 2.3 and add a warning on the old name
Release 2.4 and continue warning on the old name
Release 2.5 and rename ec2_elb_lb to _ec2_elb_lb
Release 2.6 without _ec2_elb_lb

ryansb · 2016-12-30T16:58:19Z

lib/ansible/modules/cloud/amazon/elb_application_lb.py

+
+DOCUMENTATION = '''
+---
+module: elb_lb_application


Fix this name to match file name

ryansb · 2016-12-30T17:00:00Z

lib/ansible/modules/cloud/amazon/elb_application_lb.py

+      - A list of the names or IDs of the security groups to assign to the load balancer. Required if state=present.
+    required: false
+    default: []
+  scheme:


It may be more obvious to name this parameter something like public as a boolean, but I'd be worried about AWS adding some third scheme meaning we have to obsolete the boolean and use their new value. No change required here, but would like your thoughts.

I agree. Chances of AWS creating a third schema are not low.

Also, with this scheme, at least we're following AWS API convention.

ryansb · 2016-12-30T17:01:04Z

lib/ansible/modules/cloud/amazon/elb_application_lb.py

+        ssl_policy: # The security policy that defines which ciphers and protocols are supported. The default is the current predefined security policy.
+        certificates: # The ARN of the certificate (only one certficate ARN should be provided)
+        default_actions:
+          - type: forward # Required. Only 'forward' is accepted at this time


Can this default to forward since that's the only accepted value?

Can't default this as it's not actually an Ansilble parameter. It's a child of listeners.

ryansb · 2016-12-30T17:01:48Z

lib/ansible/modules/cloud/amazon/elb_application_lb.py

+        Rules:
+          - Conditions: 
+            - Field: path-pattern
+                Values:


I think this needs to move out one indentation level to match Field

ryansb · 2016-12-30T17:01:59Z

lib/ansible/modules/cloud/amazon/elb_application_lb.py

+                  - '/test'
+            Priority: '1'
+            Actions:
+              - 'TargetGroupName': 'test-target-group'


Remove these single-quotes.

ryansb · 2016-12-30T17:02:11Z

lib/ansible/modules/cloud/amazon/elb_application_lb.py

+            - Field: path-pattern
+                Values:
+                  - '/test'
+            Priority: '1'


Can this work as an int as well, or must it be quoted?

ryansb · 2016-12-30T17:03:17Z

lib/ansible/modules/cloud/amazon/elb_application_lb.py

+    try:
+        response = connection.describe_target_groups(Names=[tg_name])
+    except botocore.exceptions.ClientError as e:
+        module.fail_json(msg=e.message, **camel_dict_to_snake_dict(e.response))


Please add the kwarg exception=traceback.format_exc() to this failure.

ryansb · 2016-12-30T17:04:07Z

lib/ansible/modules/cloud/amazon/elb_application_lb.py

+
+    try:
+        return connection.describe_load_balancers(Names=[module.params.get("name")])['LoadBalancers'][0]
+    except (ClientError, NoCredentialsError) as e:


Can you handle NoCreds separately to raise a less generic message?

The handling of the exception will be identical as it just uses the message passed from AWS

ryansb · 2016-12-30T17:04:30Z

lib/ansible/modules/cloud/amazon/elb_application_lb.py

+        try:
+            connection.modify_load_balancer_attributes(LoadBalancerArn=elb['LoadBalancerArn'], Attributes=params['Attributes'])
+        except (ClientError, NoCredentialsError) as e:
+                module.fail_json(msg=e.message, **camel_dict_to_snake_dict(e.response))


Add exception=traceback.format_exc() here as well please.

ryansb · 2016-12-30T17:06:00Z

lib/ansible/modules/cloud/amazon/elb_application_lb.py

+    try:
+        params['SecurityGroups'] = get_ec2_security_group_ids_from_names(module.params.get('security_groups'), connection_ec2, boto3=True)
+    except ValueError as e:
+        module.fail_json(msg=str(e))


Add traceback here as well (for line numbers).

s-hertel · 2017-05-15T17:58:38Z

lib/ansible/modules/cloud/amazon/elb_application_lb.py

+        certificates: # The ARN of the certificate (only one certficate ARN should be provided)
+        default_actions:
+          - type: forward # Required. Only 'forward' is accepted at this time
+            target_group_arn: # Required. The ARN of the target group


This protests if I only specify TargetGroupArn and only uses TargetGroupName (to then get the ARN).

s-hertel · 2017-05-15T18:07:25Z

lib/ansible/modules/cloud/amazon/elb_application_lb.py

+      - subnet-012345678
+      - subnet-abcdef000
+    listeners:
+      - protocol: http # Required. The protocol for connections from clients to the load balancer (http or https).


Since these are being passed directly to boto3 we should fix the docs to reflect what is expected. Only accepts Protocol and HTTP as opposed to protocol and http right now. CamelCase all the listeners options.

s-hertel · 2017-05-15T18:12:26Z

lib/ansible/modules/cloud/amazon/elb_application_lb.py

+      - subnet-012345678
+      - subnet-abcdef000
+    listeners:
+      - protocol: http # Required. The protocol for connections from clients to the load balancer (http or https).


The listeners options all need to be CamelCase for the example to work.

s-hertel · 2017-05-15T18:26:11Z

lib/ansible/modules/cloud/amazon/elb_application_lb.py

+
+    # Now, if required, set ELB listeners. Use try statement here so we can remove the ELB if this stage fails
+    try:
+        listener_changed = create_or_update_elb_listeners(connection, module, elb)


Not a blocker, but do we want a purge_listeners option? To do any modification to the ELB all the listeners must be specified each time.

Would prefer to do this later

s-hertel · 2017-05-15T18:28:19Z

lib/ansible/modules/cloud/amazon/elb_application_lb.py

+        for listener in listeners:
+            for key in listener.keys():
+                if key not in ['Protocol', 'Port', 'SslPolicy', 'Certificates', 'DefaultActions', 'Rules']:
+                    module.fail_json(msg="listeners parameter contains invalid dict keys")


Could add the expected keys to the error since CamelCase may be unexpected.

s-hertel · 2017-05-15T18:35:09Z

lib/ansible/modules/cloud/amazon/elb_application_lb.py

+    returned: when status is present
+    type: string
+    sample: internal-my-elb-123456789.ap-southeast-2.elb.amazonaws.com
+idle_timeout_timeout_seconds:


Is this return value unrelated to the idle_timeout option?

s-hertel · 2017-05-15T18:36:57Z

lib/ansible/modules/cloud/amazon/elb_application_lb.py

+try:
+    import boto3
+    from botocore.exceptions import ClientError, NoCredentialsError
+    HAS_BOTO3 = True


Can import HAS_BOTO3 instead from ansible.module_utils.ec2

ansibot · 2017-05-16T12:52:07Z

The test ansible-test sanity --test pep8 failed with the following errors:

lib/ansible/modules/cloud/amazon/elb_application_lb.py:412:1: W293 blank line contains whitespace
lib/ansible/modules/cloud/amazon/elb_application_lb.py:752:1: W293 blank line contains whitespace
lib/ansible/modules/cloud/amazon/elb_application_lb.py:827:17: E265 block comment should start with '# '

click here for bot help

jbscare · 2017-05-16T18:05:17Z

lib/ansible/modules/cloud/amazon/elb_application_lb.py

+      - Protocol: http # Required. The protocol for connections from clients to the load balancer (http or https).
+        Port: 80 # Required. The port on which the load balancer is listening.
+        SslPolicy: # The security policy that defines which ciphers and protocols are supported. The default is the current predefined security policy.
+        Certificates: # The ARN of the certificate (only one certficate ARN should be provided)


The later example of the Certificates parameter is much more clear about what's expected here (a list of dicts, and not a string of the ARN, which is what the comment here made it sound like) -- it might be nice if this example matched.

jbscare · 2017-05-16T18:05:30Z

lib/ansible/modules/cloud/amazon/elb_application_lb.py

+      - Protocol: http # Required. The protocol for connections from clients to the load balancer (http or https).
+        Port: 80 # Required. The port on which the load balancer is listening.
+        SslPolicy: # The security policy that defines which ciphers and protocols are supported. The default is the current predefined security policy.
+        Certificates: # The ARN of the certificate (only one certficate ARN should be provided)


The later example of the Certificates parameter is much more clear about what's expected here (a list of dicts, and not a string of the ARN, which is what the comment here made it sound like) -- it might be nice if this example matched.

Addressed

ansibot · 2017-05-18T11:31:55Z

The test ansible-test sanity --test pep8 failed with the following errors:

lib/ansible/modules/cloud/amazon/elb_application_lb.py:117:161: E501 line too long (177 > 160 characters)
lib/ansible/modules/cloud/amazon/elb_application_lb.py:140:161: E501 line too long (177 > 160 characters)

click here for bot help

ansibot · 2017-05-18T11:43:19Z

The test ansible-test sanity --test pep8 failed with the following errors:

lib/ansible/modules/cloud/amazon/elb_application_lb.py:37:157: W291 trailing whitespace
lib/ansible/modules/cloud/amazon/elb_application_lb.py:62:157: W291 trailing whitespace
lib/ansible/modules/cloud/amazon/elb_application_lb.py:67:157: W291 trailing whitespace
lib/ansible/modules/cloud/amazon/elb_application_lb.py:74:158: W291 trailing whitespace

click here for bot help

s-hertel · 2017-05-20T18:10:48Z

~~Looks great after pep8 is fixed.~~ Edit: I guess there's a problem maybe with the bot allowing spaces in the docs. Hm. @gundalow Do you have thoughts on that since this is a docs issue?

shipit

s-hertel · 2017-05-30T17:13:14Z

@wimnat Looking forward to getting this merged. LMK if you want me to make pep8 fixes.

wimnat · 2017-05-31T10:49:56Z

@s-hertel i think we're there. Once this and target_group is merged i'm going to have such a party :D

s-hertel · 2017-05-31T11:56:15Z

shipit

willthames · 2017-05-31T13:20:17Z

shipit

ryansb · 2017-05-31T13:31:27Z

In a followup PR it would be nice to wrap these calls in AWSRetry.backoff, since there are lots of calls hitting the same API in this module, so it'd be easy to hit limits at scale.

warrd · 2017-06-02T11:35:17Z

This PR is not python3 compatible so added a follow up #25300

ansibot added affects_2.3 This issue/PR affects Ansible v2.3 aws cloud module This issue/PR relates to a module. needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. new_plugin This PR includes a new plugin. plugin labels Dec 18, 2016

wimnat force-pushed the ansible-modules-extras/pull/2946 branch from 099b04a to 996323b Compare December 18, 2016 19:06

ansibot added needs_triage Needs a first human triage before being processed. and removed needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. triage labels Dec 18, 2016

gundalow mentioned this pull request Dec 19, 2016

New module = elb_application_lb ansible/ansible-modules-extras#2946

Closed

gundalow requested changes Dec 19, 2016

View reviewed changes

gundalow removed the needs_triage Needs a first human triage before being processed. label Dec 19, 2016

ansibot added needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. and removed needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. labels Dec 19, 2016

ryansb previously requested changes Dec 30, 2016

View reviewed changes

ansibot added needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. and removed needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. labels Jan 2, 2017

s-hertel requested changes May 15, 2017

View reviewed changes

Add listeners to return output and fixed up examples

d80342c

ansibot added the ci_verified Changes made in this PR are causing tests to fail. label May 16, 2017

jbscare reviewed May 16, 2017

View reviewed changes

review changes

a66b52d

fix line lengths

18c21f5

s-hertel approved these changes May 20, 2017

View reviewed changes

ansibot added the stale_ci This PR has been tested by CI more than one week ago. Close and re-open this PR to get it retested. label May 30, 2017

pep8 fixes

0ba17a4

s-hertel approved these changes May 31, 2017

View reviewed changes

gundalow approved these changes May 31, 2017

View reviewed changes

willthames approved these changes May 31, 2017

View reviewed changes

ryansb approved these changes May 31, 2017

View reviewed changes

ryansb merged commit 76e56bf into ansible:devel May 31, 2017

wimnat deleted the ansible-modules-extras/pull/2946 branch June 15, 2017 09:10

ansible locked and limited conversation to collaborators Apr 26, 2019

New module = elb_application_lb #19491

New module = elb_application_lb #19491

Conversation

wimnat commented Dec 18, 2016

ISSUE TYPE

COMPONENT NAME

ANSIBLE VERSION

SUMMARY

wimnat commented Dec 18, 2016

gundalow left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wimnat commented Dec 19, 2016

ryansb left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ansibot commented May 16, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ansibot commented May 18, 2017

ansibot commented May 18, 2017

s-hertel commented May 20, 2017 • edited

s-hertel commented May 30, 2017

wimnat commented May 31, 2017

s-hertel commented May 31, 2017

willthames commented May 31, 2017

ryansb commented May 31, 2017

warrd commented Jun 2, 2017

s-hertel commented May 20, 2017 •

edited