New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1.9 breaks ec2 module's compatibility with Eucalyptus #11023

Closed
ahmari6 opened this Issue May 18, 2015 · 19 comments

Comments

Projects
None yet
7 participants
@ahmari6

ahmari6 commented May 18, 2015

EC2 modules compatibility with Eucalyptus has been compromised at some point. Following example shows it's forcing the use of AWS regions:

[root@eucaclc eucalyptus-console]# ansible-playbook -vvv --private-key=adminui.private eucalyptus-userconsole-ec2_41.yml
[WARNING]: The version of gmp you have installed has a known issue regarding timing vulnerabilities when used with pycrypto. If possible, you should update it (i.e. yum update gmp).

PLAY [Stage console instance(s)] **********************************************

TASK: [Launch instance] *******************************************************
<127.0.0.1> REMOTE_MODULE ec2 region=eucalyptus keypair=adminui group=adminui instance_type=c1.medium image=emi-26B340F4 instance_tags='{"type":"adminui"}' wait=true count=1 <127.0.0.1> EXEC ['/bin/sh', '-c', 'mkdir -p $HOME/.ansible/tmp/ansible-tmp-1431935003.29-136739190654197 && echo $HOME/.ansible/tmp/ansible-tmp-1431935003.29-136739190654197']
<127.0.0.1> PUT /tmp/tmplTxF6U TO /root/.ansible/tmp/ansible-tmp-1431935003.29-136739190654197/ec2
<127.0.0.1> EXEC ['/bin/sh', '-c', u'LANG=C LC_CTYPE=C /usr/bin/python /root/.ansible/tmp/ansible-tmp-1431935003.29-136739190654197/ec2; rm -rf /root/.ansible/tmp/ansible-tmp-1431935003.29-136739190654197/ >/dev/null 2>&1']
failed: [localhost -> 127.0.0.1] => {"failed": true}
msg: value of region must be one of: ap-northeast-1,ap-southeast-1,ap-southeast-2,cn-north-1,eu-central-1,eu-west-1,eu-central-1,sa-east-1,us-east-1,us-west-1,us-west-2,us-gov-west-1, got: eucalyptus

FATAL: all hosts have already failed -- aborting

PLAY RECAP ********************************************************************
           to retry, use: --limit @/root/eucalyptus-userconsole-ec2_41.retry

localhost                  : ok=0    changed=0    unreachable=0    failed=1

[root@eucaclc eucalyptus-console]#
[root@eucaclc eucalyptus-console]# rpm -qa | grep ansible 
ansible-1.9.1-1.el6.noarch 
[root@eucaclc eucalyptus-console]#

Result without region in playbook

[root@eucaclc eucalyptus-console]# ansible-playbook -vvv --private-key=adminui.private eucalyptus-userconsole-ec2_41.yml
[WARNING]: The version of gmp you have installed has a known issue regarding timing vulnerabilities when used with pycrypto. If possible, you should update it (i.e. yum update gmp).


PLAY [Stage console instance(s)] **********************************************

TASK: [Launch instance] *******************************************************
<127.0.0.1> REMOTE_MODULE ec2 keypair=adminui group=adminui instance_type=c1.medium image=emi-26B340F4 instance_tags='{"type":"adminui"}' wait=true count=1 <127.0.0.1> EXEC ['/bin/sh', '-c', 'mkdir -p $HOME/.ansible/tmp/ansible-tmp-1431935125.16-16189386472767 && echo $HOME/.ansible/tmp/ansible-tmp-1431935125.16-16189386472767']
<127.0.0.1> PUT /tmp/tmp8uAWuX TO /root/.ansible/tmp/ansible-tmp-1431935125.16-16189386472767/ec2
<127.0.0.1> EXEC ['/bin/sh', '-c', u'LANG=C LC_CTYPE=C /usr/bin/python /root/.ansible/tmp/ansible-tmp-1431935125.16-16189386472767/ec2; rm -rf /root/.ansible/tmp/ansible-tmp-1431935125.16-16189386472767/ >/dev/null 2>&1']
failed: [localhost -> 127.0.0.1] => {"failed": true}
msg: region must be specified

FATAL: all hosts have already failed -- aborting
@ahmari6

This comment has been minimized.

ahmari6 commented May 18, 2015

Same functionality was tested against 1.8.4 and it works.

@lwade

This comment has been minimized.

Contributor

lwade commented May 18, 2015

+1

Looks like this commit may have broken it? At least its the only one I can find related to connection parameters in recent times:

ansible/ansible-modules-core@4bcbcc8#diff-9667dfcde0b7854855c94acb534b156a

cc @bennojoy

@bennojoy

This comment has been minimized.

Contributor

bennojoy commented May 18, 2015

@ahmari6 @lwade are you getting this error in 1.9.x release or the latest devel ?

@lwade

This comment has been minimized.

Contributor

lwade commented May 18, 2015

Hey @bennojoy - 1.9.x from EPEL current, I'm assuming the MFA stuff made it into the 1.9 release but I've not looked into any form of debugging

@bennojoy

This comment has been minimized.

Contributor

bennojoy commented May 18, 2015

@lwade seems like it is not the MFA but the 'vpc' commit that makes the 'region' parameter mandatory
which inturn causes the function "connect_to_aws" instead of "boto.connect_ec2_endpoint", will check how we can resolve this.
can you please check if the same fails if you specify region in 1.8.x

@teejalon

This comment has been minimized.

teejalon commented May 19, 2015

If we specify region ec2 module wants an AWS region, eucalyptus is not an valid option

[root@eucaclc eucalyptus-console]# ansible --version
[WARNING]: The version of gmp you have installed has a known issue regarding
timing vulnerabilities when used with pycrypto. If possible, you should update
it (i.e. yum update gmp).
ansible 1.8.4
configured module search path = None
[root@eucaclc eucalyptus-console]#
[root@eucaclc eucalyptus-console]# source /root/.euca/eucarc
[root@eucaclc eucalyptus-console]# euca-describe-regions
REGION eucalyptus http://10.0.0.10:8773/services/Eucalyptus
[root@eucaclc eucalyptus-console]#
[root@eucaclc eucalyptus-console]# grep region eucalyptus-userconsole-ec2_41_ans_issue11023_test.yml
local_action: ec2 region=eucalyptus keypair={{keypair}} group={{security_group}} instance_type={{instance_type}} image={{image}} instance_tags='{"type":"adminui"}' wait=true count=1
[root@eucaclc eucalyptus-console]#
[root@eucaclc eucalyptus-console]# ansible-playbook -vvv --private-key=adminui.private eucalyptus-userconsole-ec2_41_ans_issue11023_test.yml
[WARNING]: The version of gmp you have installed has a known issue regarding
timing vulnerabilities when used with pycrypto. If possible, you should update
it (i.e. yum update gmp).
PLAY [Stage console instance(s)] **********************************************
TASK: [Launch instance] *******************************************************
<127.0.0.1> count=1 group=adminui region=eucalyptus instance_type=c1.medium keypair=adminui instance_tags={"type":"adminui"} image=emi-26B340F4 wait=true
<127.0.0.1>
<127.0.0.1>
<127.0.0.1> u'LANG=C LC_CTYPE=C /usr/bin/python /root/.ansible/tmp/ansible-tmp-1432031936.16-26971824630114/ec2; rm -rf /root/.ansible/tmp/ansible-tmp-1432031936.16-26971824630114/ >/dev/null 2>&1']
failed: [localhost -> 127.0.0.1] => {"failed": true}
msg: value of region must be one of: ap-northeast-1,ap-southeast-1,ap-southeast-2,cn-north-1,eu-central-1,eu-west-1,sa-east-1,us-east-1,us-west-1,us-west-2,us-gov-west-1, got: eucalyptus
FATAL: all hosts have already failed -- aborting
PLAY RECAP ********************************************************************
to retry, use: --limit @/root/eucalyptus-userconsole-ec2_41_ans_issue11023_test.retry
localhost : ok=0 changed=0 unreachable=0 failed=1
[root@eucaclc eucalyptus-console]#

@bennojoy

This comment has been minimized.

Contributor

bennojoy commented May 20, 2015

i think we should make the 'region' parameter mandatory only if vpc_subnet_id id specified and that should restore the compatibitly with eucalyptus.

@ahmari6

This comment has been minimized.

ahmari6 commented May 20, 2015

As Eucalyptus has the concept of VPC in 4.1 release as a tech preview feature already, we would appreciate a certain support from the Ansible EC2 module that would ensure the compatibility with Euca:

  • regions are not limited to Amazon, rather checked against endpoint
  • Euca also supports vpc and subnets
@tamastarjanyi

This comment has been minimized.

tamastarjanyi commented May 23, 2015

Same error on Ubuntu 14.10 using

ii  ansible                     1.9.1-1ppa~utopic  all                A radically simple IT automation platform

As a temporary fix till an official one is made the following modification worked for me with Eucalyptus 4.0.2

1207c1207
< 
---
>     vpc=None
1219,1220c1219,1220
<     else:
<         module.fail_json(msg="region must be specified")
---
>     #else:
>     #    module.fail_json(msg="region must be specified")
1237d1236
< 
@ahmari6

This comment has been minimized.

ahmari6 commented Jun 2, 2015

@bennojoy Any updates on this?

@lwade

This comment has been minimized.

Contributor

lwade commented Jun 15, 2015

@jimi-c

This comment has been minimized.

Member

jimi-c commented Aug 20, 2015

I have not had a chance to test this yet, but I believe this does what @bennojoy describes above:

diff --git a/cloud/amazon/ec2.py b/cloud/amazon/ec2.py
index 55c45a6..c2b57eb 100644
--- a/cloud/amazon/ec2.py
+++ b/cloud/amazon/ec2.py
@@ -824,7 +824,10 @@ def create_instances(module, ec2, vpc, override_count=None):
 
     vpc_id = None
     if vpc_subnet_id:
-        vpc_id = vpc.get_all_subnets(subnet_ids=[vpc_subnet_id])[0].vpc_id
+        if not vpc:
+            module.fail_json(msg="region must be specified")
+        else:
+            vpc_id = vpc.get_all_subnets(subnet_ids=[vpc_subnet_id])[0].vpc_id
     else:
         vpc_id = None
 
@@ -1281,7 +1284,7 @@ def main():
         except boto.exception.NoAuthHandlerFound, e:
             module.fail_json(msg = str(e))
     else:
-        module.fail_json(msg="region must be specified")
+        vpc = None
 
     tagged_instances = []
 

jimi-c added a commit to ansible/ansible-modules-core that referenced this issue Aug 20, 2015

@jimi-c

This comment has been minimized.

Member

jimi-c commented Aug 20, 2015

^ It's also available via the feature branch above.

@lwade

This comment has been minimized.

Contributor

lwade commented Aug 21, 2015

Thanks @jimi-c, seems to work ok:

[root@lwade-eucadcloud2-01 ~]# ansible-playbook provision.yml -e "type=c1.medium" --private-key=creds/lwade/lwade-key.priv

PLAY [Create a sandbox instance] ***********************************************

TASK [Launch instance] *********************************************************
changed: [localhost]

TASK [Add new instance to host group] ******************************************
changed: [localhost] => (item={u'kernel': None, u'root_device_type': u'instance-store', u'private_dns_name': u'10.112.26.246', u'public_ip': u'10.112.26.246', u'private_ip': u'10.112.26.246', u'id': u'i-6c6fcacf', u'ebs_optimized': False, u'state': u'pending', u'virtualization_type': u'hvm', u'architecture': u'x86_64', u'ramdisk': None, u'block_device_mapping': {}, u'key_name': u'lwade-key', u'image_id': u'emi-ef10ed2a', u'tenancy': None, u'groups': {u'sg-db1de7be': u'default'}, u'public_dns_name': u'10.112.26.246', u'state_code': 0, u'tags': {}, u'placement': u'euca-az-01', u'ami_launch_index': u'0', u'dns_name': u'10.112.26.246', u'region': u'euca-az-0', u'launch_time': u'2015-08-21T09:51:27.979Z', u'instance_type': u'c1.medium', u'root_device_name': u'/dev/sda1', u'hypervisor': None})
@lwade

This comment has been minimized.

Contributor

lwade commented Aug 21, 2015

I've not tried this on a VPC cloud yet but I'll see if I can give it a shot. Looking at the code I think I'm right in saying that even on a VPC non-AWS cloud it shouldn't complain about region not being specified?

@jimi-c

This comment has been minimized.

Member

jimi-c commented Aug 21, 2015

@lwade it won't, unless the vpc_subnet_id is specified.

@jimi-c

This comment has been minimized.

Member

jimi-c commented Aug 21, 2015

@lwade if there are no further comments, I'll go ahead and merge this in.

@lwade

This comment has been minimized.

Contributor

lwade commented Aug 22, 2015

Yes please @jimi-c. It gets us over a hump. Next step will be to test against a VPC cloud but this immediately fixes against all current Euca clouds.

jimi-c added a commit to ansible/ansible-modules-core that referenced this issue Aug 22, 2015

@jimi-c

This comment has been minimized.

Member

jimi-c commented Aug 22, 2015

Merged into both devel and stable-1.9, so we'll cut 1.9.3-rc3 asap. Thanks for testing!

If you continue seeing any problems related to this issue, or if you have any further questions, please let us know by stopping by one of the two mailing lists, as appropriate:

Because this project is very active, we're unlikely to see comments made on closed tickets, but the mailing list is a great way to ask questions, or post if you don't think this particular issue is resolved.

Thank you!

@ansibot ansibot added the bug label Mar 6, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment