Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Salt-Cloud failing to deploy to EC2 -- "There was a profile error: encode() argument 1 must be string, not None" #38287

Closed
danbudris opened this issue Dec 15, 2016 · 40 comments
Assignees
Labels
Bug broken, incorrect, or confusing behavior fixed-pls-verify fix is linked, bug author to confirm fix P3 Priority 3 RIoT Relates to integration with cloud providers, hypervisors, API-based services, etc. Salt-Cloud severity-high 2nd top severity, seen by most users, causes major problems
Milestone

Comments

@danbudris
Copy link

danbudris commented Dec 15, 2016

Description of Issue/Question

When attempting to launch an instance into AWS EC2 via salt-cloud, using a previously working profile, all attempts fail with error:

AWS Response Status Code: 400
Error Creating on EC2 when trying to run the Initial Deployment:
encode() argument 1 must be string, not none

Setup

salt-cloud -p base_ec2_private SERVERNAME -l debug

/etc/salt/cloud.profiles.d/profiles.conf:

base_ec2_private:
    provider: my-ec2-northeast-private-ips
    script: salt_deploy_w_dns
    image: ami-XXXXXXXXX
    size: t2.large
    subnetid: subnet-XXXXXXXXX
    ssh_username: ubuntu
    securitygroupid: sg-XXXXXXXXX
    volumes:
        - { size: 55, device: /dev/sdf }
    minion:
      grains:
        setup: False
        environment_type: qa
      startup_states: highstate`

/etc/salt/cloud.providers.d/providers.conf:

my-ec2-northeast-private-ips:

    #Set the location of the Salt Master
    minion:
        master: XXXXXXXXX

    #set grains information
    grains:
        dom: XXXXXX

    #Specify that the private IP is to be used for the script execution
    ssh_interface: private_ips

    #set the credientials to be used -- these are the master role credentials, which will be appended the next time we create a working Salt master

    id: 'XXXXXXXX'
    key: 'XXXXXXXXX'

    private_key: /etc/salt/XXXX.pem
    keyname: XXXXX

    #specify the Salt Driver
    driver: ec2

Steps to Reproduce Issue

(Include debug logs if possible and relevant.)

salt-cloud -p base_ec2_private 'SERVERNAME' -l debug
[DEBUG   ] Reading configuration from /etc/salt/cloud
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/master.d/reactor.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/reactor.conf
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: MASTER
[DEBUG   ] Missing configuration file: /etc/salt/cloud.providers
[DEBUG   ] Including configuration from '/etc/salt/cloud.providers.d/providers.conf'
[DEBUG   ] Reading configuration from /etc/salt/cloud.providers.d/providers.conf
[DEBUG   ] Missing configuration file: /etc/salt/cloud.profiles
[DEBUG   ] Including configuration from '/etc/salt/cloud.profiles.d/profiles.conf'
[DEBUG   ] Reading configuration from /etc/salt/cloud.profiles.d/profiles.conf
[DEBUG   ] Including configuration from '/etc/salt/cloud.profiles.d/profiles2.conf'
[DEBUG   ] Reading configuration from /etc/salt/cloud.profiles.d/profiles2.conf
[DEBUG   ] Configuration file path: /etc/salt/cloud
[WARNING ] Insecure logging configuration detected! Sensitive data may be logged.
[INFO    ] salt-cloud starting
[DEBUG   ] Could not LazyLoad parallels.avail_sizes: 'parallels' __virtual__ returned False
[DEBUG   ] LazyLoaded parallels.avail_locations
[DEBUG   ] LazyLoaded proxmox.avail_sizes
[DEBUG   ] Could not LazyLoad saltify.destroy: 'saltify.destroy' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_sizes: 'saltify.avail_sizes' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_images: 'saltify.avail_images' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_locations: 'saltify.avail_locations' is not available.
[DEBUG   ] LazyLoaded rackspace.reboot
[DEBUG   ] LazyLoaded openstack.list_locations
[DEBUG   ] LazyLoaded rackspace.list_locations
[DEBUG   ] Could not LazyLoad parallels.avail_sizes: 'parallels' __virtual__ returned False
[DEBUG   ] LazyLoaded parallels.avail_locations
[DEBUG   ] LazyLoaded proxmox.avail_sizes
[DEBUG   ] Could not LazyLoad saltify.destroy: 'saltify.destroy' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_sizes: 'saltify.avail_sizes' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_images: 'saltify.avail_images' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_locations: 'saltify.avail_locations' is not available.
[DEBUG   ] LazyLoaded rackspace.reboot
[DEBUG   ] LazyLoaded openstack.list_locations
[DEBUG   ] LazyLoaded rackspace.list_locations
[DEBUG   ] Using AWS endpoint: ec2.us-east-1.amazonaws.com
[DEBUG   ] AWS Request: https://ec2.us-east-1.amazonaws.com/?Action=DescribeInstances&Version=2014-10-01
[DEBUG   ] AWS Response Status Code: 200
[DEBUG   ] LazyLoaded cloud.cache_node_list
[DEBUG   ] Generating minion keys for '<SERVERNAME>'
[DEBUG   ] LazyLoaded cloud.fire_event
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Initializing new IPCClient for path: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Sending event: tag = salt/cloud/<SERVERNAME>/creating; data = {'profile': 'base_ec2_private', 'event': 'starting create', '_stamp': '2016-12-15T20:27:43.460003', 'name': '<SERVERNAME>', 'provider': 'my-ec2-northeast-private-ips:ec2'}
[INFO    ] Creating Cloud VM <SERVERNAME> in us-east-1
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Initializing new IPCClient for path: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Sending event: tag = salt/cloud/<SERVERNAME>/requesting; data = {'event': 'requesting instance', '_stamp': '2016-12-15T20:27:43.487663', 'location': 'us-east-1', 'kwargs': {'SecurityGroupId.7': 's', 'SecurityGroupId.6': '0', 'SecurityGroupId.5': '1', 'SecurityGroupId.4': '-', 'SecurityGroupId.3': 'f', 'SecurityGroupId.2': 'g', 'SecurityGroupId.1': 'd', 'SecurityGroupId.0': 'c', 'SecurityGroupId.9': '8', 'ImageId': 'ami-40d28157', 'SubnetId': 'subnet-3dd7b416', 'KeyName': 'Salt_Master_Test', 'MaxCount': 1, 'Action': 'RunInstances', 'MinCount': 1, 'SecurityGroupId.8': '6', 'InstanceType': 't2.large'}}
[DEBUG   ] Using AWS endpoint: ec2.us-east-1.amazonaws.com
[DEBUG   ] AWS Request: https://ec2.us-east-1.amazonaws.com/?Action=RunInstances&ImageId=ami-40d28157&InstanceType=t2.large&KeyName=MASTER&MaxCount=1&MinCount=1&SecurityGroupId.0=c&SecurityGroupId.1=d&SecurityGroupId.2=g&SecurityGroupId.3=f&SecurityGroupId.4=-&SecurityGroupId.5=1&SecurityGroupId.6=0&SecurityGroupId.7=s&SecurityGroupId.8=6&SecurityGroupId.9=8&SubnetId=subnet-XXXXXX&Version=2014-10-01
[DEBUG   ] AWS Response Status Code: 400
[ERROR   ] Error creating <SERVERNAME> on EC2 when trying to run the initial deployment:
encode() argument 1 must be string, not None
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/salt/cloud/clouds/ec2.py", line 1921, in request_instance
    sigver='4')
  File "/usr/lib/python2.7/dist-packages/salt/utils/aws.py", line 463, in query
    result.text.encode(result.encoding)
TypeError: encode() argument 1 must be string, not None
[ERROR   ] There was a profile error: encode() argument 1 must be string, not None
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/salt/cloud/cli.py", line 284, in run
    self.config.get('names')
  File "/usr/lib/python2.7/dist-packages/salt/cloud/__init__.py", line 1458, in run_profile
    ret[name] = self.create(vm_)
  File "/usr/lib/python2.7/dist-packages/salt/cloud/__init__.py", line 1288, in create
    output = self.clouds[func](vm_)
  File "/usr/lib/python2.7/dist-packages/salt/cloud/clouds/ec2.py", line 2510, in create
    data, vm_ = request_instance(vm_, location)
  File "/usr/lib/python2.7/dist-packages/salt/cloud/clouds/ec2.py", line 1921, in request_instance
    sigver='4')
  File "/usr/lib/python2.7/dist-packages/salt/utils/aws.py", line 463, in query
    result.text.encode(result.encoding)
TypeError: encode() argument 1 must be string, not None

Versions Report

(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)

Salt Version:
           Salt: 2016.11.1

Dependency Versions:
           cffi: Not Installed
       cherrypy: 3.5.0
       dateutil: 2.4.2
          gitdb: 0.6.4
      gitpython: 1.0.1
          ioflo: Not Installed
         Jinja2: 2.8
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: 1.0.3
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
         pygit2: Not Installed
         Python: 2.7.12 (default, Nov 19 2016, 06:48:10)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 15.2.0
           RAET: Not Installed
          smmap: 0.9.0
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4

System Versions:
           dist: Ubuntu 16.04 xenial
        machine: x86_64
        release: 4.4.0-34-generic
         system: Linux
        version: Ubuntu 16.04 xenial
@danbudris danbudris changed the title Salt-Cloud failing to deploy to EC2 -- "There was a profile error: encode() argument 1 must be string, not None Salt-Cloud failing to deploy to EC2 -- "There was a profile error: encode() argument 1 must be string, not None" Dec 15, 2016
@Ch3LL
Copy link
Contributor

Ch3LL commented Dec 15, 2016

@danbudris i believe this pr: #38183 will fix not being able to connect. can you try that and report back.

I believe this will fix the 400 error but I think we should keep this open so we do not stack trace here if someone is receiving a 400 error.

@Ch3LL Ch3LL added Bug broken, incorrect, or confusing behavior severity-high 2nd top severity, seen by most users, causes major problems P3 Priority 3 Salt-Cloud RIoT Relates to integration with cloud providers, hypervisors, API-based services, etc. fixed-pls-verify fix is linked, bug author to confirm fix labels Dec 15, 2016
@Ch3LL Ch3LL added this to the Approved milestone Dec 15, 2016
@danbudris
Copy link
Author

Hi, thanks for the fast response!

I've updated:
/usr/lib/python2.7/dist-packages/salt/cloud/clouds/ec2.py

With the updated config from 38183. Still experiencing the same encoding error issue as before.

@Ch3LL
Copy link
Contributor

Ch3LL commented Dec 15, 2016

Hmmm I was getting the same error you were getting when i set a securitygroupid in the config on versoin 2016.11.1 but when i removed the securitygroupid from teh config it worked.

And I also found that adding securitygroupid back into the config on the head of 2016.11 branch it also worked so i wonder if there is a different PR that fixes the issue

Just to make sure i'm tracking down the right issue can you try one more thing. If you just remove your securitygroupid does it work?

@danbudris
Copy link
Author

Yes, if I remove the securitygroupid from /etc/salt/cloud.profiles.d/profiles.conf it successfully deploys.

I'm going to confirm that the update from the PR took and get back to you in a moment.

@Ch3LL
Copy link
Contributor

Ch3LL commented Dec 15, 2016

K perfect! Let me know if it doesn't work and I can dive in and figure out which PR fixes it exactly.

Like I stated above though we still shouldn't be stacktracing on that encoding error when we get a 400 error so regardless we should do a better job on the error reporting there

@danbudris
Copy link
Author

Ok, yes, so confirmed on my test master that PR 38183 is not a fix for this issue.
However, again confirmed that removing the security group allows for provisioning.

@danbudris
Copy link
Author

After continued testing it is confirmed that this issue persists (fails to deploy, stack trace, encoding error) even after applying pr 38183.

@Ch3LL
Copy link
Contributor

Ch3LL commented Dec 16, 2016

Thanks for re-confirming the PR does not fix the issue. I'm diving into this more right now. will update what I find. Thanks

@Ch3LL
Copy link
Contributor

Ch3LL commented Dec 16, 2016

Okay I dived a little deeper into the issue. It seems its still not working on the head of 2016.11 so my test case must have been mistaken.

But what I realized is the reason I can get it working in 2016.11.0 and not 2016.11.1 is because securitygroupid's are never set and the one i have set in my config is incorrect. And on an earlier verison of salt I am seeing a more useful error: 'Error': {'Message': 'VPC security groups may not be used for a non-VPC launch', 'Code': 'InvalidParameterCombination'}}

In other words I was legitimately receiving a 400 error. Sorry for the incorrect test case.

So if you could edit some code so we can see more information about your 400 error code.

Can you convert: result.text.encode(result.encoding) to result.text on line 463 in /usr/lib/python2.7/dist-packages/salt/utils/aws.py".
That should spit out something more useful in the logs when it states: AWS Response Text
And paste a sanitized version of the output here.

@danbudris
Copy link
Author

danbudris commented Dec 16, 2016

Here's the debug log. I tried it both with a SecurityGroupID and SubnetID and without a SubnetID. Same issue both times.

salt-cloud -p base_ec2_private bugCheck -l debug
[DEBUG   ] Reading configuration from /etc/salt/cloud
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/master.d/reactor.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/reactor.conf
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: XTUV-SALTTEST
[DEBUG   ] Missing configuration file: /etc/salt/cloud.providers
[DEBUG   ] Including configuration from '/etc/salt/cloud.providers.d/providers.conf'
[DEBUG   ] Reading configuration from /etc/salt/cloud.providers.d/providers.conf
[DEBUG   ] Missing configuration file: /etc/salt/cloud.profiles
[DEBUG   ] Including configuration from '/etc/salt/cloud.profiles.d/profiles.conf'
[DEBUG   ] Reading configuration from /etc/salt/cloud.profiles.d/profiles.conf
[DEBUG   ] Including configuration from '/etc/salt/cloud.profiles.d/profiles2.conf'
[DEBUG   ] Reading configuration from /etc/salt/cloud.profiles.d/profiles2.conf
[DEBUG   ] Configuration file path: /etc/salt/cloud
[WARNING ] Insecure logging configuration detected! Sensitive data may be logged.
[INFO    ] salt-cloud starting
[DEBUG   ] Could not LazyLoad parallels.avail_sizes: 'parallels' __virtual__ returned False
[DEBUG   ] LazyLoaded parallels.avail_locations
[DEBUG   ] LazyLoaded proxmox.avail_sizes
[DEBUG   ] Could not LazyLoad saltify.destroy: 'saltify.destroy' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_sizes: 'saltify.avail_sizes' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_images: 'saltify.avail_images' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_locations: 'saltify.avail_locations' is not available.
[DEBUG   ] LazyLoaded rackspace.reboot
[DEBUG   ] LazyLoaded openstack.list_locations
[DEBUG   ] LazyLoaded rackspace.list_locations
[DEBUG   ] Could not LazyLoad parallels.avail_sizes: 'parallels' __virtual__ returned False
[DEBUG   ] LazyLoaded parallels.avail_locations
[DEBUG   ] LazyLoaded proxmox.avail_sizes
[DEBUG   ] Could not LazyLoad saltify.destroy: 'saltify.destroy' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_sizes: 'saltify.avail_sizes' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_images: 'saltify.avail_images' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_locations: 'saltify.avail_locations' is not available.
[DEBUG   ] LazyLoaded rackspace.reboot
[DEBUG   ] LazyLoaded openstack.list_locations
[DEBUG   ] LazyLoaded rackspace.list_locations
[DEBUG   ] Using AWS endpoint: ec2.us-east-1.amazonaws.com
[DEBUG   ] AWS Request: https://ec2.us-east-1.amazonaws.com/?Action=DescribeInstances&Version=2014-10-01
[DEBUG   ] Could not LazyLoad parallels.avail_sizes: 'parallels' __virtual__ returned False
[DEBUG   ] LazyLoaded parallels.avail_locations
[DEBUG   ] LazyLoaded proxmox.avail_sizes
[DEBUG   ] Could not LazyLoad saltify.destroy: 'saltify.destroy' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_sizes: 'saltify.avail_sizes' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_images: 'saltify.avail_images' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_locations: 'saltify.avail_locations' is not available.
[DEBUG   ] LazyLoaded rackspace.reboot
[DEBUG   ] LazyLoaded openstack.list_locations
[DEBUG   ] LazyLoaded rackspace.list_locations
[DEBUG   ] Using AWS endpoint: ec2.us-east-1.amazonaws.com
[DEBUG   ] AWS Request: https://ec2.us-east-1.amazonaws.com/?Action=DescribeInstances&Version=2014-10-01
[DEBUG   ] AWS Response Status Code: 200
[DEBUG   ] AWS Response Status Code: 200
[DEBUG   ] LazyLoaded cloud.cache_node_list
[DEBUG   ] LazyLoaded cloud.cache_node_list
[DEBUG   ] Generating minion keys for 'bugCheck'
[DEBUG   ] LazyLoaded cloud.fire_event
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Initializing new IPCClient for path: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Sending event: tag = salt/cloud/bugCheck/creating; data = {'profile': 'base_ec2_private', 'event': 'starting create', '_stamp': '2016-12-16T18:50:46.227134', 'name': 'bugCheck', 'provider': 'my-ec2-northeast-private-ips:ec2'}
[INFO    ] Creating Cloud VM bugCheck in us-east-1
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Initializing new IPCClient for path: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Sending event: tag = salt/cloud/bugCheck/requesting; data = {'event': 'requesting instance', '_stamp': '2016-12-16T18:50:46.254419', 'location': 'us-east-1', 'kwargs': {'SecurityGroupId.7': 's', 'SecurityGroupId.6': '0', 'SecurityGroupId.5': '1', 'SecurityGroupId.4': '-', 'SecurityGroupId.3': 'f', 'SecurityGroupId.2': 'g', 'SecurityGroupId.1': 'd', 'SecurityGroupId.0': 'c', 'SecurityGroupId.9': '8', 'ImageId': 'ami-40d28157', 'SubnetId': 'subnet-XXXXXX', 'KeyName': KEYNAME', 'MaxCount': 1, 'Action': 'RunInstances', 'MinCount': 1, 'SecurityGroupId.8': '6', 'InstanceType': 't2.large'}}
[DEBUG   ] Using AWS endpoint: ec2.us-east-1.amazonaws.com
[DEBUG   ] AWS Request: https://ec2.us-east-1.amazonaws.com/?Action=RunInstances&ImageId=ami-40d28157&InstanceType=t2.large&KeyName=KEYNAME&MaxCount=1&MinCount=1&SecurityGroupId.0=c&SecurityGroupId.1=d&SecurityGroupId.2=g&SecurityGroupId.3=f&SecurityGroupId.4=-&SecurityGroupId.5=1&SecurityGroupId.6=0&SecurityGroupId.7=s&SecurityGroupId.8=6&SecurityGroupId.9=8&SubnetId=subnet-XXXXXXX&Version=2014-10-01
[DEBUG   ] AWS Response Status Code: 400
[ERROR   ] AWS Response Status Code and Error: [400 400 Client Error: Bad Request for url: https://ec2.us-east-1.amazonaws.com/?Action=RunInstances&ImageId=ami-40d28157&InstanceType=t2.large&KeyName=Salt_Master_Test&MaxCount=1&MinCount=1&SecurityGroupId.0=c&SecurityGroupId.1=d&SecurityGroupId.2=g&SecurityGroupId.3=f&SecurityGroupId.4=-&SecurityGroupId.5=1&SecurityGroupId.6=0&SecurityGroupId.7=s&SecurityGroupId.8=6&SecurityGroupId.9=8&SubnetId=subnet-XXXXXX&Version=2014-10-01] {'Errors': {'Error': {'Message': 'The parameter groupName cannot be used with the parameter subnet', 'Code': 'InvalidParameterCombination'}}, 'RequestID': 'c55d1ed2-9f2d-420e-9e02-fa57b2a0db6e'}
[ERROR   ] Error requesting instance: Errors
Error: There was a profile error: Failed to deploy VM

@mlalpho
Copy link
Contributor

mlalpho commented Dec 16, 2016

PR here: #38316

@Ch3LL
Copy link
Contributor

Ch3LL commented Dec 16, 2016

@danbudris perfect thanks for applyig that patch. The error {'Message': 'The parameter groupName cannot be used with the parameter subnet' I believe is what we want to be concerned about.

This is a little out of my realm at this point though and not sure as to why you are getting this conflict. ping @techhat any ideas why one would get that error above refering to the groupname and subnet parameter.

I did find this on stack overflow and maybe that is related but i'm not sure. hopefully techat has some more useful insight.

Also thanks @mlalpho for the fix. That fixes the encoding issue :)

@mlalpho
Copy link
Contributor

mlalpho commented Dec 16, 2016

yep, just reread and doesn't like it's the whole issue if he's applied your patch. don't mind me :) just saw "error: encode() argument 1 must be string, not None" which was directly related to PR :)

@Ch3LL
Copy link
Contributor

Ch3LL commented Dec 16, 2016

yep it seems there might be two issues going on here. Oh and i just had him switch it back to the original code so we could just quickly get the real reason behind the 400 error. I didn't mean for my patch to be the solution just a workaround.but yeah looks like your pr will fix that encode error 👍

@danbudris
Copy link
Author

danbudris commented Dec 16, 2016

Hi! Thanks again for being extremely responsive.
I've applied #38316 just to be sure --

When launching using

securitygroupid: sg-XXXXXXXXX

In /etc/salt/cloud.profiles.d/profiles.conf

I still recieve the following 400 response:

salt-cloud -p base_ec2_test BugCheck3 -l debug
[DEBUG   ] Reading configuration from /etc/salt/cloud
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/master.d/reactor.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/reactor.conf
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: XTUV-SALTTEST
[DEBUG   ] Missing configuration file: /etc/salt/cloud.providers
[DEBUG   ] Including configuration from '/etc/salt/cloud.providers.d/providers.conf'
[DEBUG   ] Reading configuration from /etc/salt/cloud.providers.d/providers.conf
[DEBUG   ] Missing configuration file: /etc/salt/cloud.profiles
[DEBUG   ] Including configuration from '/etc/salt/cloud.profiles.d/profiles.conf'
[DEBUG   ] Reading configuration from /etc/salt/cloud.profiles.d/profiles.conf
[DEBUG   ] Including configuration from '/etc/salt/cloud.profiles.d/profiles2.conf'
[DEBUG   ] Reading configuration from /etc/salt/cloud.profiles.d/profiles2.conf
[DEBUG   ] Configuration file path: /etc/salt/cloud
[WARNING ] Insecure logging configuration detected! Sensitive data may be logged.
[INFO    ] salt-cloud starting
[DEBUG   ] Could not LazyLoad parallels.avail_sizes: 'parallels' __virtual__ returned False
[DEBUG   ] LazyLoaded parallels.avail_locations
[DEBUG   ] LazyLoaded proxmox.avail_sizes
[DEBUG   ] Could not LazyLoad saltify.destroy: 'saltify.destroy' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_sizes: 'saltify.avail_sizes' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_images: 'saltify.avail_images' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_locations: 'saltify.avail_locations' is not available.
[DEBUG   ] LazyLoaded rackspace.reboot
[DEBUG   ] LazyLoaded openstack.list_locations
[DEBUG   ] LazyLoaded rackspace.list_locations
[DEBUG   ] Could not LazyLoad parallels.avail_sizes: 'parallels' __virtual__ returned False
[DEBUG   ] LazyLoaded parallels.avail_locations
[DEBUG   ] LazyLoaded proxmox.avail_sizes
[DEBUG   ] Could not LazyLoad saltify.destroy: 'saltify.destroy' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_sizes: 'saltify.avail_sizes' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_images: 'saltify.avail_images' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_locations: 'saltify.avail_locations' is not available.
[DEBUG   ] LazyLoaded rackspace.reboot
[DEBUG   ] LazyLoaded openstack.list_locations
[DEBUG   ] LazyLoaded rackspace.list_locations
[DEBUG   ] Using AWS endpoint: ec2.us-east-1.amazonaws.com
[DEBUG   ] AWS Request: https://ec2.us-east-1.amazonaws.com/?Action=DescribeInstances&Version=2014-10-01
[DEBUG   ] Could not LazyLoad parallels.avail_sizes: 'parallels' __virtual__ returned False
[DEBUG   ] LazyLoaded parallels.avail_locations
[DEBUG   ] LazyLoaded proxmox.avail_sizes
[DEBUG   ] Could not LazyLoad saltify.destroy: 'saltify.destroy' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_sizes: 'saltify.avail_sizes' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_images: 'saltify.avail_images' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_locations: 'saltify.avail_locations' is not available.
[DEBUG   ] LazyLoaded rackspace.reboot
[DEBUG   ] LazyLoaded openstack.list_locations
[DEBUG   ] LazyLoaded rackspace.list_locations
[DEBUG   ] Using AWS endpoint: ec2.us-east-1.amazonaws.com
[DEBUG   ] AWS Request: https://ec2.us-east-1.amazonaws.com/?Action=DescribeInstances&Version=2014-10-01
[DEBUG   ] AWS Response Status Code: 200
[DEBUG   ] LazyLoaded cloud.cache_node_list
[DEBUG   ] AWS Response Status Code: 200
[DEBUG   ] LazyLoaded cloud.cache_node_list
[DEBUG   ] Generating minion keys for 'BugCheck3'
[DEBUG   ] LazyLoaded cloud.fire_event
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Initializing new IPCClient for path: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Sending event: tag = salt/cloud/BugCheck3/creating; data = {'profile': 'base_ec2_test', 'event': 'starting create', '_stamp': '2016-12-16T19:26:39.582736', 'name': 'BugCheck3', 'provider': 'my-ec2-northeast-private-ips:ec2'}
[INFO    ] Creating Cloud VM BugCheck3 in us-east-1
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Initializing new IPCClient for path: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Sending event: tag = salt/cloud/BugCheck3/requesting; data = {'event': 'requesting instance', '_stamp': '2016-12-16T19:26:39.609874', 'location': 'us-east-1', 'kwargs': {'SecurityGroupId.7': '7', 'SecurityGroupId.6': '5', 'SecurityGroupId.5': '2', 'SecurityGroupId.4': 's', 'SecurityGroupId.3': '-', 'SecurityGroupId.2': 'g', 'SecurityGroupId.1': 'd', 'SecurityGroupId.0': 'a', 'SecurityGroupId.9': '8', 'ImageId': 'ami-e13739f6', 'SubnetId': 'subnet-XXXXXX', 'KeyName': 'Salt_Master_Test', 'MaxCount': 1, 'Action': 'RunInstances', 'MinCount': 1, 'SecurityGroupId.8': '9', 'InstanceType': 't2.large'}}
[DEBUG   ] Using AWS endpoint: ec2.us-east-1.amazonaws.com
[DEBUG   ] AWS Request: https://ec2.us-east-1.amazonaws.com/?Action=RunInstances&ImageId=ami-e13739f6&InstanceType=t2.large&KeyName=Salt_Master_Test&MaxCount=1&MinCount=1&SecurityGroupId.0=a&SecurityGroupId.1=d&SecurityGroupId.2=g&SecurityGroupId.3=-&SecurityGroupId.4=s&SecurityGroupId.5=2&SecurityGroupId.6=5&SecurityGroupId.7=7&SecurityGroupId.8=9&SecurityGroupId.9=8&SubnetId=subnet-XXXXXX&Version=2014-10-01
[DEBUG   ] AWS Response Status Code: 400
[ERROR   ] AWS Response Status Code and Error: [400 400 Client Error: Bad Request for url: https://ec2.us-east-1.amazonaws.com/?Action=RunInstances&ImageId=ami-e13739f6&InstanceType=t2.large&KeyName=Salt_Master_Test&MaxCount=1&MinCount=1&SecurityGroupId.0=a&SecurityGroupId.1=d&SecurityGroupId.2=g&SecurityGroupId.3=-&SecurityGroupId.4=s&SecurityGroupId.5=2&SecurityGroupId.6=5&SecurityGroupId.7=7&SecurityGroupId.8=9&SecurityGroupId.9=8&SubnetId=subnet-XXXXXXX&Version=2014-10-01] {'Errors': {'Error': {'Message': 'The parameter groupName cannot be used with the parameter subnet', 'Code': 'InvalidParameterCombination'}}, 'RequestID': '1f601978-bc56-4b5e-990c-fc6c7c8410f6'}
[ERROR   ] Error requesting instance: Errors
Error: There was a profile error: Failed to deploy VM

When deploying using securitygroup: NameOfGroup in /etc/salt/cloud.profiles.d/profiles.conf the machine is created, but without a security group:

salt-cloud -p base_ec2_test BugCheck2 -l debug
[DEBUG   ] Reading configuration from /etc/salt/cloud
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/master.d/reactor.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/reactor.conf
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: XTUV-SALTTEST
[DEBUG   ] Missing configuration file: /etc/salt/cloud.providers
[DEBUG   ] Including configuration from '/etc/salt/cloud.providers.d/providers.conf'
[DEBUG   ] Reading configuration from /etc/salt/cloud.providers.d/providers.conf
[DEBUG   ] Missing configuration file: /etc/salt/cloud.profiles
[DEBUG   ] Including configuration from '/etc/salt/cloud.profiles.d/profiles.conf'
[DEBUG   ] Reading configuration from /etc/salt/cloud.profiles.d/profiles.conf
[DEBUG   ] Including configuration from '/etc/salt/cloud.profiles.d/profiles2.conf'
[DEBUG   ] Reading configuration from /etc/salt/cloud.profiles.d/profiles2.conf
[DEBUG   ] Configuration file path: /etc/salt/cloud
[WARNING ] Insecure logging configuration detected! Sensitive data may be logged.
[INFO    ] salt-cloud starting
[DEBUG   ] Could not LazyLoad parallels.avail_sizes: 'parallels' __virtual__ returned False
[DEBUG   ] LazyLoaded parallels.avail_locations
[DEBUG   ] LazyLoaded proxmox.avail_sizes
[DEBUG   ] Could not LazyLoad saltify.destroy: 'saltify.destroy' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_sizes: 'saltify.avail_sizes' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_images: 'saltify.avail_images' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_locations: 'saltify.avail_locations' is not available.
[DEBUG   ] LazyLoaded rackspace.reboot
[DEBUG   ] LazyLoaded openstack.list_locations
[DEBUG   ] LazyLoaded rackspace.list_locations
[DEBUG   ] Could not LazyLoad parallels.avail_sizes: 'parallels' __virtual__ returned False
[DEBUG   ] LazyLoaded parallels.avail_locations
[DEBUG   ] LazyLoaded proxmox.avail_sizes
[DEBUG   ] Could not LazyLoad saltify.destroy: 'saltify.destroy' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_sizes: 'saltify.avail_sizes' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_images: 'saltify.avail_images' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_locations: 'saltify.avail_locations' is not available.
[DEBUG   ] LazyLoaded rackspace.reboot
[DEBUG   ] LazyLoaded openstack.list_locations
[DEBUG   ] LazyLoaded rackspace.list_locations
[DEBUG   ] Using AWS endpoint: ec2.us-east-1.amazonaws.com
[DEBUG   ] AWS Request: https://ec2.us-east-1.amazonaws.com/?Action=DescribeInstances&Version=2014-10-01
[DEBUG   ] Could not LazyLoad parallels.avail_sizes: 'parallels' __virtual__ returned False
[DEBUG   ] LazyLoaded parallels.avail_locations
[DEBUG   ] LazyLoaded proxmox.avail_sizes
[DEBUG   ] Could not LazyLoad saltify.destroy: 'saltify.destroy' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_sizes: 'saltify.avail_sizes' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_images: 'saltify.avail_images' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_locations: 'saltify.avail_locations' is not available.
[DEBUG   ] LazyLoaded rackspace.reboot
[DEBUG   ] LazyLoaded openstack.list_locations
[DEBUG   ] LazyLoaded rackspace.list_locations
[DEBUG   ] Using AWS endpoint: ec2.us-east-1.amazonaws.com
[DEBUG   ] AWS Request: https://ec2.us-east-1.amazonaws.com/?Action=DescribeInstances&Version=2014-10-01
[DEBUG   ] AWS Response Status Code: 200
[DEBUG   ] AWS Response Status Code: 200
[DEBUG   ] LazyLoaded cloud.cache_node_list
[DEBUG   ] LazyLoaded cloud.cache_node_list
[DEBUG   ] Generating minion keys for 'BugCheck2'
[DEBUG   ] LazyLoaded cloud.fire_event
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Initializing new IPCClient for path: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Sending event: tag = salt/cloud/BugCheck2/creating; data = {'profile': 'base_ec2_test', 'event': 'starting create', '_stamp': '2016-12                                           -16T19:24:04.953104', 'name': 'BugCheck2', 'provider': 'my-ec2-northeast-private-ips:ec2'}
[INFO    ] Creating Cloud VM BugCheck2 in us-east-1
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Initializing new IPCClient for path: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Sending event: tag = salt/cloud/BugCheck2/requesting; data = {'event': 'requesting instance', '_stamp': '2016-12-16T19:24:04.980464',                                            'location': 'us-east-1', 'kwargs': {'ImageId': 'ami-e13739f6', 'SubnetId': 'subnet-XXXXXXX', 'KeyName': 'Salt_Master_Test', 'MaxCount': 1, 'Acti                                           on': 'RunInstances', 'MinCount': 1, 'InstanceType': 't2.large'}}
[DEBUG   ] Using AWS endpoint: ec2.us-east-1.amazonaws.com
[DEBUG   ] AWS Request: https://ec2.us-east-1.amazonaws.com/?Action=RunInstances&ImageId=ami-e13739f6&InstanceType=t2.large&KeyName=Salt_Master_T                                           est&MaxCount=1&MinCount=1&SubnetId=subnet-XXXXXXX&Version=2014-10-01
[DEBUG   ] AWS Response Status Code: 200
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Initializing new IPCClient for path: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Sending event: tag = salt/cloud/BugCheck2/querying; data = {'instance_id': 'i-026e04f380ce0dcfa', '_stamp': '2016-12-16T19:24:06.10545                                           8', 'event': 'querying instance'}
[DEBUG   ] The new VM instance_id is i-026e04f380ce0dcfa
[DEBUG   ] Using AWS endpoint: ec2.us-east-1.amazonaws.com
[DEBUG   ] AWS Request: https://ec2.us-east-1.amazonaws.com/?Action=DescribeInstances&InstanceId.1=i-026e04f380ce0dcfa&Version=2014-10-01
[DEBUG   ] AWS Response Status Code: 503
[ERROR   ] AWS Response Status Code and Error: [503 503 Server Error: Service Unavailable for url: https://ec2.us-east-1.amazonaws.com/?Action=De                                           scribeInstances&InstanceId.1=i-026e04f380ce0dcfa&Version=2014-10-01] {'Errors': {'Error': {'Message': 'The service is unavailable. Please try aga                                           in shortly.', 'Code': 'Unavailable'}}, 'RequestID': '9f4c0fec-41cf-4e19-92ee-5265381586e8'}; Attempts remaining: 4
[DEBUG   ] AWS Request: https://ec2.us-east-1.amazonaws.com/?Action=DescribeInstances&InstanceId.1=i-026e04f380ce0dcfa&Version=2014-10-01
[DEBUG   ] AWS Response Status Code: 200
[DEBUG   ] The query returned: [{'ownerId': '893642485598', 'groupSet': None, 'reservationId': 'r-07bdd4c9452627953', 'instancesSet': {'item': {'                                           productCodes': None, 'vpcId': 'vpc-6ac5c70f', 'instanceId': 'i-026e04f380ce0dcfa', 'imageId': 'ami-e13739f6', 'keyName': 'Salt_Master_Test', 'cli                                           entToken': None, 'subnetId': 'subnet-XXXXXXX', 'amiLaunchIndex': '0', 'instanceType': 't2.large', 'groupSet': {'item': {'groupName': 'default',                                            'groupId': 'sg-ff6c8d98'}}, 'monitoring': {'state': 'disabled'}, 'dnsName': None, 'privateIpAddress': '10.162.27.62', 'virtualizationType': 'hvm'                                           , 'privateDnsName': 'ip-10-162-27-62.ec2.internal', 'reason': None, 'sourceDestCheck': 'true', 'blockDeviceMapping': {'item': {'deviceName': '/de                                           v/sda1', 'ebs': {'status': 'attaching', 'deleteOnTermination': 'true', 'volumeId': 'vol-039ff285472fb963a', 'attachTime': '2016-12-16T19:24:06.00                                           0Z'}}}, 'placement': {'groupName': None, 'tenancy': 'default', 'availabilityZone': 'us-east-1d'}, 'instanceState': {'code': '0', 'name': 'pending                                           '}, 'networkInterfaceSet': {'item': {'status': 'in-use', 'macAddress': '12:84:e9:b0:7e:c0', 'sourceDestCheck': 'true', 'vpcId': 'vpc-6ac5c70f', '                                           description': None, 'networkInterfaceId': 'eni-8d86527c', 'groupSet': {'item': {'groupName': 'default', 'groupId': 'sg-ff6c8d98'}}, 'attachment':                                            {'status': 'attaching', 'deviceIndex': '0', 'deleteOnTermination': 'true', 'attachmentId': 'eni-attach-3b88cbb8', 'attachTime': '2016-12-16T19:2                                           4:06.000Z'}, 'subnetId': 'subnet-XXXXXXX', 'ownerId': '893642485598', 'privateIpAddressesSet': {'item': {'primary': 'true', 'privateIpAddress':                                            '10.162.27.62'}}, 'privateIpAddress': '10.162.27.62'}}, 'ebsOptimized': 'false', 'launchTime': '2016-12-16T19:24:06.000Z', 'architecture': 'x86_6                                           4', 'hypervisor': 'xen', 'rootDeviceType': 'ebs', 'rootDeviceName': '/dev/sda1'}}}]
[DEBUG   ] Waiting for VM IP. Giving up in 00:10:00.
[DEBUG   ] Using AWS endpoint: ec2.us-east-1.amazonaws.com
[DEBUG   ] AWS Request: https://ec2.us-east-1.amazonaws.com/?Action=DescribeInstances&InstanceId.1=i-026e04f380ce0dcfa&Version=2014-10-01
[DEBUG   ] AWS Response Status Code: 200
[DEBUG   ] Returned query data: [{'ownerId': '893642485598', 'groupSet': None, 'reservationId': 'r-07bdd4c9452627953', 'instancesSet': {'item': {                                           'productCodes': None, 'vpcId': 'vpc-6ac5c70f', 'instanceId': 'i-026e04f380ce0dcfa', 'imageId': 'ami-e13739f6', 'keyName': 'Salt_Master_Test', 'cl                                           ientToken': None, 'subnetId': 'subnet-XXXXXXX', 'amiLaunchIndex': '0', 'instanceType': 't2.large', 'groupSet': {'item': {'groupName': 'default',                                            'groupId': 'sg-ff6c8d98'}}, 'monitoring': {'state': 'disabled'}, 'dnsName': None, 'privateIpAddress': '10.162.27.62', 'virtualizationType': 'hvm                                           ', 'privateDnsName': 'ip-10-162-27-62.ec2.internal', 'reason': None, 'sourceDestCheck': 'true', 'blockDeviceMapping': {'item': {'deviceName': '/d                                           ev/sda1', 'ebs': {'status': 'attaching', 'deleteOnTermination': 'true', 'volumeId': 'vol-039ff285472fb963a', 'attachTime': '2016-12-16T19:24:06.0                                           00Z'}}}, 'placement': {'groupName': None, 'tenancy': 'default', 'availabilityZone': 'us-east-1d'}, 'instanceState': {'code': '0', 'name': 'pendin                                           g'}, 'networkInterfaceSet': {'item': {'status': 'in-use', 'macAddress': '12:84:e9:b0:7e:c0', 'sourceDestCheck': 'true', 'vpcId': 'vpc-6ac5c70f',                                            'description': None, 'networkInterfaceId': 'eni-8d86527c', 'groupSet': {'item': {'groupName': 'default', 'groupId': 'sg-ff6c8d98'}}, 'attachment'                                           : {'status': 'attaching', 'deviceIndex': '0', 'deleteOnTermination': 'true', 'attachmentId': 'eni-attach-3b88cbb8', 'attachTime': '2016-12-16T19:                                           24:06.000Z'}, 'subnetId': 'subnet-XXXXXXX', 'ownerId': '893642485598', 'privateIpAddressesSet': {'item': {'primary': 'true', 'privateIpAddress':                                            '10.162.27.62'}}, 'privateIpAddress': '10.162.27.62'}}, 'ebsOptimized': 'false', 'launchTime': '2016-12-16T19:24:06.000Z', 'architecture': 'x86_                                           64', 'hypervisor': 'xen', 'rootDeviceType': 'ebs', 'rootDeviceName': '/dev/sda1'}}}]
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Initializing new IPCClient for path: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Sending event: tag = salt/cloud/BugCheck2/tagging; data = {'_stamp': '2016-12-16T19:24:08.470266', 'event': 'setting tags', 'tags': {'                                           Name': 'BugCheck2'}}
[DEBUG   ] Tags to set for BugCheck2: {'Name': 'BugCheck2'}
[DEBUG   ] Using AWS endpoint: ec2.us-east-1.amazonaws.com
[DEBUG   ] AWS Request: https://ec2.us-east-1.amazonaws.com/?Action=CreateTags&ResourceId.1=i-026e04f380ce0dcfa&Tag.0.Key=Name&Tag.0.Value=BugChe                                           ck2&Version=2014-10-01
[DEBUG   ] AWS Response Status Code: 200
[DEBUG   ] Using AWS endpoint: ec2.us-east-1.amazonaws.com
[DEBUG   ] AWS Request: https://ec2.us-east-1.amazonaws.com/?Action=DescribeTags&Filter.1.Name=resource-id&Filter.1.Value=i-026e04f380ce0dcfa&Ver                                           sion=2014-10-01
[DEBUG   ] AWS Response Status Code: 200
[DEBUG   ] Setting the tags returned: [{'resourceType': 'instance', 'resourceId': 'i-026e04f380ce0dcfa', 'value': 'BugCheck2', 'key': 'Name'}]
[INFO    ] Created node BugCheck2
[INFO    ] Salt node data. Private_ip: 10.162.27.62
[INFO    ] Salt interface set to: 10.162.27.62
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Initializing new IPCClient for path: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Sending event: tag = salt/cloud/BugCheck2/waiting_for_ssh; data = {'_stamp': '2016-12-16T19:24:08.736971', 'ip_address': '10.162.27.62                                           ', 'event': 'waiting for ssh'}
[DEBUG   ] Attempting connection to host 10.162.27.62 on port 22
[DEBUG   ] Caught exception in wait_for_port: timed out
[DEBUG   ] Retrying connection to host 10.162.27.62 on port 22 (try 1)
[DEBUG   ] Caught exception in wait_for_port: timed out
[DEBUG   ] Retrying connection to host 10.162.27.62 on port 22 (try 2)
[DEBUG   ] Caught exception in wait_for_port: timed out
[DEBUG   ] Retrying connection to host 10.162.27.62 on port 22 (try 3)
[DEBUG   ] Caught exception in wait_for_port: timed out
[DEBUG   ] Retrying connection to host 10.162.27.62 on port 22 (try 4)

@Ch3LL
Copy link
Contributor

Ch3LL commented Dec 16, 2016

@danbudris thanks for trying that. do you have the patch from #38183 applied as well?

@danbudris
Copy link
Author

Yep, 38183 is still applied

@Ch3LL
Copy link
Contributor

Ch3LL commented Dec 16, 2016

K thanks for clarifying. yeah i've hit my limit on this one and hoping @techhat has more insight.

@danbudris
Copy link
Author

Thanks for all your help @Ch3LL

@meggiebot meggiebot removed the fixed-pls-verify fix is linked, bug author to confirm fix label Dec 19, 2016
@meggiebot meggiebot added this to the Nitrogen 2 milestone Dec 19, 2016
@ankitforcode
Copy link

@Ch3LL hey, any update the issue seems to be with 2016.11.1 and 2016.11.0 and any previous version.

@ankitforcode
Copy link

ankitforcode commented Dec 27, 2016

[ERROR   ] Error creating <SERVERNAME> on EC2 when trying to run the initial deployment: 
encode() argument 1 must be string, not None
[ERROR   ] There was a profile error: encode() argument 1 must be string, not None

@danbudris
Copy link
Author

@ankitforcode I've had to revert to a previous version of Salt in order to continue my deployments

@fabianmurariu
Copy link

This happened to me because:

salt.utils.aws.py line 463
result.text.encode(result.encoding) and in some cases encoding comes back None

I replaced it with
result.text.encode(result.encoding if result.encoding else 'utf-8')

It seems to be fixed in develop as well

@MDPolaris
Copy link

I'm not sure if this is helpful, but I applied the change from @fabianmurariu and that allowed me to see I had a bad value for my keypair name. Hopefully this helps narrow down where the issue is occurring.

@Ch3LL
Copy link
Contributor

Ch3LL commented Jan 6, 2017

The stack trace and original issue has been resolved by #38316

I have kept this open to see if we can't resolve @danbudris 's other issue that we found once the stack trace stopped occuring.

If @techhat does not get to this, i'll ping him personally on monday to see if we can't get this issue further resolved for you @danbudris

@Ch3LL
Copy link
Contributor

Ch3LL commented Jan 11, 2017

@danbudris just spoke with @techhat offline. Would you be willing to open a separate issue with details just about the error {'Message': 'The parameter groupName cannot be used with the parameter subnet' error with all the information, so we can close this issue since the initial stack trace error was resolved and this error is now a seperate error? The issue has gotten a little messy to try to distinguish between the two so would really help him to look through just one issue.

@danbudris
Copy link
Author

danbudris commented Jan 12, 2017 via email

@meggiebot meggiebot modified the milestones: Nitrogen 3, Nitrogen 2 Jan 18, 2017
@thetoolsmith
Copy link
Contributor

thetoolsmith commented Jan 19, 2017

I'm also getting with with 2016.11.1. I removed the security group entirely from the profile and still get
[DEBUG ] AWS Response Status Code: 400
[ERROR ] Error creating ap-ui1.us-east-1b.dev.org.com on EC2 when trying to run the initial deployment:
encode() argument 1 must be string, not None
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/salt/cloud/clouds/ec2.py", line 1921, in request_instance
sigver='4')
File "/usr/lib/python2.7/site-packages/salt/utils/aws.py", line 463, in query
result.text.encode(result.encoding)
TypeError: encode() argument 1 must be string, not None
[ERROR ] Caught Exception, terminating workers
TRACE: encode() argument 1 must be string, not None
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/salt/cloud/init.py", line 60, in call
ret = func(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/salt/cloud/init.py", line 2311, in create_multiprocessing
local_master=parallel_data['local_master']
File "/usr/lib/python2.7/site-packages/salt/cloud/init.py", line 1288, in create
output = self.cloudsfunc
File "/usr/lib/python2.7/site-packages/salt/cloud/clouds/ec2.py", line 2510, in create
data, vm
= request_instance(vm_, location)
File "/usr/lib/python2.7/site-packages/salt/cloud/clouds/ec2.py", line 1921, in request_instance
sigver='4')
File "/usr/lib/python2.7/site-packages/salt/utils/aws.py", line 463, in query
result.text.encode(result.encoding)
TypeError: encode() argument 1 must be string, not None

My profile looks like this:
xxx-create-ec2-us-east-1-dev:
provider: xxx-ec2-us-east-1-dev
script_args: '-P git v2016.11.1'
minion:
environment: dev
startup_states: sls
sls_list:
- new-instance
availability_zone: us-east-1b
image: ami-xxxxxxxxx
ssh_username: centos
network_interfaces:
- DeviceIndex: 0
SubnetId: subnet-xxxxxx
allocate_new_eip: False

My provider looks like this:
xxx-ec2-us-east-1-dev:
rename_on_destroy: False
minion:
master: x.x.x.x
size: t2.small
ssh_interface: private_ips
id: xxxxxxxxxxxxxxxxx
key: xxxxxxxxxxxxxxxx
private_key: /home/centos/.ssh/mykey
keyname: mykey
location: us-east-1
driver: ec2

My cloud map looks like this:
xxx-create-ec2-us-east-1-dev:

  • mymachine.org.com:
    size: m4.xlarge
    ebs_optimized: False
    block_device_mappings:
    - DeviceName: /dev/sda1
    Ebs.VolumeSize: 100
    Ebs.VolumeType: gp2
    del_all_vols_on_destroy: True
    volumes:
    grains:
    node.index: 1
    role: ui
    role.id: ui
    node_location: us-east-1b

I too started out using provider.conf profile.conf and maps that worked on a previous salt version (2015.8.8.2)

@mlalpho
Copy link
Contributor

mlalpho commented Jan 19, 2017

@thetoolsmith that specific error has been fixed and merged into develop (#38316)

See if you still have any problems after switching your salt installation to develop branch

@thetoolsmith
Copy link
Contributor

@mlalpho Thanks. This might be a stupid question, but what's the best approach to switch to develop branch on my newly setup salt-master?
I don't want to rebuild it obviously.

@mlalpho
Copy link
Contributor

mlalpho commented Jan 19, 2017

@thetoolsmith If you don't want to reinstall salt or use salt-bootstrap and you're feeling brave, you could overwrite the file utils/aws.py with this

run salt-call grains.get saltpath for the location

@thetoolsmith
Copy link
Contributor

This fixed my issue. Just provisioned a vm in aws. However I got a new issue. Should I open another issue? Looks like the bootstrap failed.

SaltCloudSystemExit: Command 'ssh -t -t -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oControlPath=none -oPasswordAuthentication=no -oChallengeResponseAuthentication=no -oPubkeyAuthentication=yes -oIdentitiesOnly=yes -oKbdInteractiveAuthentication=no -i /home/centos/.ssh/mykey -p 22 centos@10.0.1.81 'sudo /tmp/.saltcloud-8310720c-f6c4-4c37-aa3c-7ea46733ed7e/deploy.sh -c '"'"'/tmp/.saltcloud-8310720c-f6c4-4c37-aa3c-7ea46733ed7e'"'"' -P git v2016.11.1'' failed. Exit code: 255
[DEBUG ] LazyLoaded json.output
{
"mynode.us-east-1b.dev.org.com": {
"Error": "Command 'ssh -t -t -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oControlPath=none -oPasswordAuthentication=no -oChallengeResponseAuthentication=no -oPubkeyAuthentication=yes -oIdentitiesOnly=yes -oKbdInteractiveAuthentication=no -i /home/centos/.ssh/mykey -p 22 centos@10.0.1.81 'sudo /tmp/.saltcloud-8310720c-f6c4-4c37-aa3c-7ea46733ed7e/deploy.sh -c '"'"'/tmp/.saltcloud-8310720c-f6c4-4c37-aa3c-7ea46733ed7e'"'"' -P git v2016.11.1'' failed. Exit code: 255"

@thetoolsmith
Copy link
Contributor

running deploy.sh from the new vm console yields a bit more:

[root@ip-10-0-1-81 .saltcloud-8310720c-f6c4-4c37-aa3c-7ea46733ed7e]# ./deploy.sh -P 'git v2016.11.1'

  • INFO: Running version: 2016.10.25

  • INFO: Executed by: /bin/sh

  • INFO: Command line: './deploy.sh -P git v2016.11.1'

  • INFO: System Information:

  • INFO: CPU: GenuineIntel

  • INFO: CPU Arch: x86_64

  • INFO: OS Name: Linux

  • INFO: OS Version: 3.10.0-327.10.1.el7.x86_64

  • INFO: Distribution: CentOS 7.2

  • INFO: Installing minion

  • INFO: Found function install_centos_git

  • INFO: Found function config_salt

  • INFO: Found function preseed_master

  • INFO: Found function install_centos_git

  • INFO: Found function install_centos_git

  • INFO: Found function install_centos_git

  • INFO: Found function daemons_running

  • INFO: Found function install_centos_git

  • INFO: Running install_centos_git()
    python2: can't open file 'setup.py': [Errno 2] No such file or directory

  • ERROR: Failed to run install_centos_git()!!!

@mlalpho
Copy link
Contributor

mlalpho commented Jan 19, 2017

@thetoolsmith I'd recommend opening up another issue or if you patched the file try doing a clean salt reinstall of your master to develop just to be sure that it's not an issue with the patched file.

@thetoolsmith
Copy link
Contributor

I just opened saltstack/salt-bootstrap#1019.

Thank you.

@meggiebot meggiebot removed this from the Nitrogen 3 milestone Jan 30, 2017
@Ch3LL Ch3LL added this to the Approved milestone Jan 30, 2017
@gtmanfred gtmanfred added the fixed-pls-verify fix is linked, bug author to confirm fix label Mar 1, 2017
@gtmanfred
Copy link
Contributor

The original issue here should be fixed in #39228

Thanks,
Daniel

@arthurzenika
Copy link
Contributor

affected by this too

@Ch3LL
Copy link
Contributor

Ch3LL commented Mar 6, 2017

@arthurlogilab this original issue has been resolved with #39228

I will close this issue and wait for @danbudris to open a separate issue with the separate error he is getting.

@Ch3LL Ch3LL closed this as completed Mar 6, 2017
@bogdanr
Copy link
Contributor

bogdanr commented Jul 19, 2017

In which versions should we expect to see this fixed?
I can still see the issue in salt-cloud 2016.11.2 (Carbon)

@gtmanfred
Copy link
Contributor

The easiest way to do this is to check which tags a commit is in

git tag --contains <commitid>| head -n1

In this case, the first tag that the commit in that PR is in is

$ git tag --contains afee047b082f18a4cd83c712d667ce1b818488f2 | head -n1
v2016.11.3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug broken, incorrect, or confusing behavior fixed-pls-verify fix is linked, bug author to confirm fix P3 Priority 3 RIoT Relates to integration with cloud providers, hypervisors, API-based services, etc. Salt-Cloud severity-high 2nd top severity, seen by most users, causes major problems
Projects
None yet
Development

No branches or pull requests