salt-cloud -m fails with softlayer

[xpender]

salt-cloud -m map.file fails if using softlayer as cloud provider. Map files contains two instances, one of them is created & running, the other one not. With no created instances it worked.

Output

[INFO    ] salt-cloud starting
[WARNING ] DEPRECATED: Mako will no longer be the default renderer for Salt Cloud maps in the Lithium release
[INFO    ] Applying map from 'sl.map'.
[INFO    ] POST https://api.softlayer.com/xmlrpc/v3.1/SoftLayer_Account
[INFO    ] Starting new HTTPS connection (1): api.softlayer.com
[ERROR   ] There was a query error: 'state'
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/salt/cloud/cli.py", line 347, in run
    dmap = mapper.map_data()
  File "/usr/lib/python2.7/dist-packages/salt/cloud/__init__.py", line 1407, in map_data
    matching = get_matching_by_name(name)
  File "/usr/lib/python2.7/dist-packages/salt/cloud/__init__.py", line 1396, in get_matching_by_name
    matches[driver] = details['state']
KeyError: 'state'

Version-Report

           Salt: 2014.1.13
         Python: 2.7.3 (default, Mar 13 2014, 11:03:55)
         Jinja2: 2.6
       M2Crypto: 0.21.1
 msgpack-python: 0.1.10
   msgpack-pure: Not Installed
       pycrypto: 2.6
         PyYAML: 3.10
          PyZMQ: 13.1.0
            ZMQ: 3.2.3

Patch which fixed it for me

--- softlayer.py.old    2014-11-04 08:06:54.000000000 -0600
+++ softlayer.py    2014-11-04 08:21:47.000000000 -0600
@@ -550,6 +550,8 @@
             ret[node]['public_ips'] = nodes[node]['primaryIpAddress']
         if 'primaryBackendIpAddress' in nodes[node]:
             ret[node]['private_ips'] = nodes[node]['primaryBackendIpAddress']
+        if 'status' in nodes[node]:
+            ret[node]['state'] = str(nodes[node]['status']['name'])
     return ret

I'm not sure if state gets evaluated by salt-cloud. I looked into the digital_ocean.py and there it also simply takes API response and casts to string: 'state': str(node['status'])

[basepi]

Hrm, I'm not familiar withe the softlayer code, but if that fixed it for you you should probably submit it as a pull request for better review.

Thanks for the report and the patch!

[basepi]

@techhat ping

[kremso]

@basepi @techhat Any update on this? This happened to me (on salt-cloud 2014.7.2 (Helium)) and it basically made salt-cloud useless as I can't spin any new servers.

[kremso]

And I can confirm that @techhat's patch fixes the problem.

[pnixon-kwicr]

Ran into this today as well on 2014.7.1

[robholmes]

Why is this still broken after all this time? @techhat's patch does not fix the issue.

I can confirm this is an issue on: salt-cloud 2014.7.4 (Helium)

[rallytime]

It looks to me like no one ever made an actual pull request to the salt code base with this change. I'll do that now.

[rallytime]

Can anyone give the pull request above a try? I only applied the patch suggested by @xpender.

[kremso]

@rallytime I just tried it. Running salt-cloud from develop branch exhibits the error. Running from the 2014.7 branch with the fix fails with:

Traceback (most recent call last):
  File "/usr/local/bin/salt-cloud", line 10, in <module>
    execfile(__file__)
  File "/home/kremso/projects/salt/scripts/salt-cloud", line 6, in <module>
    from salt.scripts import salt_cloud
  File "/home/kremso/projects/salt/salt/scripts.py", line 17, in <module>
    import salt.cli
  File "/home/kremso/projects/salt/salt/cli/__init__.py", line 13, in <module>
    import salt.cli.caller
ImportError: No module named cli.caller

I'm not a python programmer, so I might be doing something stupid. I just checked out the branch, ran pip install -e . and then ran salt-cloud normally. The same for the other branch.

[rallytime]

@kremso The cli.caller issue is actually a different bug. When you switch back and forth between the 2014.7 and the 2015.2 or develop branches, you'll see that error with any command, even a simple salt --versions-report on the 2014.7 branch. This bug, with its solutions and work-arounds, is discussed in #20924. It has to do with some refactoring done on the 2015.2 branch to the CLI that isn't present on 2014.7 and some wires are getting crossed for some reason.

Basically you'll need to wipe out your salt installation and then pip install again. Sorry for the troubles there. It's definitely annoying. :-|

[kremso]

@rallytime Thanks. I can confirm that the problem is fixed (at least for me).

[rallytime]

Awesome! Thanks very much for trying that out @xpender! Since this has been confirmed as fixed, I am going to close this.

Unfortunately, this fix did not make it into the upcoming 2014.7.5 release, as the pull request was made after the release was tagged. I don't know if we'll be doing another release on the 2014.7 branch or not, but if we do, this fix will be in 2014.7.6. Otherwise, the fix will definitely be in the upcoming 2015.2.0 release. Thanks!

[robholmes]

@rallytime and @xpender I've tried adding the 2 lines changed in your pull request into my /usr/lib/python2.7/dist-packages/salt/cloud/clouds/softlayer.py file, but I still get the There was a query error: 'state' error when running salt-cloud -m /etc/salt/cloud.maps.d/my-cloud.map

For my scenario I already have some servers up and running, and I've added another server into my map. Maybe I'm making assumptions, but I believed this would find the new server in the map file which doesn't exist and create it. Am I doing something wrong?