Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

salt-cloud map parallel provisioning -P option failed on 2015.8.5 #31632

Closed
zieba88 opened this issue Mar 2, 2016 · 3 comments
Closed

salt-cloud map parallel provisioning -P option failed on 2015.8.5 #31632

zieba88 opened this issue Mar 2, 2016 · 3 comments
Labels
Bug broken, incorrect, or confusing behavior fixed-pls-verify fix is linked, bug author to confirm fix P2 Priority 2 RIoT Relates to integration with cloud providers, hypervisors, API-based services, etc. Salt-Cloud severity-medium 3rd level, incorrect or bad functionality, confusing and lacks a work around
Milestone
Approved

Comments

@zieba88
Copy link

zieba88 commented Mar 2, 2016

When running salt-cloud -m /etc/salt/cloud.maps.d/map.conf -P -l trace, it fails. Only happens when trying to run with the -P (parallel) option.

cent6_2:
  provider: vcenter02
  clonefrom: CentOS6.6_64bit
  num_cpus: 2
  memory: 2GB
  domain: domain.com
  dns_servers:
    - 192.168.14.11
    - 192.168.14.12
  resourcepool: Resources
  cluster: qa
  datastore: datastore1
  datacenter: dlsb
  host: hostname
  power_on: True
  deploy: True
  ssh_username: root
  password: passwd
  minion:
    master: 192.168.40.43
  grains:
    location: ny
    env: qa
    role: web


cloud.maps.d]# cat map.conf 
cent6_2:
  - dlsbweb01:
      devices:
        network:
          Network adapter 1:
            name: Kickstart Network
            switch_type: standard
            ip: 192.168.40.248
            gateway: [192.168.40.1]
            subnet_mask: 255.255.255.0
  - dlsbweb02:
      devices:
        network:
          Network adapter 1:
            name: Kickstart Network
            switch_type: standard
            ip: 192.168.40.249
            gateway: [192.168.40.1]
            subnet_mask: 255.255.255.0


[DEBUG   ] Connection to 192.168.40.249 closed.
[DEBUG   ]  *  INFO: Salt installed!
[DEBUG   ] Connection to 192.168.40.248 closed.
[DEBUG   ]  *  INFO: Salt installed!
[DEBUG   ] Executed command "/tmp/.saltcloud-73e57e99-789c-4379-bc47-6410b7a78c74/deploy.sh -c '/tmp/.saltcloud-73e57e99-789c-4379-bc47-6410b7a78c74'"
[DEBUG   ] SSH command: "ssh -t -t -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oControlPath=none -p 22 root@192.168.40.249 rm -f '/tmp/.saltcloud-73e57e99-789c-4379-bc47-6410b7a78c74/deploy.sh'"
[DEBUG   ] Executed command "/tmp/.saltcloud-8f078916-9324-44ba-b4a9-c0db91995654/deploy.sh -c '/tmp/.saltcloud-8f078916-9324-44ba-b4a9-c0db91995654'"
[DEBUG   ] SSH command: "ssh -t -t -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oControlPath=none -p 22 root@192.168.40.248 rm -f '/tmp/.saltcloud-8f078916-9324-44ba-b4a9-c0db91995654/deploy.sh'"
[DEBUG   ] Child Forked! PID: 15369  STDOUT_FD: 21  STDERR_FD: 23
[DEBUG   ] Child Forked! PID: 15370  STDOUT_FD: 21  STDERR_FD: 23
[DEBUG   ] Terminal Command: /bin/sh -c ssh -t -t -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oControlPath=none -p 22 root@192.168.40.248 'rm -f '"'"'/tmp/.saltcloud-8f078916-9324-44ba-b4a9-c0db91995654/deploy.sh'"'"''
[DEBUG   ] Terminal Command: /bin/sh -c ssh -t -t -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oControlPath=none -p 22 root@192.168.40.249 'rm -f '"'"'/tmp/.saltcloud-73e57e99-789c-4379-bc47-6410b7a78c74/deploy.sh'"'"''
[DEBUG   ] Warning: Permanently added '192.168.40.249' (RSA) to the list of known hosts.
[DEBUG   ] root@192.168.40.249's password:
[DEBUG   ] Warning: Permanently added '192.168.40.248' (RSA) to the list of known hosts.
[DEBUG   ] root@192.168.40.248's password:
[DEBUG   ] Connection to 192.168.40.249 closed.
[DEBUG   ] Connection to 192.168.40.248 closed.
[DEBUG   ] Removed /tmp/.saltcloud-73e57e99-789c-4379-bc47-6410b7a78c74/deploy.sh
[DEBUG   ] Removed /tmp/.saltcloud-8f078916-9324-44ba-b4a9-c0db91995654/deploy.sh
[DEBUG   ] SSH command: "ssh -t -t -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oControlPath=none -p 22 root@192.168.40.249 rm -f '/tmp/.saltcloud-73e57e99-789c-4379-bc47-6410b7a78c74/minion.pub'"
[DEBUG   ] SSH command: "ssh -t -t -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oControlPath=none -p 22 root@192.168.40.248 rm -f '/tmp/.saltcloud-8f078916-9324-44ba-b4a9-c0db91995654/minion.pub'"
[DEBUG   ] Child Forked! PID: 15372  STDOUT_FD: 21  STDERR_FD: 23
[DEBUG   ] Child Forked! PID: 15373  STDOUT_FD: 21  STDERR_FD: 23
[DEBUG   ] Terminal Command: /bin/sh -c ssh -t -t -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oControlPath=none -p 22 root@192.168.40.248 'rm -f '"'"'/tmp/.saltcloud-8f078916-9324-44ba-b4a9-c0db91995654/minion.pub'"'"''
[DEBUG   ] Terminal Command: /bin/sh -c ssh -t -t -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oControlPath=none -p 22 root@192.168.40.249 'rm -f '"'"'/tmp/.saltcloud-73e57e99-789c-4379-bc47-6410b7a78c74/minion.pub'"'"''
[DEBUG   ] Warning: Permanently added '192.168.40.249' (RSA) to the list of known hosts.
[DEBUG   ] Warning: Permanently added '192.168.40.248' (RSA) to the list of known hosts.
[DEBUG   ] root@192.168.40.249's password:
[DEBUG   ] root@192.168.40.248's password:
[DEBUG   ] Connection to 192.168.40.249 closed.
[DEBUG   ] Connection to 192.168.40.248 closed.
[DEBUG   ] Removed /tmp/.saltcloud-8f078916-9324-44ba-b4a9-c0db91995654/minion.pub
[DEBUG   ] Removed /tmp/.saltcloud-73e57e99-789c-4379-bc47-6410b7a78c74/minion.pub
[DEBUG   ] SSH command: "ssh -t -t -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oControlPath=none -p 22 root@192.168.40.248 rm -f '/tmp/.saltcloud-8f078916-9324-44ba-b4a9-c0db91995654/minion.pem'"
[DEBUG   ] SSH command: "ssh -t -t -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oControlPath=none -p 22 root@192.168.40.249 rm -f '/tmp/.saltcloud-73e57e99-789c-4379-bc47-6410b7a78c74/minion.pem'"
[DEBUG   ] Child Forked! PID: 15374  STDOUT_FD: 21  STDERR_FD: 23
[DEBUG   ] Child Forked! PID: 15375  STDOUT_FD: 21  STDERR_FD: 23
[DEBUG   ] Terminal Command: /bin/sh -c ssh -t -t -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oControlPath=none -p 22 root@192.168.40.248 'rm -f '"'"'/tmp/.saltcloud-8f078916-9324-44ba-b4a9-c0db91995654/minion.pem'"'"''
[DEBUG   ] Terminal Command: /bin/sh -c ssh -t -t -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oControlPath=none -p 22 root@192.168.40.249 'rm -f '"'"'/tmp/.saltcloud-73e57e99-789c-4379-bc47-6410b7a78c74/minion.pem'"'"''
[DEBUG   ] Warning: Permanently added '192.168.40.248' (RSA) to the list of known hosts.
[DEBUG   ] Warning: Permanently added '192.168.40.249' (RSA) to the list of known hosts.
[DEBUG   ] root@192.168.40.249's password:
[DEBUG   ] root@192.168.40.248's password:
[DEBUG   ] Connection to 192.168.40.249 closed.
[DEBUG   ] Connection to 192.168.40.248 closed.
[DEBUG   ] Removed /tmp/.saltcloud-8f078916-9324-44ba-b4a9-c0db91995654/minion.pem
[DEBUG   ] Removed /tmp/.saltcloud-73e57e99-789c-4379-bc47-6410b7a78c74/minion.pem
[DEBUG   ] SSH command: "ssh -t -t -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oControlPath=none -p 22 root@192.168.40.248 rm -f '/tmp/.saltcloud-8f078916-9324-44ba-b4a9-c0db91995654/grains'"
[DEBUG   ] SSH command: "ssh -t -t -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oControlPath=none -p 22 root@192.168.40.249 rm -f '/tmp/.saltcloud-73e57e99-789c-4379-bc47-6410b7a78c74/grains'"
[DEBUG   ] Child Forked! PID: 15376  STDOUT_FD: 21  STDERR_FD: 23
[DEBUG   ] Child Forked! PID: 15377  STDOUT_FD: 21  STDERR_FD: 23
[DEBUG   ] Terminal Command: /bin/sh -c ssh -t -t -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oControlPath=none -p 22 root@192.168.40.248 'rm -f '"'"'/tmp/.saltcloud-8f078916-9324-44ba-b4a9-c0db91995654/grains'"'"''
[DEBUG   ] Terminal Command: /bin/sh -c ssh -t -t -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oControlPath=none -p 22 root@192.168.40.249 'rm -f '"'"'/tmp/.saltcloud-73e57e99-789c-4379-bc47-6410b7a78c74/grains'"'"''
[DEBUG   ] Warning: Permanently added '192.168.40.249' (RSA) to the list of known hosts.
[DEBUG   ] root@192.168.40.249's password:
[DEBUG   ] Warning: Permanently added '192.168.40.248' (RSA) to the list of known hosts.
[DEBUG   ] root@192.168.40.248's password:
[DEBUG   ] Connection to 192.168.40.249 closed.
[DEBUG   ] Connection to 192.168.40.248 closed.
[DEBUG   ] Removed /tmp/.saltcloud-73e57e99-789c-4379-bc47-6410b7a78c74/grains
[DEBUG   ] Removed /tmp/.saltcloud-8f078916-9324-44ba-b4a9-c0db91995654/grains
[DEBUG   ] SSH command: "ssh -t -t -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oControlPath=none -p 22 root@192.168.40.249 rm -f '/tmp/.saltcloud-73e57e99-789c-4379-bc47-6410b7a78c74/minion'"
[DEBUG   ] SSH command: "ssh -t -t -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oControlPath=none -p 22 root@192.168.40.248 rm -f '/tmp/.saltcloud-8f078916-9324-44ba-b4a9-c0db91995654/minion'"
[DEBUG   ] Child Forked! PID: 15378  STDOUT_FD: 21  STDERR_FD: 23
[DEBUG   ] Child Forked! PID: 15379  STDOUT_FD: 21  STDERR_FD: 23
[DEBUG   ] Terminal Command: /bin/sh -c ssh -t -t -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oControlPath=none -p 22 root@192.168.40.248 'rm -f '"'"'/tmp/.saltcloud-8f078916-9324-44ba-b4a9-c0db91995654/minion'"'"''
[DEBUG   ] Terminal Command: /bin/sh -c ssh -t -t -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oControlPath=none -p 22 root@192.168.40.249 'rm -f '"'"'/tmp/.saltcloud-73e57e99-789c-4379-bc47-6410b7a78c74/minion'"'"''
[DEBUG   ] Warning: Permanently added '192.168.40.248' (RSA) to the list of known hosts.
[DEBUG   ] Warning: Permanently added '192.168.40.249' (RSA) to the list of known hosts.
[DEBUG   ] root@192.168.40.248's password:
[DEBUG   ] root@192.168.40.249's password:
[DEBUG   ] Connection to 192.168.40.248 closed.
[DEBUG   ] Connection to 192.168.40.249 closed.
[DEBUG   ] Removed /tmp/.saltcloud-73e57e99-789c-4379-bc47-6410b7a78c74/minion
[DEBUG   ] Removed /tmp/.saltcloud-8f078916-9324-44ba-b4a9-c0db91995654/minion
[DEBUG   ] MasterEvent PUB socket URI: ipc:///var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PUB socket URI: ipc:///var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: ipc:///var/run/salt/master/master_event_pull.ipc
[DEBUG   ] MasterEvent PULL socket URI: ipc:///var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Sending event - data = {'host': '192.168.40.249', 'name': 'dlsbweb02', '_stamp': '2016-03-02T18:19:36.067144', 'event': 'dlsbweb02 has been deployed at 192.168.40.249'}
[DEBUG   ] Sending event - data = {'host': '192.168.40.248', 'name': 'dlsbweb01', '_stamp': '2016-03-02T18:19:36.067528', 'event': 'dlsbweb01 has been deployed at 192.168.40.248'}
[INFO    ] Salt installed on dlsbweb02
[INFO    ] Salt installed on dlsbweb01
[DEBUG   ] MasterEvent PUB socket URI: ipc:///var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: ipc:///var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Sending event - data = {'profile': 'cent6_2', 'event': 'created instance', '_stamp': '2016-03-02T18:19:36.922794', 'name': 'dlsbweb01', 'provider': 'vcenter02:vmware'}
[DEBUG   ] MasterEvent PUB socket URI: ipc:///var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: ipc:///var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Sending event - data = {'profile': 'cent6_2', 'event': 'created instance', '_stamp': '2016-03-02T18:19:36.932306', 'name': 'dlsbweb02', 'provider': 'vcenter02:vmware'}
[ERROR   ] There was a query error: itervalues() takes exactly 1 argument (2 given)
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/salt/cloud/cli.py", line 339, in run
    ret = mapper.run_map(dmap)
  File "/usr/lib/python2.6/site-packages/salt/cloud/__init__.py", line 2211, in run_map
    lambda x: x['level']):
TypeError: itervalues() takes exactly 1 argument (2 given)

salt --versions-report
Salt Version:
           Salt: 2015.8.5

Dependency Versions:
         Jinja2: 2.2.1
       M2Crypto: 0.20.2
           Mako: Not Installed
         PyYAML: 3.11
          PyZMQ: 14.5.0
         Python: 2.6.6 (r266:84292, Jul 23 2015, 15:22:56)
           RAET: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.5
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 1.4.1
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
        libgit2: Not Installed
        libnacl: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
         pygit2: Not Installed
   python-gnupg: Not Installed
          smmap: Not Installed
        timelib: Not Installed

System Versions:
           dist: centos 6.7 Final
        machine: x86_64
        release: 2.6.32-504.el6.x86_64
         system: CentOS 6.7 Final
@patrickpilch
Copy link

Elaborating on the information provided, the issue occurs at

for key, val in six.itervalues(groupby(iter(dmap['create'])),
                                        lambda x: x['level']):

If I found the correct definition of six.itervalues (for python 2), it's defined as

def itervalues(d, **kw):
        return iter(d.itervalues(**kw))

six.itervalues is defined with one argument, but it's being passed two arguments, an itertools.groupby object and a lambda, therefore resulting in the TypeError complaining about argument amounts.

@jfindlay jfindlay added Bug broken, incorrect, or confusing behavior severity-medium 3rd level, incorrect or bad functionality, confusing and lacks a work around P2 Priority 2 Salt-Cloud RIoT Relates to integration with cloud providers, hypervisors, API-based services, etc. labels Mar 3, 2016
@jfindlay jfindlay added this to the Approved milestone Mar 3, 2016
@jfindlay
Copy link
Contributor

jfindlay commented Mar 3, 2016

@zieba88, thanks for reporting. @patrickpilch, you are welcome to submit a pull request to fix this.

cachedout pushed a commit to cachedout/salt that referenced this issue Apr 7, 2016
Fix salt-cloud paralell provisioning
127c082 
Closes saltstack#31632
@cachedout cachedout mentioned this issue Apr 7, 2016
Merged
@rallytime rallytime added the fixed-pls-verify fix is linked, bug author to confirm fix label Apr 7, 2016
@zieba88
Copy link
Author

zieba88 commented May 19, 2016

@jfindlay @rallytime @cachedout @patrickpilch

Verified the fix works. Thanks All!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug broken, incorrect, or confusing behavior fixed-pls-verify fix is linked, bug author to confirm fix P2 Priority 2 RIoT Relates to integration with cloud providers, hypervisors, API-based services, etc. Salt-Cloud severity-medium 3rd level, incorrect or bad functionality, confusing and lacks a work around
Projects
None yet
Milestone
Approved
Development

No branches or pull requests
4 participants
@jfindlay @patrickpilch @rallytime @zieba88