Skip to content

[BUG] RuntimeError: dictionary changed size during iteration in apply_sdb #58819

New issue
New issue
Open
Open
[BUG] RuntimeError: dictionary changed size during iteration in apply_sdb#58819
Labels
Salt-APIbugbroken, incorrect, or confusing behaviorseverity-medium3rd level, incorrect or bad functionality, confusing and lacks a work around
Milestone
Approved
@ggiesen

Description

@ggiesen
ggiesen
opened on Oct 26, 2020

Description
I receive the following error when sending reactor events to run salt-cloud commands. I believe it occurs when I fire an event to create a VM, and then before the VM is fnished being created, fire another event to create another VM.

[DEBUG   ] dictionary changed size during iteration
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/salt/utils/process.py", line 491, in _thread_target
    func(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/salt/client/mixins.py", line 306, in low
    self.mminion  # pylint: disable=W0104
  File "/usr/lib/python3.6/site-packages/salt/client/mixins.py", line 265, in mminion
    self.opts, states=False, rend=False
  File "/usr/lib/python3.6/site-packages/salt/minion.py", line 968, in __init__
    opts["conf_file"], ignore_config_errors=ignore_config_errors, role="master"
  File "/usr/lib/python3.6/site-packages/salt/config/__init__.py", line 2190, in minion_config
    apply_sdb(opts)
  File "/usr/lib/python3.6/site-packages/salt/config/__init__.py", line 2346, in apply_sdb
    for key, value in six.iteritems(sdb_opts):
RuntimeError: dictionary changed size during iteration

It then seems to interrupt creation of both VMs. I'm using the salt-cloud vmware provider

Setup

  1. Set up an YAML SDB using GPG which contains the credentials for your salt-cloud provider

  2. Set up a cloud provider in /etc/salt/cloud.providers.d:

YOR:
  driver: vmware
  user: sdb://secret/vsphere:username
  password: sdb://secret/vsphere:password
  url: 'sphere.host.example.com'
  protocol: 'https'
  port: 443

Steps to Reproduce the behavior
(Include debug logs if possible and relevant)

  1. Call webhook to Salt-API to create events to create two VMs, firing the 2nd one before the first one is created.
[DEBUG   ] Gathering reactors for tag salt/netapi/hook/netbox/virtualization/virtual-machines
[DEBUG   ] Compiling reactions for tag salt/netapi/hook/netbox/virtualization/virtual-machines
[DEBUG   ] Re-using gitfs object for process 192406
[DEBUG   ] Re-using gitfs object for process 192406
[DEBUG   ] Re-using gitfs object for process 192406
[DEBUG   ] In saltenv 'base', looking at rel_path 'reactor/vmware_create.sls' to resolve 'salt://reactor/vmware_create.sls'
[DEBUG   ] In saltenv 'base', ** considering ** path '/var/cache/salt/master/files/base/reactor/vmware_create.sls' to resolve 'salt://reactor/vmware_create.sls'
[DEBUG   ] Fetching file from saltenv 'base', ** attempting ** 'salt://reactor/vmware_create.sls'
[DEBUG   ] No dest file found
[DEBUG   ] Re-using gitfs object for process 192406
[DEBUG   ] Re-using gitfs object for process 192406
[DEBUG   ] Re-using gitfs object for process 192406
[DEBUG   ] Re-using gitfs object for process 192406
[INFO    ] Fetching file from saltenv 'base', ** done ** 'reactor/vmware_create.sls'
[DEBUG   ] compile template: /var/cache/salt/master/files/base/reactor/vmware_create.sls
[DEBUG   ] LazyLoaded yaml.get
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] LazyLoaded gpg.render
[DEBUG   ] LazyLoaded config.get
[DEBUG   ] Reading GPG keys from: /etc/salt/gpgkeys
[DEBUG   ] LazyLoaded yaml.get
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] LazyLoaded gpg.render
[DEBUG   ] LazyLoaded config.get
[DEBUG   ] Reading GPG keys from: /etc/salt/gpgkeys
{'new_compute_instance': {'runner.cloud.create': [{'args': [{'provider': 'MY'},
                                                            {'instances': 'uycustmr-mon01'},
                                                            {'cluster': 'MY_Cloud'},
                                                            {'datacenter': 'MY'},
                                                            {'datastore': 'syortin01-T1'},
                                                            {'folder': 'CL '
                                                                       'Client '
                                                                       'Monitoring'},
                                                            {'hardware_version': 13},
                                                            {'image': 'oracleLinux7_64Guest'},
                                                            {'num_cpus': 3},
                                                            {'memory': '6144MB'},
                                                            {'devices': {'cd': {'CD/DVD drive 1': {'controller': 'IDE '
                                                                                                                 '0'}},
                                                                         'disk': {'Hard disk 1': {'controller': 'SCSI '
                                                                                                                'controller '
                                                                                                                '0',
                                                                                                  'size': 90,
                                                                                                  'thin_provision': True}},
                                                                         'ide': {'IDE 0': {}},
                                                                         'network': {'Network adapter 1': {'adapter_type': 'vmxnet3',
                                                                                                           'gateway': ['192.0.2.129'],
                                                                                                           'ip': '192.0.2.174',
                                                                                                           'name': 'pgPROVID-THE120%2fMon%2fMY_DMZ_Isolated',
                                                                                                           'subnet_mask': '255.255.255.128',
                                                                                                           'switch_type': 'distributed'},
                                                                                     'Network adapter 2': {'adapter_type': 'vmxnet3',
                                                                                                           'ip': '192.0.2.116',
                                                                                                           'name': 'pgCUSTMR-THE10%2fMon%2fMY',
                                                                                                           'subnet_mask': '255.255.255.240',
                                                                                                           'switch_type': 'distributed'}},
                                                                         'scsi': {'SCSI controller 0': {'type': 'paravirtual'}}}},
                                                            {'domain': 'example.com'},
                                                            {'dns_servers': ['198.51.100.29',
                                                                             '203.0.113.138']},
                                                            {'power_on': False},
                                                            {'annotation': 'Created '
                                                                           'by '
                                                                           'Salt-Cloud'},
                                                            {'deploy': False},
                                                            {'customization': True},
                                                            {'ssh_username': 'root'},
                                                            {'password': 'root'},
                                                            {'minion': {'master': '192.0.2.68'}}]}]}}
[PROFILE ] Time (in seconds) to render '/var/cache/salt/master/files/base/reactor/vmware_create.sls' using 'py' renderer: 1.0254533290863037
[DEBUG   ] ThreadPool executing func: <bound method SyncClientMixin.low of <salt.runner.RunnerClient object at 0x7f10101359e8>> with args=('cloud.create', {'name': 'new_compute_instance', '__sls__': '/var/cache/salt/master/files/base/reactor/vmware_create.sls', '__id__': 'new_compute_instance', 'order': 1, '__user__': 'Reactor', '__state__': 'runner', 'arg': (), 'kwarg': {'provider': 'MY', 'instances': 'uycustmr-mon01', 'cluster': 'MY_Cloud', 'datacenter': 'MY', 'datastore': 'syortin01-T1', 'folder': 'CL Client Monitoring', 
'hardware_version': 13, 'image': 'oracleLinux7_64Guest', 'num_cpus': 3, 'memory': '6144MB', 'devices': {'ide': {'IDE 0': {}}, 'cd': {'CD/DVD drive 1': {'controller': 'IDE 0'}}, 'scsi': {'SCSI controller 0': {'type': 'paravirtual'}}, 'disk': {'Hard disk 1': {'controller': 'SCSI controller 0', 'size': 90, 'thin_provision': True}}, 'network': {'Network adapter 1': {'adapter_type': 'vmxnet3', 'name': 'pgPROVID-THE120%2fMon%2fMY_DMZ_Isolated', 'switch_type': 'distributed', 'ip': '192.0.2.174', 'subnet_mask': '255.255.255.128', 'gateway': ['192.0.2.129']}, 'Network adapter 2': {'adapter_type': 'vmxnet3', 'name': 'pgCUSTMR-THE10%2fMon%2fMY', 'switch_type': 'distributed', 'ip': '192.0.2.116', 'subnet_mask': '255.255.255.240'}}}, 'domain': 'clinfra.internal', 'dns_servers': ['198.51.100.29', 
'203.0.113.138'], 'power_on': False, 'annotation': 'Created by Salt-Cloud', 'deploy': False, 'customization': True, 'ssh_username': 'root', 'password': 'root', 'minion': {'master': '192.0.2.68'}}}) kwargs={}
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Gathering reactors for tag salt/netapi/hook/netbox/virtualization/virtual-machines
[DEBUG   ] Compiling reactions for tag salt/netapi/hook/netbox/virtualization/virtual-machines
[DEBUG   ] Re-using gitfs object for process 192406
[DEBUG   ] Including configuration from '/etc/salt/master.d/clmon.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/clmon.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/git_pillar.conf'
[DEBUG   ] Re-using gitfs object for process 192406
[DEBUG   ] Reading configuration from /etc/salt/master.d/git_pillar.conf
[DEBUG   ] Re-using gitfs object for process 192406
[DEBUG   ] Including configuration from '/etc/salt/master.d/gitfs.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/gitfs.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/netbox.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/netbox.conf
[DEBUG   ] In saltenv 'base', looking at rel_path 'reactor/vmware_create.sls' to resolve 'salt://reactor/vmware_create.sls'
[DEBUG   ] In saltenv 'base', ** considering ** path '/var/cache/salt/master/files/base/reactor/vmware_create.sls' to resolve 'salt://reactor/vmware_create.sls'
[DEBUG   ] Including configuration from '/etc/salt/master.d/reactor.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/reactor.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/saltgui.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/saltgui.conf
[DEBUG   ] Fetching file from saltenv 'base', ** attempting ** 'salt://reactor/vmware_create.sls'
[DEBUG   ] No dest file found
[DEBUG   ] Re-using gitfs object for process 192406
[DEBUG   ] Including configuration from '/etc/salt/master.d/secret.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/secret.conf
[DEBUG   ] Guessing ID. The id can be explicitly set in /etc/salt/minion
[DEBUG   ] Re-using gitfs object for process 192406
[DEBUG   ] Re-using gitfs object for process 192406
[DEBUG   ] Re-using gitfs object for process 192406
[INFO    ] Fetching file from saltenv 'base', ** done ** 'reactor/vmware_create.sls'
[DEBUG   ] compile template: /var/cache/salt/master/files/base/reactor/vmware_create.sls
[DEBUG   ] LazyLoaded yaml.get
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] Found minion id from generate_minion_id(): master.example.com
[DEBUG   ] LazyLoaded gpg.render
[DEBUG   ] LazyLoaded config.get
[DEBUG   ] Reading GPG keys from: /etc/salt/gpgkeys
<truncated>
[DEBUG   ] LazyLoaded yaml.get
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] LazyLoaded gpg.render
[DEBUG   ] LazyLoaded config.get
[DEBUG   ] Reading GPG keys from: /etc/salt/gpgkeys
{'new_compute_instance': {'runner.cloud.create': [{'args': [{'provider': 'MY'},
                                                            {'instances': 'uycustmr-mon02'},
                                                            {'cluster': 'MY_Cloud'},
                                                            {'datacenter': 'MY'},
                                                            {'datastore': 'syortin01-T1'},
                                                            {'folder': 'CL '
                                                                       'Client '
                                                                       'Monitoring'},
                                                            {'hardware_version': 13},
                                                            {'image': 'oracleLinux7_64Guest'},
                                                            {'num_cpus': 3},
                                                            {'memory': '6144MB'},
                                                            {'devices': {'cd': {'CD/DVD drive 1': {'controller': 'IDE '
                                                                                                                 '0'}},
                                                                         'disk': {'Hard disk 1': {'controller': 'SCSI '
                                                                                                                'controller '
                                                                                                                '0',
                                                                                                  'size': 90,
                                                                                                  'thin_provision': True}},
                                                                         'ide': {'IDE 0': {}},
                                                                         'network': {'Network adapter 1': {'adapter_type': 'vmxnet3',
                                                                                                           'gateway': ['192.0.2.129'],
                                                                                                           'ip': '192.0.2.176',
                                                                                                           'name': 'pgPROVID-THE120%2fMon%2fMY_DMZ_Isolated',
                                                                                                           'subnet_mask': '255.255.255.128',
                                                                                                           'switch_type': 'distributed'},
                                                                                     'Network adapter 2': {'adapter_type': 'vmxnet3',
                                                                                                           'ip': '192.0.2.116',
                                                                                                           'name': 'pgCUSTMR-THE11%2fMon%2fMY',
                                                                                                           'subnet_mask': '255.255.255.240',
                                                                                                           'switch_type': 'distributed'}},
                                                                         'scsi': {'SCSI controller 0': {'type': 'paravirtual'}}}},
                                                            {'domain': 'example.com'},
                                                            {'dns_servers': ['198.51.100.29',
                                                                             '203.0.113.138']},
                                                            {'power_on': False},
                                                            {'annotation': 'Created '
                                                                           'by '
                                                                           'Salt-Cloud'},
                                                            {'deploy': False},
                                                            {'customization': True},
                                                            {'ssh_username': 'root'},
                                                            {'password': 'root'},
                                                            {'minion': {'master': '192.0.2.68'}}]}]}}
[PROFILE ] Time (in seconds) to render '/var/cache/salt/master/files/base/reactor/vmware_create.sls' using 'py' renderer: 1.093775987625122
[DEBUG   ] ThreadPool executing func: <bound method SyncClientMixin.low of <salt.runner.RunnerClient object at 0x7f10101359e8>> with args=('cloud.create', {'name': 'new_compute_instance', '__sls__': '/var/cache/salt/master/files/base/reactor/vmware_create.sls', '__id__': 'new_compute_instance', 'order': 1, '__user__': 'Reactor', '__state__': 'runner', 'arg': (), 'kwarg': {'provider': 'MY', 'instances': 'uycustmr-mon02', 'cluster': 'MY_Cloud', 'datacenter': 'MY', 'datastore': 'syortin01-T1', 'folder': 'CL Client Monitoring', 'hardware_version': 13, 'image': 'oracleLinux7_64Guest', 'num_cpus': 3, 'memory': '6144MB', 'devices': {'ide': {'IDE 0': {}}, 'cd': {'CD/DVD drive 1': {'controller': 'IDE 0'}}, 'scsi': {'SCSI controller 0': {'type': 'paravirtual'}}, 'disk': {'Hard disk 1': {'controller': 'SCSI controller 0', 'size': 90, 'thin_provision': True}}, 'network': {'Network adapter 1': {'adapter_type': 'vmxnet3', 'name': 'pgPROVID-THE120%2fMon%2fMY_DMZ_Isolated', 'switch_type': 'distributed', 'ip': '192.0.2.176', 'subnet_mask': '255.255.255.128', 'gateway': ['192.0.2.129']}, 'Network adapter 2': {'adapter_type': 'vmxnet3', 'name': 'pgCUSTMR-THE11%2fMon%2fMY', 'switch_type': 'distributed', 'ip': '192.0.2.116', 'subnet_mask': '255.255.255.240'}}}, 'domain': 'clinfra.internal', 'dns_servers': ['198.51.100.29', '203.0.113.138'], 'power_on': False, 'annotation': 'Created by Salt-Cloud', 'deploy': False, 'customization': True, 'ssh_username': 'root', 'password': 'root', 'minion': {'master': '192.0.2.68'}}}) kwargs={}
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/master.d/clmon.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/clmon.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/git_pillar.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/git_pillar.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/gitfs.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/gitfs.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/netbox.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/netbox.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/reactor.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/reactor.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/saltgui.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/saltgui.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/secret.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/secret.conf
[DEBUG   ] Guessing ID. The id can be explicitly set in /etc/salt/minion
[DEBUG   ] Found minion id from generate_minion_id(): master.example.com
[DEBUG   ] LazyLoaded yaml.get
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] LazyLoaded gpg.render
[DEBUG   ] LazyLoaded config.get
[DEBUG   ] Reading GPG keys from: /etc/salt/gpgkeys
<truncated>
[DEBUG   ] LazyLoaded yaml.get
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] LazyLoaded gpg.render
[DEBUG   ] LazyLoaded config.get
[DEBUG   ] Reading GPG keys from: /etc/salt/gpgkeys
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/master.d/clmon.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/clmon.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/git_pillar.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/git_pillar.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/gitfs.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/gitfs.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/netbox.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/netbox.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/reactor.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/reactor.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/saltgui.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/saltgui.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/secret.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/secret.conf
[DEBUG   ] Guessing ID. The id can be explicitly set in /etc/salt/minion
[DEBUG   ] Found minion id from generate_minion_id(): master.example.com
[DEBUG   ] LazyLoaded yaml.get
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] LazyLoaded gpg.render
[DEBUG   ] LazyLoaded config.get
[DEBUG   ] Reading GPG keys from: /etc/salt/gpgkeys
<truncated>
[DEBUG   ] LazyLoaded yaml.get
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] LazyLoaded gpg.render
[DEBUG   ] LazyLoaded config.get
[DEBUG   ] Reading GPG keys from: /etc/salt/gpgkeys
[DEBUG   ] Sending event: tag = salt/wheel/20201026224315510084/new; data = {'fun': 'wheel.config.values', 'jid': '20201026224315510084', 'tag': 'salt/wheel/20201026224315510084', 'user': 'ggiesen', '_stamp': '2020-10-26T22:43:15.510151'}
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Sending event: tag = salt/wheel/20201026224315510705/new; data = {'fun': 'wheel.config.values', 'jid': '20201026224315510705', 'user': 'UNKNOWN', 'fun_args': [], '_stamp': '2020-10-26T22:43:15.513624'}
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/master.d/clmon.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/clmon.conf
[DEBUG   ] Gathering reactors for tag salt/wheel/20201026224315510084/new
[DEBUG   ] Including configuration from '/etc/salt/master.d/git_pillar.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/git_pillar.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/gitfs.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/gitfs.conf
[DEBUG   ] LazyLoaded yaml.get
[DEBUG   ] Including configuration from '/etc/salt/master.d/netbox.conf'
[DEBUG   ] Gathering reactors for tag salt/wheel/20201026224315510705/new
[DEBUG   ] Reading configuration from /etc/salt/master.d/netbox.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/reactor.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/reactor.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/saltgui.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/saltgui.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/secret.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/secret.conf
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] LazyLoaded gpg.render
[DEBUG   ] LazyLoaded config.get
[DEBUG   ] Reading GPG keys from: /etc/salt/gpgkeys
<truncated>
[DEBUG   ] Reading GPG keys from: /etc/salt/gpgkeys
[DEBUG   ] LazyLoaded yaml.get
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] LazyLoaded gpg.render
[DEBUG   ] LazyLoaded config.get
[DEBUG   ] Reading GPG keys from: /etc/salt/gpgkeys
[DEBUG   ] dictionary changed size during iteration
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/salt/utils/process.py", line 491, in _thread_target
    func(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/salt/client/mixins.py", line 306, in low
    self.mminion  # pylint: disable=W0104
  File "/usr/lib/python3.6/site-packages/salt/client/mixins.py", line 265, in mminion
    self.opts, states=False, rend=False
  File "/usr/lib/python3.6/site-packages/salt/minion.py", line 968, in __init__
    opts["conf_file"], ignore_config_errors=ignore_config_errors, role="master"
  File "/usr/lib/python3.6/site-packages/salt/config/__init__.py", line 2190, in minion_config
    apply_sdb(opts)
  File "/usr/lib/python3.6/site-packages/salt/config/__init__.py", line 2346, in apply_sdb
    for key, value in six.iteritems(sdb_opts):
RuntimeError: dictionary changed size during iteration
[DEBUG   ] LazyLoaded yaml.get
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] LazyLoaded gpg.render
[DEBUG   ] LazyLoaded config.get
[DEBUG   ] Reading GPG keys from: /etc/salt/gpgkeys
<truncated>
[DEBUG   ] LazyLoaded yaml.get
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] LazyLoaded gpg.render
[DEBUG   ] LazyLoaded config.get
[DEBUG   ] Reading GPG keys from: /etc/salt/gpgkeys
[DEBUG   ] dictionary changed size during iteration
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/salt/utils/process.py", line 491, in _thread_target
    func(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/salt/client/mixins.py", line 306, in low
    self.mminion  # pylint: disable=W0104
  File "/usr/lib/python3.6/site-packages/salt/client/mixins.py", line 265, in mminion
    self.opts, states=False, rend=False
  File "/usr/lib/python3.6/site-packages/salt/minion.py", line 968, in __init__
    opts["conf_file"], ignore_config_errors=ignore_config_errors, role="master"
  File "/usr/lib/python3.6/site-packages/salt/config/__init__.py", line 2190, in minion_config
    apply_sdb(opts)
  File "/usr/lib/python3.6/site-packages/salt/config/__init__.py", line 2346, in apply_sdb
    for key, value in six.iteritems(sdb_opts):
RuntimeError: dictionary changed size during iteration
[DEBUG   ] LazyLoaded yaml.get
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] LazyLoaded gpg.render
[DEBUG   ] LazyLoaded config.get

Expected behavior
To VMs to be created

Versions Report

salt --versions-report (Provided by running salt --versions-report. Please also mention any differences in master/minion versions.) 
salt --versions-report
Salt Version:
           Salt: 3001.1

Dependency Versions:
           cffi: 1.14.0
       cherrypy: unknown
       dateutil: 2.6.1
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
         Jinja2: 2.11.2
        libgit2: 0.26.8
       M2Crypto: 0.35.2
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.6.2
   mysql-python: Not Installed
      pycparser: 2.20
       pycrypto: Not Installed
   pycryptodome: Not Installed
         pygit2: 0.26.4
         Python: 3.6.8 (default, Apr 16 2020, 01:36:27)
   python-gnupg: Not Installed
         PyYAML: 5.3.1
          PyZMQ: 19.0.0
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.3.3

System Versions:
           dist: centos 8 Core
         locale: UTF-8
        machine: x86_64
        release: 4.18.0-193.19.1.el8_2.x86_64
         system: Linux
        version: CentOS Linux 8 Core

Metadata

Metadata

Assignees

No one assigned

    Labels

    Salt-APIbugbroken, incorrect, or confusing behaviorseverity-medium3rd level, incorrect or bad functionality, confusing and lacks a work around

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions