Repository navigation
Closed
Labels
Description
Description
% env HOMEBREW_PREFIX=/opt/homebrew salt-ssh 'strida-prod' test.ping
[ERROR ] An un-handled exception was caught by salt's global exception handler:
TypeError: cannot pickle '_thread._local' object
Traceback (most recent call last):
File "/opt/homebrew/bin/salt-ssh", line 8, in <module>
sys.exit(salt_ssh())
File "/opt/homebrew/Cellar/salt/3004/libexec/lib/python3.10/site-packages/salt/scripts.py", line 459, in salt_ssh
client.run()
File "/opt/homebrew/Cellar/salt/3004/libexec/lib/python3.10/site-packages/salt/cli/ssh.py", line 22, in run
ssh.run()
File "/opt/homebrew/Cellar/salt/3004/libexec/lib/python3.10/site-packages/salt/client/ssh/__init__.py", line 787, in run
for ret in self.handle_ssh():
File "/opt/homebrew/Cellar/salt/3004/libexec/lib/python3.10/site-packages/salt/client/ssh/__init__.py", line 606, in handle_ssh
routine.start()
File "/opt/homebrew/Cellar/python@3.10/3.10.1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
File "/opt/homebrew/Cellar/python@3.10/3.10.1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/opt/homebrew/Cellar/python@3.10/3.10.1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/context.py", line 284, in _Popen
return Popen(process_obj)
File "/opt/homebrew/Cellar/python@3.10/3.10.1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/popen_spawn_posix.py", line 32, in __init__
super().__init__(process_obj)
File "/opt/homebrew/Cellar/python@3.10/3.10.1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/popen_fork.py", line 19, in __init__
self._launch(process_obj)
File "/opt/homebrew/Cellar/python@3.10/3.10.1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/popen_spawn_posix.py", line 47, in _launch
reduction.dump(process_obj, fp)
File "/opt/homebrew/Cellar/python@3.10/3.10.1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle '_thread._local' object
Traceback (most recent call last):
File "/opt/homebrew/bin/salt-ssh", line 8, in <module>
sys.exit(salt_ssh())
File "/opt/homebrew/Cellar/salt/3004/libexec/lib/python3.10/site-packages/salt/scripts.py", line 459, in salt_ssh
client.run()
File "/opt/homebrew/Cellar/salt/3004/libexec/lib/python3.10/site-packages/salt/cli/ssh.py", line 22, in run
ssh.run()
File "/opt/homebrew/Cellar/salt/3004/libexec/lib/python3.10/site-packages/salt/client/ssh/__init__.py", line 787, in run
for ret in self.handle_ssh():
File "/opt/homebrew/Cellar/salt/3004/libexec/lib/python3.10/site-packages/salt/client/ssh/__init__.py", line 606, in handle_ssh
routine.start()
File "/opt/homebrew/Cellar/python@3.10/3.10.1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
File "/opt/homebrew/Cellar/python@3.10/3.10.1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/opt/homebrew/Cellar/python@3.10/3.10.1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/context.py", line 284, in _Popen
return Popen(process_obj)
File "/opt/homebrew/Cellar/python@3.10/3.10.1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/popen_spawn_posix.py", line 32, in __init__
super().__init__(process_obj)
File "/opt/homebrew/Cellar/python@3.10/3.10.1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/popen_fork.py", line 19, in __init__
self._launch(process_obj)
File "/opt/homebrew/Cellar/python@3.10/3.10.1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/popen_spawn_posix.py", line 47, in _launch
reduction.dump(process_obj, fp)
File "/opt/homebrew/Cellar/python@3.10/3.10.1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle '_thread._local' object
Setup
brew install saltstack on macOS Monterey
Saltfile
salt-ssh:
config_dir: ./salt_root
max_procs: 30
wipe_ssh: true
log_file: ./salt_root/log_ssh
ssh_log_file: ./salt_root/log_ssh
salt_root/roster
strida-prod:
host: 192.168.20.1
user: root
priv: ~/.ssh/id_ed25519
Steps to Reproduce the behavior
Attempt to use salt-ssh to connect to a remote host
Expected behavior
salt-ssh to run the test.ping command to validate a connection to the remote host.
Screenshots
See stack trace above.
Versions Report
salt --versions-report
(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)% env HOMEBREW_PREFIX=/opt/homebrew salt --versions-report
Salt Version:
Salt: 3004
Dependency Versions:
cffi: 1.14.6
cherrypy: unknown
dateutil: 2.8.0
docker-py: Not Installed
gitdb: 4.0.5
gitpython: 3.1.12
Jinja2: 2.11.3
libgit2: 1.3.0
M2Crypto: Not Installed
Mako: 1.1.4
msgpack: 1.0.2
msgpack-pure: Not Installed
mysql-python: Not Installed
pycparser: 2.19
pycrypto: 3.11.0
pycryptodome: 3.9.8
pygit2: 1.7.0
Python: 3.10.1 (main, Dec 6 2021, 22:18:13) [Clang 13.0.0 (clang-1300.0.29.3)]
python-gnupg: 0.4.4
PyYAML: 5.4.1
PyZMQ: 21.0.2
smmap: 3.0.2
timelib: 0.2.4
Tornado: 4.5.3
ZMQ: 4.3.4
System Versions:
dist: darwin 21.2.0
locale: utf-8
machine: arm64
release: 21.2.0
system: Darwin
version: 12.1 arm64
Reacted by Timid Robot Zehta, Michael Warkentin and Adam S.
Activity
lubyou commentedon Jan 31, 2022
Hi,
I ran into the same issue.
I installed salt-ssh 3004 into a venv.
This seems broken on when using python 3.8, 3.9 or 3.10 from brew.
If I install salt-ssh from pip in a 3.7 venv, it seems to work.
Working:
lubyou commentedon Feb 1, 2022
Looks like the issue is due to the start method of the multiprocessing method being changed to spawn.
When set to fork, the issue does not occur.
CodingMinds commentedon Apr 16, 2022
For others coming by via Google:
Workaround to get it running with the latest homebrew:
Modified
/opt/homebrew/opt/salt/libexec/lib/python3.10/site-packages/salt/client/ssh/__init__.pyand addedmultiprocessing.set_start_method("fork")as line 204.At least
salt-ssh $HOSTNAME test.pingseems to work now.johnkeates commentedon Aug 19, 2022
Can confirm that this is still an issue on the latest OS, Python and salt-ssh packages as well as the fix still being switching the method to fork.
garethgreenaway commentedon Sep 27, 2022
Anyone seeing this issue, can you please upgrade to the latest Salt release (3005) and see if the issue still occurs. Thanks!
johnkeates commentedon Sep 27, 2022
This was solved on 3005 for me.
garethgreenaway commentedon Oct 4, 2022
@johnkeates Thanks for the update. Since this one was fixed in the 3005 release I'll go ahead and close this one out, if the issue persists please feel free to open a new issue or request that this issue be reopened. Thanks!