Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TypeError: cannot pickle '_thread.lock' object #236

Closed
M0dred opened this issue May 11, 2020 · 8 comments
Closed

TypeError: cannot pickle '_thread.lock' object #236

M0dred opened this issue May 11, 2020 · 8 comments

Comments

@M0dred
Copy link

M0dred commented May 11, 2020

Traceback (most recent call last):
File "sublist3r.py", line 1013, in
interactive()
File "sublist3r.py", line 1010, in interactive
res = main(domain, threads, savefile, ports, silent=False, verbose=verbose, enable_bruteforce=enable_bruteforce, engines=engines)
File "sublist3r.py", line 952, in main
enum.start()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/context.py", line 283, in _Popen
return Popen(process_obj)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in init
super().init(process_obj)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_fork.py", line 19, in init
self._launch(process_obj)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch
reduction.dump(process_obj, fp)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle '_thread.lock' object

Python 3.8.2

@rebelnn
Copy link

rebelnn commented May 16, 2020

Getting the same as OP -- Python 3.8.0

@amustaque97
Copy link
Contributor

amustaque97 commented Jun 6, 2020

Hey @M0dred & @rebelnn, sorry for the late reply. I'm unable to reproduce the issue, I suggest you update python to the latest version as there is no issue with the thread locking.
Screenshot from 2020-06-07 00-16-22

If the issue is resolved please mark it as close. Thank You.

@hassankhan98
Copy link

hassankhan98 commented Jul 2, 2020

my python version is 3.8.3 but still not working?

@christopherowen
Copy link

christopherowen commented Jul 15, 2020

Python 3.8.3 (the latest available in macOS homebrew) experiences this.

% python3 --version
Python 3.8.3
sublist3r % python3 sublist3r.py -d starlink.com

(...)

Traceback (most recent call last):
  File "sublist3r.py", line 1013, in <module>
    interactive()
  File "sublist3r.py", line 1010, in interactive
    res = main(domain, threads, savefile, ports, silent=False, verbose=verbose, enable_bruteforce=enable_bruteforce, engines=engines)
  File "sublist3r.py", line 952, in main
    enum.start()
  File "/usr/local/Cellar/python@3.8/3.8.3_2/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/usr/local/Cellar/python@3.8/3.8.3_2/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/usr/local/Cellar/python@3.8/3.8.3_2/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/context.py", line 283, in _Popen
    return Popen(process_obj)
  File "/usr/local/Cellar/python@3.8/3.8.3_2/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/usr/local/Cellar/python@3.8/3.8.3_2/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/usr/local/Cellar/python@3.8/3.8.3_2/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/usr/local/Cellar/python@3.8/3.8.3_2/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle '_thread.lock' object

@christopherowen
Copy link

christopherowen commented Jul 15, 2020

GoogleCloudPlatform/gsutil#961 (comment)

Tracking this down, this error comes from a change in Python 3.8 in the multiprocessing library:

Changed in version 3.8: On macOS, the spawn start method is now the default. The fork start method should be considered unsafe as it can lead to crashes of the subprocess. See bpo-33725.

Spawn is being run for those using MacOs and Python 3.8+ by default since nothing is explicitly set either through get_context or set_start_method.

@allekmott
Copy link
Contributor

allekmott commented Jul 15, 2020

I've created #242 to address this.

@christopherowen
Copy link

christopherowen commented Jul 16, 2020

This is a much better implementation than my hack to change the processing context to fork.

Also, there's another PR lying around that changes the multiprocessing to use a pool. It may be interesting too.

@aboul3la
Copy link
Owner

aboul3la commented Jul 29, 2020

Thanks guys for bringing this to my attention, I just merged #242 that should fix this issue.

wopperer added a commit to wopperer/Turbolist3r that referenced this issue Jun 6, 2021
fleetcaptain added a commit to fleetcaptain/Turbolist3r that referenced this issue Jun 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants