Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[funfuzz] Rewrite fork_join using concurrent.futures #139

Open
nth10sd opened this issue Oct 20, 2017 · 2 comments
Open

[funfuzz] Rewrite fork_join using concurrent.futures #139

nth10sd opened this issue Oct 20, 2017 · 2 comments
Assignees

Comments

@nth10sd
Copy link
Contributor

nth10sd commented Oct 20, 2017

After #135 landed, Windows is unable to spawn multiple processes when fuzzing. When I manually bypass the call to fork_join, fuzzing works as expected with a single thread.

We can either:

  • Fix the current fork_join code
  • Rewrite using concurrent.futures, and we can import the futures package to make it work on Python 2.7.

I'm leaning towards the second option, after getting #138 in place. Edit: it may be more important to unbreak this first.

@nth10sd
Copy link
Contributor Author

nth10sd commented Oct 20, 2017

The output with stack (when fixated to one process) is:

=== Waiting for child #0 (8448) to finish... ===
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "c:\mozilla-build\python\lib\multiprocessing\forking.py", line 380, in main
    prepare(preparation_data)
  File "c:\mozilla-build\python\lib\multiprocessing\forking.py", line 509, in prepare
    '__parents_main__', file, path_name, etc
  File "c:\mozilla-build\python\lib\site-packages\funfuzz\bot.py", line 23, in <module>
    from .js import build_options
ValueError: Attempted relative import in non-package
=== Child process #0 exited with code 1 ===

Apparently this is because multiprocessing in Python 2 does not play nice with explicit relative imports (whereas Python 3.4+ seems fine). It had been added in Python 2.6 which is now end-of-life.

In conclusion, our custom multiple process fuzzing implementation from Sep 2012 (fork_join) does not work on Windows with the new module packaging system.

@nth10sd
Copy link
Contributor Author

nth10sd commented Oct 21, 2017

Another way is to wait this out till our Python 3 migration is complete, where the issue here will be fixed by itself by only using Python 3 on Windows.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant