Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

workers misbehaves #14

Closed
xalgord opened this issue Aug 3, 2023 · 2 comments
Closed

workers misbehaves #14

xalgord opened this issue Aug 3, 2023 · 2 comments

Comments

@xalgord
Copy link

xalgord commented Aug 3, 2023

When I set workers to somewhat like high numbers, it gets locked to (Scanning 0/total_urls) and doesn't start scanning. Sometimes it happens even if I set workers to 30 while without workers it works fine.

@xalgord
Copy link
Author

xalgord commented Aug 3, 2023

I think this is due to race conditions, while multiple threads trying to acquire the same lock

@xalgord
Copy link
Author

xalgord commented Aug 3, 2023

I've rewritten the app.py script code using threading.Semaphore() instead of threading.lock() with some other minor changes in the script and its working fine. You can use as many threads as you want (but don't use more than 100). Making a PR now.

@xalgord xalgord changed the title Setting workers misbehaves workers misbehaves Aug 3, 2023
@xalgord xalgord closed this as completed Jan 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant