Added multithreading for 4x quicker applying #21
Conversation
|
Nicely done. I did a similar thing with my fork, 4 threads at a time for a few hours - 1500+ applications. Now I'm wishing I'd used futures instead of multiprocessing - looks a lot cleaner. Too bad the reCaptcha ruined my fun |
|
Yes, futures is incredible. Makes multithreading so much easier, and isn't that hard to implement. Whenever manually working with threads outside of threadpools my brain just dies. |
| # Setup | ||
|
|
||
| On mac/pc: | ||
| # Setup (Mac/Windows/Linux) |
There was a problem hiding this comment.
This is great that you've been able to streamline the process for chromedriver as such. Code related to this from this PR should be merged to main immediately.
There was a problem hiding this comment.
I created a seperate pull request due to this one becoming a bit buggy due to many different purposes.
| driver.close() | ||
| continue | ||
| def main(): | ||
| global processes |
There was a problem hiding this comment.
This is the proper way to run multiple things simultaneously. This functionality mirrors what I believe is the intent of #16 , but uses the proper library.
|
Sorry all, this is behind many commits and I am meaning to rework lots of this. I have created a seperate request for webdriver-manager. I will create one later for multiproccessing after captcha and webdriver-manager are implemented. For now, defer to #27 |
Hey all! I recently came across from r/antiwork and thought I would give this script a try. Thanks to the awesome work of SeanDaBlack and other contributors, I found it does. However, as the selenium snob I am, I felt it had a bit to be desired. First of all, it is now supported on linux, and requires no webdriver installation. This was done via a pip dependency called
webdriver_manager. I then sped up the program to send about three at a time (not too many, chrome windows can be laggy). This was done through thefuturesdependency, which allows the simple creation of threadpools. Best of luck, comrades!