Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds ability to run parallel windows with multiprocessing #39

Closed

Conversation

BigweldIndustries
Copy link
Contributor

This is a big change that can greatly increase speed. I used the futures library to bring multiprocessing to the application. When it starts, it asks how many parallel processes you would like. It opens that amount of windows, each one generates just as it normally would. I made one slight change, that being start_driver() is now initialize_driver() due to the fact they each seperate process needs to start their own driver that does not close each time, and stays open. Same functionality. Also, this is essentially just a re-do of #21, which is very outdated and convoluted. Also this is a more ideal approach, and somewhat like what #16 strived to accomplish. Hopefully in the future we can achieve headless chrome + proxies. I may work on those. Solidarity forever comrades!

@pws1453
Copy link
Contributor

pws1453 commented Dec 12, 2021

I've built out the headless code fairly well in my PR here #25. I use argparse to give a user the option to pass a --debug option to run headed Chrome, and to otherwise run headless. If you'd like to implement the same strategies I'm using to do this (or change them a bit), you're more than welcome to. Let me know if you do implement this though, so I can close my PR.

@BigweldIndustries
Copy link
Contributor Author

I am curious about how exactly you did that, because when I tried making it headless, it was unable to locate certain elements such as buttons.

@pws1453
Copy link
Contributor

pws1453 commented Dec 12, 2021

Same issue - Going to return my PR to draft and work it out

@BigweldIndustries
Copy link
Contributor Author

I would like to add, another advantage of this implementation is that you can let it run in the background without interfering with you tasks, (the windows are reused and dont auto focus)

@BigweldIndustries
Copy link
Contributor Author

Read #54

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants