UPDATE 2022: Monster has changed their site from static to dynamic, which now breaks this project. I'm keeping this repo up as a proof of concept.
I followed a tutorial from Real Python and Martin Breuss.
In the tutorial, they teach you about Beautiful Soup by creating a web scraper that searches Monster.com for software developer positions.
I've built upon it by allowing users to pass along arguments to the script so that users can get more tailored results, making it a command line app. You can pass along city or state and language to get the results you need. In the future, you can pass along another argument (called Role) so that you can look for more relevant positions (e.g front end, data scientist, etc).
There are two available scripts:
-
RP_Tutorial.py: The sample script from the tutorial with my additional comments and UI changed to help first timers follow along easier -
job_scraper.py: The command line app to help you scrape relevant jobs from Monster by City and Language
- Python3
- beautifulsoup4
- requests
You can go to repl.it where I'm hosting the app in a personal repl. When you click the link, just hit the button up top that says Run with the forward arrow and the app will start up.
Clone this repo. Then set your environment set up by running the following command in your terminal:
pip install -r requirements.txt
Run job_scraper.py to start scraping for jobs.
OR if you're using Replit, simply hit Run at the top.
If this doesn't work, feel free to reach out!