beta-v621

• Added Past Month button to the config file
• Changed the scraping logic yet again. It will keep scraping jobs until it has enough valid jobs or has run out of jobs to scrape. It will now also always perform one loop of the scrape for every location even if we are over the threshold. This ensures that we always pull something from every location
• Fixed requirements.txt
• In regards to main.py
  • Fix logs directory not being made with the right name
  • Added a whole bunch more of logging. This is in an effort to make troubleshooting a lot easier. I want the user to have all the information I need to help them already ready to go
  • Added the search to our output. Now we can see which searches are yielding which results (if any)
  • Added further protection against LinkedIn's bot detection redirects
  • Change the timespan logic
    • The user no longer says if they want to do past day, week, or month. It will now always start with day and keep going until it gets enough valid jobs. However, if it runs out of jobs and does not have enough, it will re-do the search but this time with past week and then past month. As soon as it gets enough jobs from past week or month it will break out and not continue.
  • Added further protections against erroring out when there are no results
  • Added stat and log tracking for how many duplicates, previously found, and dud job postings we find. This will help the user understand why this script takes so long to run
  • Added more safeguards
  • Added a post script report that dumps all the stats about the run
• Figured out exactly what I needed in order to setup cron and now can add it to documentation