Skip to content

Unnniii/Job-Scraper

Repository files navigation

Job Bot — Setup Guide

Scrapes career pages of 36 UK visa-sponsoring companies daily and writes new jobs matching your titles to an Excel spreadsheet.


Your job titles (pre-configured)

  • QA Analyst / QA Engineer / QA Online Tester
  • Quality Assurance Analyst / Engineer
  • Test Analyst / Test Engineer / Digital Test Engineer
  • Business Analyst / Senior Business Analyst / Digital Business Analyst
  • Service Analyst / Operations Analyst / Supply Chain Analyst
  • Process Analyst / Process Improvement Analyst / Continuous Improvement Analyst

Quick Start

Step 1 — Install Python

Download from https://python.org (3.10 or higher)

Step 2 — Install dependencies

Open a terminal / command prompt in this folder and run:

pip install -r requirements.txt

Step 3 — Test it manually

python scraper.py

This will create Jobs_Found.xlsx and jobs.db in the same folder. Open the Excel file to see what was found.


Schedule it to run every morning

Windows (Task Scheduler)

  1. Open Task Scheduler (search in Start Menu)
  2. Click Create Basic Task
  3. Name: Job Bot
  4. Trigger: Daily, set time to 8:00 AM
  5. Action: Start a Program
  6. Program: Browse to run_job_bot.bat in this folder
  7. Click Finish

Mac / Linux (cron)

chmod +x run_job_bot.sh
crontab -e

Add this line (replace the path with your actual folder path):

0 8 * * * /Users/yourname/job_bot/run_job_bot.sh

Output — Jobs_Found.xlsx

Column What it contains
Date Found Date the bot found this job
Company Employer name
Job Title Exact title from their careers page
Location Where the role is based
Salary Salary if listed (not all sites show it)
Posted Date When they posted it (if available)
Status You update this: New / Applied / Rejected
Apply Link Direct link to the job posting

The Dashboard tab shows live counts of New vs Applied jobs.


Files

job_bot/
├── scraper.py          ← Main bot
├── excel_writer.py     ← Excel output handler
├── requirements.txt    ← Python dependencies
├── run_job_bot.bat     ← Windows scheduler script
├── run_job_bot.sh      ← Mac/Linux cron script
├── README.md           ← This file
├── jobs.db             ← Auto-created: deduplication database
├── Jobs_Found.xlsx     ← Auto-created: your job spreadsheet
└── job_bot.log         ← Auto-created: daily run log

Adding / removing companies

Edit COMPANIES list in scraper.py. Each entry is: ("Company Name", "careers_page_url", "strategy")

Adding / removing job titles

Edit JOB_KEYWORDS list in scraper.py. All lowercase. Bot does case-insensitive matching.


Troubleshooting

Bot finds 0 jobs for a company Some career pages are fully JavaScript-rendered (React/Angular SPAs). The generic scraper can't read those. For those companies, either:

  • Use the job board versions (LinkedIn, Reed) as backup
  • Or upgrade to Playwright (ask for the Playwright version of this bot)

Excel file is locked Close the Excel file before running the bot. Python can't write to an open Excel file on Windows.

Rate limiting / getting blocked The bot already adds random 1.5–3.5s delays between requests. If a company blocks you, increase the delay or reduce run frequency.

About

Scrapes Jobs for me

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors