This app automates a headless browser to search for jobs on Indeed/Linkedin/Builtin job boards and presents results in a tabular form such that you can see all info about companies and job posts at a glance.
It runs in Docker locally only and all data are saved to your database locally.
You can
- create crawling tasks for Indeed, Linkedin, Builtin job boards
- view and remove unfit job posts from results view
- save notes about job posts and companies
- mute certain companies and remove all their posts from view
- view and unmute muted companies
- save job posts for later by marking them as 'plan apply' or 'applied'
- React
- Tabulator
- Python
- Asyncio
- BeautifulSoup
- Playwright
- Flask
- SQLAlchemy
- Postgres
- Celery
- Redis
- GitLab CI
- Docker
See Docker Hub for usage
- There is a button [UPDATE USER] to update Linkedin credentials once the previous account "expires".
create the file "/.env.dev.postgres"
POSTGRES_USER=postgres POSTGRES_PASSWORD=1 DATABASE_NAME=bas
# create and activate venv and install dependencies:
cd backend
pip install poetry==1.1.14
python -m venv .venv
. .venv/bin/activate
poetry install
playwright install chromium
playwright install-deps
# start dbms:
docker-compose -f docker-compose.dev.yml up -d
# create backend db migration:
cd backend/src
flask db init # adds support for db migrations
flask db migrate # creates migration script
flask db upgrade # applies changes to db
# start celery worker:
celery -A app.celery worker --loglevel=info --concurrency=1 # process 1 concurrent task in a queue
# start backend:
export FLASK_DEBUG=1
flask run -p 5000
# start frontend:
export PORT=3001
npm run start
# a separate script for manual testing of selectors, not part of the application
python -m bas_app.scraper.man
# clear redis cache when necessary
docker exec redis_dev redis-cli -c "flushall"
