Welcome to house-finder
A task scheduler runs every now and then to automatically find apartments instead of doing it manully everyday. Tech: Azure function, Docker, Puppeteer, S3.
- A function gets triggered on a timer.
- Gets existing data from S3 bucket.
- Goes to website 1, scrap all data.
- Compares with the existing data from S3, and remove duplicates to reduce overhead and only keep the new ones.
- Makes api calls to binmap to filter out unwanted data.
- Goes to website 2, repeat steps from 4 - 5.
- Upload both the latest data and old data to S3, so in the future don't scrap them again.
- Send email only with the latest data.
npm run start
npm run test
- Github: @YIZHUANG
Show your support