Skip to content


Repository files navigation


GitHub Actions status codecov
Crawl and merge ptt/imdb/yahoo movie data, help easy search high rating movie in Chinese and English.

Quick Start

  • nvm use
  • npm install -g yarn
  • docker compose up
  • yarn
  • yarn build && yarn start then open http://localhost:3003

Get the latest Data locally

  1. yarn setup
  2. yarn mergedata


Please open three command line:
For UI developement, server run at http://localhost:3004:

  1. docker compose up
  2. yarn start
  3. yarn webpack

For server developement, server run at http://localhost:3003:

  1. docker compose up
  2. yarn tsc:w
  3. TZ=Asia/Taipei yarn nodemon


yarn test

Debug Test in Vscode

  1. yarn tsc:w
  2. Edit Debug Test section in ./.vscode/launch.json, for Example, if you want to debug netflixCrawler.ts file, modify the args as the following:
"name": "Debug Test",
"args": [
  1. Set the breakpoint and click the Debug Test in Vscode Run and Debug debug test

Application flow

  1. Server start
  2. Load data in cache, include recent movie list, all merged data
  3. Start scheduler for crawl yahoo/imdb/ptt system diagram


  • To manually run a single crawler you could reference firstTimeSetup.ts, and run yarn setup


The project UI is using Material-UI



gcloud auth application-default login
cd terraform
terraform init
terraform plan


It is automatically built and deploy on heroku.