Skip to content

Demo app to scrape any data from any public website with Puppeteer and Bright Data

Notifications You must be signed in to change notification settings

kumamk/web-scraper

Repository files navigation

Description

Web-scraper using Nestjs framework Typescript. Idea is to visit any website using puppeteer core lib and fetch details of searched item.

Pre Requirements

  • puppeteer-core - library that helps drive anything that supports DevTools protocol. A lightweight version of Puppeteer that can launch an existing browser installation or connect to a remote one

  • brightdata.com - web data platfrom used for scraping browser in use with puppeteer-core lib.

Installation

$ pnpm install

Running the app

# development
$ pnpm run start

# watch mode
$ pnpm run start:dev

# production mode
$ pnpm run start:prod

Test

# unit tests
$ pnpm run test

# e2e tests
$ pnpm run test:e2e

# test coverage
$ pnpm run test:cov

About

Demo app to scrape any data from any public website with Puppeteer and Bright Data

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published