Skip to content
This repository has been archived by the owner on Aug 12, 2020. It is now read-only.

Puppeteer script that lets you list the pages you want to crawl

License

Notifications You must be signed in to change notification settings

Armitage35/puppeteer_crawler

Repository files navigation

Puppeteer

Total alerts CodeFactor GitHub license

This repo provide its users with a script that enables them to crawl and take screenshots of as many pages as they want to.

How to use

  1. Upload the list of URLs you want to parse in the pages.json file
  2. Run yarn start or npm start
  3. Once complete, your screenshots will be available within a "screenshots" folder

Options

  • You can change the parallelism parameter in crawler.js. This will enable your machine to load more tabs at once but will consume more ressources.

Note: if you are looking for a tool that will crawl a site on its own, you may be better off using Pappet.

About

Puppeteer script that lets you list the pages you want to crawl

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published