Skip to content

Proof of concept for a distributed puppeteer crawling service.

Notifications You must be signed in to change notification settings

nduvieilh/puppeteer-crawler-test

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

puppeteer-crawler-test

Proof of concept for a distributed puppeteer crawling service.

What does it do?

Given a list of state names it will navigate to the corresponding wikipedia.com page, take a screenshot, and save it in the screenshots folder. It utilizes a homegrown load balancer concept and Google's puppeteer library to allow multiple simultaneous asynchronous requests to be processed at the same time.

About

Proof of concept for a distributed puppeteer crawling service.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published