Using Puppeteer to Scrape Websites with Infinite Scrolling
This directory is centered around scrape-infinite-scroll.js, which uses Puppeteer to scrape infinite scroll items from a demo page set up for it. The script's implementation details are described in the Using Puppeteer to Scrape Websites with Infinite Scrolling article published on the Intoli blog. Customizing the script should be straightfoward after reading this article.
git clone https://github.com/Intoli/intoli-article-materials.git cd intoli-article-materials/articles/scrape-infinite-scroll
And then install Puppeteer with
Finally, run the script with