Skip to content

ngrash/fetchdom

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

19 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

fetchdom

You are in the middle of a web scraping job when you realize the page that holds all the juicy info you desire uses ReactJS and AJAX.

If you are me, you write a PhantomJS script to fetch the page, then realize that it was worse than you thought, as the page uses lazy loading to load more content as you scroll. So you recursively scroll down until scrolling does not trigger any more requests. You think this could be useful for others, so you publish it to npm.

If you are not me, you just run npm install --global fetchdom and continue writing your regular expressions to parse the HTML.

Usage: fetchdom [options] <url>

  A command-line tool for saving the rendered DOM of a website.


  Options:

    -V, --version            output the version number
    -s, --save-image <path>  write PNG screenshot to <path>
    -h, --help               output usage information

About

A command-line tool for saving the rendered DOM of a website.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published