Skip to content
Web grep: search all rendered resources used by a URI
JavaScript
Branch: master
Clone or download
stav Merge pull request #1 from Gallaecio/patch-1
Improve the installation instructions
Latest commit 66991d7 Apr 4, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.gitignore do find Aug 8, 2018
LICENSE Create LICENSE Aug 8, 2018
README.md Improve the installation instructions Apr 4, 2019
index.js cleanups Aug 8, 2018
package.json rename Aug 8, 2018
wgrep.js rename Aug 8, 2018

README.md

wgrep

Web grep: search all rendered resources used by a URI

Synopsis

This node command-line utility uses a headless browser (Puppeteer) to render a webpage and download all resources it may need. These resources including the original HTML are all saved locally which it searches one-by-one for a text string.

Code Example

$ wgrep domain http://example.com
Calling for "domain" at "http://example.com" with user "undefined"
Downloading "http://example.com"
http://example.com
seems ok
Finding "domain"
  * output/index.html

Motivation

This project was created to aid in the scraping of web pages. I could see some text on the screen and I wanted to know what file that text came from, perhaps from some XHR resource downloaded with JavaScript.

Installation

$ git clone https://github.com/stav/wgrep.git
$ cd wgrep
$ npm install -g
added 5 packages from 3 contributors and audited 55 packages in 1.06s
found 0 vulnerabilities

License

Apache 2.0

You can’t perform that action at this time.