No description, website, or topics provided.
JavaScript
Switch branches/tags
Nothing to show
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
example
lib
.gitignore
README.md
package.json

README.md

#HCrawler

A hierachical crawler with concurrency control. Provide DOM facility for fetch data from web sites.

Quick Example

crawler.run(

  //href array
  href_array,

  // parse function for each level
  [
    parse_href,
    parse_info
  ],
  
  // callback function
  function (results) {
    save_csv('info.csv');
  },

  // breadth first strategy
  'breadth'
);

How to

Pls see vessel_crawler.js for detail.

Require

async, cheerio