Skip to content

jun85664396/auto-web-crawler

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

20 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

auto-web-crawler

###0.0.5 Update

  • add sorting

##Installation

npm install awc

##Usage

var awc = require("awc");

var getIndex = function(callback){

	//Write your code
	//ex) var idx = [8911,8912,8913,8914,8915]; //url Suffixes.

	callback(idx);
};

awc.run({
    "minute":1, // If Minute Parameter is null,Minute Default Value is 60.
    "url":"https://github.com/joyent/node/issues/", //Idx Default Prefix.
    "idxFunc":getIndex, //Index List
    "rule":{"title":"$('.gh-header-title').text().trim()"}, //Jquery Selector Rule.
    "done":function(res){
    
		console.log(res); //Return Value { '0': { title: 'events.markdown document improvements\n #8911' } ...}
		//Write your code
		
    }
});

##Test

$ node test.js

##Parameter

  • minute int
  • url String
  • idxFunc Function // Require parameter callback
  • rule Hash

#use

  • jsdom
  • fs

About

auto web crawler , nodeJs

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published