a spider for github trending
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
cron
downloader
parser
store
.gitignore
.travis.yml
LICENSE
README.md
app.js
config.js
logger.js
package.json
spider.js

README.md

spider-trending

Build Status JavaScript Style Guide license

专门用于爬取 github trending 的爬虫

Requirements

  • node 越新越好
  • redis 2.8+

Usage

修改redis配置文件,添加 notify-keyspace-events Ex 选项

git clone https://github.com/xiedacon/spider-trending.git
cd spider-trending
npm install
npm start

Datas

[
  {
    "name": "language",
    "href": "language href",
    "repos": [
      {
        "author": "repository author",
        "repoName": "repository name",
        "content": "repository introduction",
        "programmingLanguage": "repository language",
        "starTotal": "total star",
        "fork": "fork",
        "starIncrease": "increase star"
      }
    ]
  }
]

TODO

  • 定时任务控制
  • pm2
  • 部署树莓派
  • 数据分析(一个月后)

License

MIT