Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Web crawler in Node.js
JavaScript CSS
branch: master

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
api
cli Drop old code
crawler
web
.gitignore
README.md

README.md

UniCrawler

There are some different parts:

Requirements

API

The API allow you to configure and control the crawling process.

Cli

Some scripts to send request to the API.

Crawler

Processing agent. You can start more agent to speed up the process.

Web

It's the WebUI. Don't forget to copy config.dist.js to config.js and setup the URL to you API.

Something went wrong with that request. Please try again.