Skip to content

medialab/UniCrawler

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

UniCrawler

There are some different parts:

Requirements

API

The API allow you to configure and control the crawling process.

Cli

Some scripts to send request to the API.

Crawler

Processing agent. You can start more agent to speed up the process.

Web

It's the WebUI. Don't forget to copy config.dist.js to config.js and setup the URL to you API.

Releases

No releases published

Packages

No packages published

Languages

  • JavaScript 57.5%
  • CSS 42.4%
  • Shell 0.1%