There are some different parts:
The API allow you to configure and control the crawling process.
Some scripts to send request to the API.
Processing agent. You can start more agent to speed up the process.
It's the WebUI. Don't forget to copy config.dist.js
to config.js
and setup the URL to you API.