Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
A monitoring system for Heritrix 3.
branch: master

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
monitrix-loader
monitrix-webapp Startup interval time changed to 2 minutes (issue #21)
.gitignore
LICENSE.txt
README.md
start-monitrix.sh

README.md

monitrix

monitrix is a monitoring/analytics frontend for the Heritrix 3 Web crawler. Visit the Wiki for information about:

Developers: Quick Start

To start monitrix in development mode, change into the project root folder and type play run. The application will be at http://localhost:9000.

To generate an Eclipse project, type play eclipse.

Getting Data into monitrix

To load data into monitrix, enter the 'Admin' section, and enter the absolute path of a log file in the form. The log file should immediately appear in the list above the form, with status 'CATCHING UP' (or 'PENDING', followed shortly thereafter by 'CATCHING UP'). monitrix will now load the log file into the database. After the upload is complete, monitrix will continuously check the log file for updates. Warning: Loading data takes time! On my machine, a 10 GB log sample currently takes about 1 hour to process!

Alternatively, you can also populate the database 'manually' using either of the following Java utilities, located in the /test folder of the project:

  • uk.bl.monitrix.util.BatchLogProcessor will load a log file into the database in one go and then terminate.
  • uk.bl.monitrix.util.IncrementalLogProcessor will load a log file into the database, and then continue to monitor that file (and incrementally sync the DB) until it is terminated forcefully.
Something went wrong with that request. Please try again.