Skip to content

0xSolanaceae/NeuroLog

Repository files navigation

NeuroLog: ML-Powered Log Analysis

Overview

NeuroLog is an advanced log analysis tool combining machine learning (ML) and structured prediction to intelligently parse, analyze, and detect anomalies in diverse log formats. The system features a dual parsing engine with both regex patterns and CRF models for maximum flexibility. Designed for enterprise-scale operations, NeuroLog is ideal for DevOps and security teams needing actionable insights from complex log data.

Benchmark Results

Metric NeuroLog ELK Stack Graylog
Format Accuracy 94.1% 81.2% 78.9%
Parser Recall 96.8% 82.4% 85.1%
RAM Efficiency 1.1GB 3.8GB 2.9GB

Technical Paper


Key Features

  • Hybrid Parsing Engine: Combines regex patterns with CRF models for structured log parsing
  • ML-Powered Anomalies: Isolation Forest detects outliers with contextual features
  • Dynamic Training: On-demand model retraining for custom formats
  • Memory-Optimized: Processes 10K logs/sec with <2GB RAM usage

Installation

  1. Clone repository:
git clone https://github.com/0xSolanaceae/NeuroLog.git
cd NeuroLog
  1. Install dependencies with Poetry:
poetry install

Usage

1. Activate environment

poetry self add poetry-plugin-shell
poetry shell
cd src

2. Full Pipeline Analysis

python3 neurolog.py analyze logs/path/to/log.log --output results/anomalies.csv

(test usage)

python3 src/neurolog.py analyze src/logs/apache.log
  • Automatic format detection
  • Anomaly scoring with explanations
  • Multi-format output support (CSV/JSON/HTML)

Example

3. CRF Model Management

# Train new model
python3 src/neurolog.py train-crf --output src/models/crf_model.pkl

License

GPLv3 - See LICENSE for details.

About

ML-Powered Log Analysis

Topics

Resources

License

Stars

Watchers

Forks

Contributors