Apache Log Analyzer

A powerful and efficient Apache log analysis tool built with Vue.js and Node.js. This tool provides an intuitive web interface that supports real-time log analysis, multi-dimensional statistics, and visualization report generation, helping administrators quickly understand server access patterns and anomalies.

Key Features

• 🔍 Advanced Log Search

  • Multi-level Search: Support for refined searching within search results
  • Flexible Search Criteria: Search by time, IP, URL, status code, and more
  • Real-time Results: Instant display with auto-wrap and copy support

• 📊 Comprehensive Data Analysis

  • IP Statistics: Identify high-frequency visitors and potential crawlers/attacks
  • Time-based Analysis: Analyze traffic patterns by hour/minute/second
  • Security Monitoring: Detect suspicious requests, including SQL injection and XSS attacks
  • Performance Analysis: Identify slow requests for website optimization
  • Professional Reports: Integrated GoAccess for detailed visual analysis reports

• 📝 Extensive Log Support

  • Multi-site Support: Analyze both PC and mobile logs simultaneously
  • Multi-period Analysis: Compare today's and yesterday's logs
  • Automatic Log Retrieval: Support for automatic log download and updates

• 🛡️ Robust Security

  • Access Control: Password-based access protection
  • Rate Limiting: Prevent malicious requests and resource abuse
  • JWT Authentication: Secure and reliable identity verification
  • User Isolation: Run as www user for enhanced security

• 💾 Superior Performance

  • Smart Caching: Automatic caching of common queries
  • Pagination: Batch loading for large datasets
  • Asynchronous Processing: Non-blocking log analysis
  • Scheduled Cleanup: Automatic cache expiration management

System Requirements

• Runtime Environment:

  • Node.js >= 16 (recommended 20)
  • pnpm (Package Manager)
  • PM2 (Process Manager)
  • GoAccess >= 1.4 (Report Generation Tool)
  • Bash Shell (for report generation script)

• System Permissions:

  • Apache log file read access
  • Application directory write access (for caching)
  • www user execution permissions
  • Execute permission for rebuild-report.sh script

Detailed Installation

1. Install GoAccess (Required Component):

# CentOS/RHEL Systems
sudo yum install goaccess

# Ubuntu/Debian Systems
sudo apt-get update
sudo apt-get install goaccess

# Verify Installation
goaccess --version
# Should display version info, e.g.: GoAccess - 1.4.6

2. Get Project Code:

# Clone Repository
git clone [repository-url]
cd apache-log-viewer

# Set Correct Directory Permissions (adjust path as needed)
sudo chown -R www:www $(pwd)

3. Install Dependencies:

# Install Node.js Dependencies (from project directory)
pnpm install

# If using www user
sudo -u www pnpm install

4. Configure Environment:

# Copy environment configuration template
cp .env.example .env.production

# Edit configuration (paths now use relative paths by default)
vim .env.production

# Note: Path configurations are optional and default to project directory:
# - DIST_DIR defaults to ./dist
# - REBUILD_SCRIPT_PATH defaults to ./rebuild-report.sh
# - GOACCESS_CONF defaults to ./goaccess.conf

5. Build Frontend:

# Build Production Code
pnpm run build
# Deployment files will be generated in the dist directory

6. Configure and Start Service:

# Make report generation script executable
chmod +x rebuild-report.sh

# Start Service with PM2
pm2 start app.js --name log-analyzer

# Save PM2 Process List for Auto-restart
pm2 save

# Set PM2 Startup Script (if not already set)
pm2 startup

7. Post-Server Restart Deployment:

# Navigate to Project Directory
cd /path/to/apache-log-viewer

# Rebuild and Restart Service
pnpm run build && pm2 restart log-analyzer

Configuration Details

1. Path Configuration: The project now uses relative paths by default, making it portable across different environments:

# .env.example (all paths are optional)
# If not set, defaults to project directory paths:
# DIST_DIR=./dist
# REBUILD_SCRIPT_PATH=./rebuild-report.sh
# GOACCESS_CONF=./goaccess.conf

# You can override with absolute paths if needed:
DIST_DIR=/custom/path/to/dist
REBUILD_SCRIPT_PATH=/custom/path/to/rebuild-report.sh
GOACCESS_CONF=/custom/path/to/goaccess.conf

2. Log File Configuration (via environment variables):

# Configure log file paths in .env.production
LOG_PATH_PC_TODAY=/www/wwwlogs/chinatruck.org-access_log

3. Service Configuration:

• Service Port: Default 25635 (configurable via PORT env variable)
• Process Management: PM2 for service stability
• Log Analysis: Integrated GoAccess for deep analysis
• Working Directory: Automatically detected from project location

Usage Guide

1. System Access:

   • Browser Access: http://your-domain:25635
   • Initial access requires configured password
   • Session maintained after login (JWT-based)

2. Log Analysis Features:

   • Log Selection:

     • Choose log file from dropdown menu
     • Switch between PC and mobile logs
     • View today's and yesterday's logs

   • Search Functionality:

     • Precise keyword search
     • Filter within search results
     • Search by time period, IP, URL, etc.

   • Group Statistics:

     • IP Grouping: View highest frequency IPs
     • Time Grouping: Traffic statistics by hour/minute/second
     • Automatic Sorting: Results sorted by frequency

3. Result Processing:

   • Display Control:
     • Auto-wrap: Long content wraps automatically
     • Record Count: Shows total matching records
   • Data Operations:
     • Copy Function: One-click result copying
     • Export: Result export support (planned)

4. Cache Management:

   • Rebuild Function:
     • Clear cache via "Rebuild" button
     • Limited to once per minute
     • Automatically downloads latest log files
   • Cache Strategy:
     • Auto-caching of common queries
     • Smart cache expiration control
     • Periodic expired cache cleanup

5. Report Generation:

   • View Reports:

     • Click "Open Report" to view analysis report
     • Reports open in new window
     • Support report refresh and export

   • Report Generation:

     • Initial use requires clicking "Rebuild"
     • Progress display during generation
     • Auto-opens report upon completion

   • Report Contents:

     • Traffic statistics and trends
     • Visitor sources and geography
     • Request resources and status statistics
     • Browser and OS distribution
     • Crawler and bot access analysis

Important Notes

1. Permission Management:

   • File Permissions:

     • Ensure www user has log read access
     • Ensure cache directory is writable
     • Regular permission checks

   • Runtime Permissions:

     • Service must run as www user
     • Correct PM2 process permissions
     • Avoid running as root

2. Performance Optimization:

   • Search Optimization:

     • Use precise search terms
     • Avoid overly broad keywords
     • Utilize multi-level search effectively

   • Cache Usage:

     • Use rebuild function wisely
     • Mind rebuild frequency limits
     • Understand cache expiration

   • Resource Management:

     • Regular old report cleanup
     • Monitor disk space usage
     • Watch log file sizes

3. Security Measures:

   • Access Control:

     • Regular password changes
     • Avoid simple passwords
     • Timely JWT key updates

   • Deployment Security:

     • Avoid public internet exposure
     • Use reverse proxy protection
     • Configure appropriate firewall rules

   • Data Security:

     • Monitor sensitive log information
     • Regular anomaly checks
     • Protect configuration files

Technical Architecture

• Frontend Technology:

  • Vue 3: Modern reactive framework
  • Vue Router: Client-side routing
  • Axios: HTTP request handling
  • Modern UI Components

• Backend Technology:

  • Node.js: High-performance server
  • Express: Lightweight web framework
  • PM2: Process management and load balancing
  • JWT: Secure authentication

• Core Features:

  • GoAccess: Professional log analyzer
  • NodeCache: Efficient caching system
  • Shell Commands: System-level log processing

Troubleshooting

1. Service Access Issues:

   • Symptoms: Unable to access service
   • Diagnostic Steps:
     • Check PM2 status: pm2 list
     • View error logs: pm2 logs log-analyzer
     • Check port usage: lsof -i:25635
     • Verify firewall settings
   • Solutions:
     • Restart service: pm2 restart log-analyzer
     • Check configuration files
     • Verify permissions

2. Server Restart Issues:

   • Automatic Recovery:

     • PM2 auto-starts service
     • Ensure pm2 save was executed
     • Verify PM2 startup configuration

   • Manual Restart:

     cd /path/to/apache-log-viewer
     pnpm run build && pm2 restart log-analyzer

3. Cache-Related Issues:

   • Cache Clearing:

     • Use rebuild button to clear cache
     • Note frequency limit (once per minute)
     • Wait for cleanup completion

   • Performance Tuning:

     • Use cache judiciously
     • Avoid frequent clearing
     • Monitor cache size

4. Report Generation Issues:

   • Prerequisites:

     • Verify GoAccess: goaccess --version
     • Check log file permissions
     • Verify disk space availability

   • Troubleshooting:

     • Initial use requires report generation
     • Report generation takes time
     • Check error logs for issues

   • Best Practices:

     • Choose optimal generation times
     • Regular report updates
     • Maintain clean report directory

Development Roadmap

1. Near-term Plans:

   • Add more statistical dimensions
   • Optimize report generation speed
   • Implement report export functionality

2. Feature Iterations:

   • Support additional log formats
   • Add custom analysis rules
   • Enhance search performance

3. Continuous Improvements:

   • Enhance error handling
   • Improve user experience
   • Strengthen security features