Analytics Agent

AI-powered CSV data analysis tool that enables natural language queries for pattern detection, statistical analysis, and behavioral insights.

Features

• CSV Data Processing: Automatically reads and parses CSV files with various delimiters and encodings
• Intelligent Query Processing: Detects query patterns and executes analytics functions directly for instant results (< 100ms)
• Dual Processing: Uses built-in analytics for statistics, falls back to LLM for complex interpretation
• Natural Language Queries: Ask questions in English or Russian
• Statistical Analysis: Mean, median, mode, percentiles, outliers, correlations (computed accurately, not estimated)
• Pattern Detection: Trends, anomalies, behavioral sequences
• Funnel Analysis: Drop-off points, conversion rates (instant calculation)
• Frequency Analysis: Most common values, error patterns (instant calculation)
• Multi-file Support: Analyze multiple CSV files simultaneously
• 10-100x Faster: Common queries answered in milliseconds vs seconds

Requirements

• Node.js (v14 or higher)
• LLM server (choose one):
  • LM Studio - Local model hosting with OpenAI API (recommended for ease of use)
  • Ollama - Fast local inference server
  • OpenAI API - Cloud API (requires API key)
  • Any OpenAI-compatible API (vLLM, Text Generation WebUI, etc.)

Installation

1. Clone the repository
2. Install dependencies:

   npm install

3. Configure environment variables:

   cp .env.example .env

4. Edit .env file with your Ollama server settings

Configuration

Edit .env file:

Using LM Studio (Recommended)

# API Configuration
API_TYPE=openai

# OpenAI-compatible Server Configuration
OPENAI_HOST=127.0.0.1
OPENAI_PORT=1234
OPENAI_MODEL=your-model-name

# Data folder for CSV files
DATA_FOLDER=./data

Using Ollama

# API Configuration
API_TYPE=ollama

# Ollama Server Configuration
OLLAMA_HOST=192.168.100.183
OLLAMA_PORT=11434
OLLAMA_MODEL=tinyllama

# Data folder for CSV files
DATA_FOLDER=./data

See OPENAI_API_SUPPORT.md for more configuration options.

Usage

Start the Analytics Agent:

npm start

Commands

• /load - Load CSV files from the data folder
• /datasets - List all loaded datasets
• /info <name> - Show detailed information about a dataset
• /clear - Clear the screen
• /help - Show help message
• /exit or /quit - Exit the application

Example Workflow

1. Place your CSV files in the data folder
2. Start the agent: npm start
3. Load the data: /load
4. Ask questions about your data:
   • "What error occurs most frequently?"
   • "Where do users drop off?"
   • "Show me patterns in user behavior"
   • "Calculate statistics for column X"

Sample Data

The project includes 10 diverse sample datasets (270+ rows):

1. sample_errors.csv - Error tracking
2. sample_funnel.csv - Conversion funnel
3. ecommerce_transactions.csv - E-commerce sales
4. website_analytics.csv - Website traffic
5. performance_metrics.csv - API performance
6. customer_feedback.csv - Reviews (includes Russian text)
7. daily_metrics.csv - Business KPIs time-series
8. user_sessions.csv - User behavior
9. ab_test_results.csv - A/B testing
10. server_logs.csv - Server monitoring

See SAMPLE_DATA_GUIDE.md for 100+ example queries.

Supported CSV Formats

• Delimiters: Comma (,), semicolon (;), tab (\t), pipe (|)
• Encodings: UTF-8, UTF-16, ASCII, Windows-1251
• Headers: Files with or without header rows
• File size: Up to 1GB per file

Analytics Capabilities

Error Analysis

• Most common errors
• Error frequency
• Error patterns by severity
• Time-based error distribution

User Behavior

• Drop-off points in funnels
• Conversion rates
• User journeys
• Session analysis

Pattern Recognition

• Trends in time-series data
• Anomaly detection
• Correlation analysis
• Behavioral sequences

Statistical Analysis

• Descriptive statistics (mean, median, mode)
• Percentiles and quartiles
• Standard deviation
• Outlier detection
• Distribution analysis

Architecture

The Analytics Agent consists of three main modules:

csvParser.js

• Reads and parses CSV files
• Detects delimiters and encodings
• Validates data structure
• Handles special characters and escape sequences

analyticsEngine.js

• Statistical calculations
• Pattern detection algorithms
• Funnel analysis
• Correlation and trend analysis
• Data aggregation and grouping

index.js

• CLI interface
• Ollama API integration
• Command processing
• Context management for conversations

Example Queries

Error Analysis

• "What error occurs most frequently?"
• "Show me errors by severity"
• "Какая ошибка чаще всего встречается?"

E-commerce

• "What is the most popular product category?"
• "Calculate total revenue by payment method"
• "Which users made multiple purchases?"

Performance Monitoring

• "Which endpoint has the slowest response time?"
• "Are there any performance outliers?"
• "Show correlation between CPU usage and response time"

Customer Feedback

• "What is the average rating by category?"
• "Show me all negative feedback"
• "Покажи отзывы на русском языке"

Funnel & Conversion

• "Where do users drop off?"
• "Calculate conversion rates by device type"
• "Which variant performs better in the A/B test?"

Time Series

• "Show me the trend in active users"
• "Is revenue increasing or decreasing?"
• "Calculate week-over-week growth"

See SAMPLE_DATA_GUIDE.md for 100+ more query examples.

Development

Project Structure

js-cli/
├── index.js              # Main application
├── csvParser.js          # CSV parsing module
├── analyticsEngine.js    # Analytics engine
├── data/                 # CSV data folder
│   ├── sample_errors.csv
│   ├── sample_funnel.csv
│   └── README.md
├── package.json
├── .env                  # Configuration
└── README.md

Adding New Analytics Functions

To add new analytics capabilities, extend the analyticsEngine.js module:

// Example: Add a new analysis method
customAnalysis(dataset, params) {
  // Your analysis logic here
  return results;
}

Troubleshooting

Connection Issues

• Ensure your LLM server is running (LM Studio, Ollama, etc.)
• Verify API_TYPE is set correctly (ollama or openai)
• Check host and port settings in .env
• For LM Studio: Ensure local server is enabled in the UI
• For Ollama: Run ollama serve

CSV Parsing Issues

• Verify file encoding (UTF-8 recommended)
• Check delimiter format
• Ensure consistent column structure

No Data Loaded

• Place CSV files in the data folder
• Run /load command
• Check console for error messages

Contributing

Contributions are welcome! Please ensure:

• Code follows existing style
• New features include documentation
• CSV parsing maintains data integrity

License

ISC

Acknowledgments

Built with:

• Node.js
• Ollama for LLM capabilities
• dotenv for configuration management