MainframePulse RT Backend

Real-time mainframe event processing with Confluent Kafka and Google Gemini AI.

🚀 Quick Start

Prerequisites

• Node.js 16+
• Confluent Cloud account (or local Kafka)
• Google Gemini API key

Installation

1. Clone the repository:

   git clone https://github.com/yourusername/mainframe-pulse-rt.git
   cd mainframe-pulse-rt

2. Install dependencies:

   npm install

3. Configure environment variables:

   cp .env.example .env

   Edit .env and add your credentials:

   # Gemini AI Configuration (REQUIRED)
   GEMINI_API_KEY=your_gemini_api_key_here
   
   # Confluent Cloud Kafka Configuration (REQUIRED)
   KAFKA_BOOTSTRAP=your-kafka-bootstrap-server:9092
   KAFKA_TOPIC=mainframe-events
   KAFKA_GROUP_ID=mainframe-pulse-consumer
   KAFKA_API_KEY=your_confluent_api_key
   KAFKA_API_SECRET=your_confluent_api_secret
   
   # Server Configuration (OPTIONAL)
   PORT=3000

4. Start the server:

   npm start

5. Open the dashboard: Navigate to http://localhost:3000/dashboard

🔧 Configuration

Getting API Keys

Google Gemini API Key

1. Go to Google AI Studio
2. Create a new API key
3. Copy the key to your .env file

Confluent Cloud Setup

1. Sign up at Confluent Cloud
2. Create a cluster and topic
3. Generate API key and secret
4. Copy credentials to your .env file

📡 API Endpoints

• GET / - Landing page
• GET /dashboard - Real-time dashboard
• GET /health - Health check
• GET /events/latest - Get latest processed events with AI insights
• POST /events/simulate - Generate sample events for testing
• POST /events/startStream - Start continuous event generation
• POST /events/stopStream - Stop event generation
• GET /events/stats - Get processing statistics

🎯 Features

• Real-time Event Processing: Live mainframe event analysis
• AI-Powered Insights: Google Gemini analyzes events for ABEND codes, causes, and fixes
• Interactive Dashboard: Real-time charts, tables, and event details
• Kafka Integration: Confluent Cloud streaming with SSL/SASL authentication
• Event Simulation: Generate realistic mainframe events for testing
• Time Filtering: View events by time windows (30s, 5m, all)
• Event Details: Click any event for comprehensive analysis

🏗️ Architecture

• Backend: Node.js + Express
• Streaming: Confluent Kafka (KafkaJS)
• AI Analysis: Google Gemini Pro
• Frontend: Vanilla JavaScript with real-time updates
• Storage: In-memory (last 200 events)

🧪 Testing

Test event simulation:

curl -X POST http://localhost:3000/events/simulate \
  -H "Content-Type: application/json"

📊 Sample Event Format

{
  "id": "evt_1703331234567_abc123def",
  "type": "SMF30",
  "job_name": "PAYROLL0123",
  "abend_code": "S0C4",
  "severity": "high",
  "system": "PROD-SYS1",
  "cpu_time": 1250,
  "timestamp": "2024-01-15T10:30:00Z",
  "message": "SMF30 event in PAYROLL0123 - ABEND S0C4 detected"
}

🔒 Security Notes

• Never commit .env files to version control
• Use environment variables for all sensitive data
• Rotate API keys regularly
• Use HTTPS in production

🚀 Deployment

Environment Variables for Production

Set these environment variables in your deployment platform:

GEMINI_API_KEY=your_production_gemini_key
KAFKA_BOOTSTRAP=your_production_kafka_bootstrap
KAFKA_API_KEY=your_production_kafka_key
KAFKA_API_SECRET=your_production_kafka_secret
KAFKA_TOPIC=mainframe-events
PORT=3000

📝 License

MIT License - see LICENSE file for details

🤝 Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests if applicable
5. Submit a pull request

📞 Support

For issues and questions:

• Create an issue on GitHub
• Check the documentation
• Review the console logs for debugging

---

MainframePulse RT - Real-time mainframe intelligence powered by AI 🚀