Skip to content

AI Powered Features

Jump to bottom
github-actions[bot] edited this page Jun 1, 2026 · 3 revisions

AI-Powered Features

**Referenced Files in This Document** - [README.md](file://README.md) - [server/lib/openai.ts](file://server/lib/openai.ts) - [server/routes.ts](file://server/routes.ts) - [server/index.ts](file://server/index.ts) - [client/src/pages/ai-tutor.tsx](file://client/src/pages/ai-tutor.tsx) - [client/src/hooks/use-chat-ws.ts](file://client/src/hooks/use-chat-ws.ts) - [client/src/lib/chat-api.ts](file://client/src/lib/chat-api.ts) - [MESSAGEPAL_IMPLEMENTATION_SUMMARY.md](file://MESSAGEPAL_IMPLEMENTATION_SUMMARY.md) - [message_pal.md](file://message_pal.md) - [.env.example](file://.env.example) - [playback-engine.ts](file://client/src/lib/playback-engine.ts) - [WhiteboardCanvas.tsx](file://client/src/components/ai-classroom/WhiteboardCanvas.tsx) - [ai-classroom.ts](file://server/routes/ai-classroom.ts) - [generator.ts](file://server/services/study-arena/generator.ts) - [OPENMAIC_CHEAT_SHEET.md](file://docs/OPENMAIC_CHEAT_SHEET.md)

Table of Contents

  1. Introduction
  2. Project Structure
  3. Core Components
  4. Architecture Overview
  5. Detailed Component Analysis
  6. Dependency Analysis
  7. Performance Considerations
  8. Troubleshooting Guide
  9. Conclusion
  10. Appendices

Introduction

This document explains the AI-powered features of PersonalLearningPro, focusing on the OpenAI integration for the AI tutor, intelligent test creation, automated answer evaluation, and performance analysis. It documents the AI assistant chat interface, conversation context management, and response generation. It also covers the performance analytics system using AI insights for learning pattern recognition, API integration patterns, prompt engineering strategies, response handling, model selection, cost optimization, fallback mechanisms, and the MessagePal AI framework implementation within the platform.

Project Structure

The AI capabilities are implemented across the backend server and the React frontend:

  • Backend exposes REST endpoints and integrates OpenAI for chat, evaluation, study plan generation, and performance analysis.
  • Frontend provides the AI tutor chat UI and integrates with the backend via HTTP requests and WebSocket for real-time messaging (MessagePal).
  • Environment variables configure OpenAI access keys and session secrets.
graph TB
subgraph "Frontend (React)"
AITutor["AI Tutor Page<br/>client/src/pages/ai-tutor.tsx"]
ChatWS["Chat WebSocket Hook<br/>client/src/hooks/use-chat-ws.ts"]
ChatAPI["Messaging REST API Client<br/>client/src/lib/chat-api.ts"]
end
subgraph "Backend (Express)"
Routes["Routes<br/>server/routes.ts"]
OpenAI["OpenAI Integration<br/>server/lib/openai.ts"]
Index["Server Entry<br/>server/index.ts"]
end
Env[".env Configuration<br/>.env.example"]
AITutor --> |HTTP| Routes
Routes --> |OpenAI| OpenAI
ChatWS --> |WebSocket| Index
ChatAPI --> |HTTP| Index
Env --> OpenAI
Env --> Index
Loading

Diagram sources

  • client/src/pages/ai-tutor.tsx
  • client/src/hooks/use-chat-ws.ts
  • client/src/lib/chat-api.ts
  • server/routes.ts
  • server/lib/openai.ts
  • server/index.ts
  • .env.example

Section sources

  • README.md
  • server/lib/openai.ts
  • server/routes.ts
  • server/index.ts
  • client/src/pages/ai-tutor.tsx
  • client/src/hooks/use-chat-ws.ts
  • client/src/lib/chat-api.ts
  • .env.example

Core Components

  • OpenAI integration module encapsulates:
    • AI chat with system prompt injection
    • Subjective answer evaluation with JSON response parsing and bounds checking
    • Study plan generation with JSON schema enforcement
    • Test performance analysis with JSON schema enforcement
  • REST routes expose:
    • AI chat endpoint for interactive tutoring
    • AI evaluation endpoint for teacher-driven grading
    • Integration points for OCR and test management
  • AI tutor page manages:
    • Conversation history persisted in local storage
    • Message composition and submission to backend
    • Tabbed workflows for explain, solve, and visualize
  • MessagePal framework provides:
    • Real-time messaging with WebSocket and HTTP fallback
    • Persistent storage using Cassandra
    • Role-based communication and read receipts

Section sources

  • server/lib/openai.ts
  • server/routes.ts
  • client/src/pages/ai-tutor.tsx
  • MESSAGEPAL_IMPLEMENTATION_SUMMARY.md
  • message_pal.md

Architecture Overview

The AI features integrate with the broader platform as follows:

  • AI tutor chat uses a dedicated endpoint that forwards messages to OpenAI and returns natural language responses.
  • Automated evaluation leverages OpenAI to score subjective answers and provide feedback, with robust fallbacks.
  • Performance analytics uses OpenAI to analyze aggregated test results and produce actionable recommendations.
  • MessagePal provides real-time communication channels for student-teacher and administrative interactions, complementing AI tutoring.
sequenceDiagram
participant Client as "AI Tutor UI<br/>client/src/pages/ai-tutor.tsx"
participant API as "Backend Routes<br/>server/routes.ts"
participant OAI as "OpenAI Module<br/>server/lib/openai.ts"
Client->>API : POST /api/ai-chat {messages}
API->>OAI : aiChat(messages)
OAI-->>API : {content}
API-->>Client : {content}
Loading

Diagram sources

  • client/src/pages/ai-tutor.tsx
  • server/routes.ts
  • server/lib/openai.ts

Detailed Component Analysis

OpenAI Integration Module

The OpenAI integration module centralizes AI capabilities:

  • Model selection: gpt-4o is configured as the default model.
  • Chat completions: injects a system prompt if none is provided.
  • Evaluation: enforces JSON output with bounded score and confidence values.
  • Study plan: returns structured plan and resources in JSON.
  • Performance analysis: returns average score, hardest questions, and recommendations in JSON.
  • Fallbacks: gracefully handle AI failures with sensible defaults.
flowchart TD
Start(["Call evaluateSubjectiveAnswer"]) --> BuildMsg["Build system and user messages"]
BuildMsg --> CallOAI["Call OpenAI chat.completions.create"]
CallOAI --> ParseResp{"Parse JSON response"}
ParseResp --> |Success| Bounds["Clamp score/confidence to expected ranges"]
ParseResp --> |Error| Fallback["Return fallback evaluation"]
Bounds --> ReturnOK["Return evaluation result"]
Fallback --> ReturnOK
Loading

Diagram sources

  • server/lib/openai.ts

Section sources

  • server/lib/openai.ts

AI Chat Endpoint and UI

The AI chat endpoint and UI coordinate:

  • Endpoint: validates messages array and delegates to the OpenAI module.
  • UI: maintains conversation history in local storage, composes messages, and displays assistant responses.
  • Prompt engineering: system prompt tailored for high school-level explanations.
sequenceDiagram
participant UI as "AI Tutor UI<br/>client/src/pages/ai-tutor.tsx"
participant Route as "Route /api/ai-chat<br/>server/routes.ts"
participant OAI as "aiChat<br/>server/lib/openai.ts"
UI->>UI : Append user message to conversation
UI->>Route : POST /api/ai-chat {messages}
Route->>OAI : aiChat(messages)
OAI-->>Route : {content}
Route-->>UI : {content}
UI->>UI : Append assistant message to conversation
Loading

Diagram sources

  • client/src/pages/ai-tutor.tsx
  • server/routes.ts
  • server/lib/openai.ts

Section sources

  • server/routes.ts
  • client/src/pages/ai-tutor.tsx
  • server/lib/openai.ts

Automated Answer Evaluation

The evaluation endpoint:

  • Validates teacher permissions and test ownership.
  • Retrieves associated question rubric and marks.
  • Calls the evaluation function and updates the answer with score, confidence, and feedback.
  • Provides a fallback evaluation on failure.
sequenceDiagram
participant Teacher as "Teacher UI"
participant Route as "Route /api/evaluate<br/>server/routes.ts"
participant Eval as "evaluateSubjectiveAnswer<br/>server/lib/openai.ts"
participant Store as "Storage"
Teacher->>Route : POST /api/evaluate {answerId}
Route->>Store : Load answer, attempt, test, question
Route->>Eval : evaluateSubjectiveAnswer(text, question, rubric, marks)
Eval-->>Route : {score, confidence, feedback}
Route->>Store : Update answer with evaluation
Route-->>Teacher : Updated answer
Loading

Diagram sources

  • server/routes.ts
  • server/lib/openai.ts

Section sources

  • server/routes.ts
  • server/lib/openai.ts

Performance Analytics

The performance analysis endpoint:

  • Aggregates test results and calls the analysis function.
  • Returns average score, the hardest questions, and recommendations.
  • Includes JSON schema enforcement and fallback computation.
sequenceDiagram
participant Admin as "Admin/Teacher UI"
participant Route as "Route analyzeTestPerformance<br/>server/routes.ts"
participant OAI as "analyzeTestPerformance<br/>server/lib/openai.ts"
Admin->>Route : POST /api/analytics/performance {results}
Route->>OAI : analyzeTestPerformance(results)
OAI-->>Route : {averageScore, hardestQuestions, recommendations}
Route-->>Admin : Analysis results
Loading

Diagram sources

  • server/lib/openai.ts

Section sources

  • server/lib/openai.ts

MessagePal AI Framework

MessagePal provides a real-time messaging backbone:

  • WebSocket service with authentication, typing indicators, read receipts, and persistent storage.
  • REST API endpoints for conversations and message history.
  • Cassandra-backed storage with materialized views for efficient queries.
  • Frontend components and hooks for seamless integration.
graph TB
subgraph "MessagePal Backend"
WS["WebSocket Service<br/>server/messagepal/index.ts"]
Routes["REST Routes<br/>server/messagepal/routes.ts"]
Store["Cassandra Store<br/>server/messagepal/cassandra-message-store.ts"]
Schema["Cassandra Schema<br/>server/messagepal/cassandra-schema.cql"]
end
subgraph "MessagePal Frontend"
Panel["MessagePal Panel<br/>client/src/components/messagepal/messagepal-panel.tsx"]
Hook["use-messagepal-ws<br/>client/src/components/messagepal/use-messagepal-ws.ts"]
end
WS --> Store
Routes --> Store
Panel --> Hook
Hook --> WS
Loading

Diagram sources

  • MESSAGEPAL_IMPLEMENTATION_SUMMARY.md
  • message_pal.md

Section sources

  • MESSAGEPAL_IMPLEMENTATION_SUMMARY.md
  • message_pal.md

Study Arena Classroom Engine (OpenMAIC)

The Study Arena feature integrates the OpenMAIC classroom playback and generation engine, providing interactive AI-generated classrooms for structured learning.

Core Capabilities & Architecture

  • Stateless Generation: The generation backend is fully stateless. The user's requirements travel to the backend, which runs a two-stage LangGraph pipeline:
    • Stage 1 (Outline Generation): Builds a structured sequence of scene outlines (SceneOutline[]).
    • Stage 2 (Scene Generation): Translates outlines into concrete React slides, interactive widgets, quiz questions, and whiteboard actions.
  • Multi-Provider LLM Engine: The generator dynamically cascades through a chain of AI providers if configured endpoints fail: Gemini → Anthropic → DeepSeek → Qwen → OpenRouter → Kimi → Grok → Ollama → OpenAI.
  • Zustand State Management: The client side keeps all active playback, canvas elements, settings, and chat history in synced Zustand stores (main-store.ts, canvas.ts, settings.ts).

Playback State Machine (PlaybackEngine)

The PlaybackEngine manages a rich state machine governing how lessons play:

stateDiagram-v2
    [*] --> Idle
    Idle --> Playing : start()
    Playing --> Paused : pause()
    Paused --> Playing : resume()
    Playing --> Live : confirmDiscussion() / handleUserInterrupt()
    Live --> Paused : pause()
    Paused --> Live : resume()
    Live --> Idle : handleEndDiscussion()
Loading
  • Playback Controls: Standard control panel exposing play, pause, resume, skip, and stop callbacks mapped via the usePlayback hook.
  • Audio Pipeline:
    • Text-to-Speech (TTS): Features sentence-chunked browser SpeechSynthesis (avoiding Chrome's 15s utterance limit) and a togglable server-based TTS mode (hitting /api/ai-classroom/tts backed by OpenAI TTS).
    • Automatic Speech Recognition (ASR): Enables voice inputs by recording the microphone (MediaRecorder) and posting to /api/ai-classroom/asr (backed by Whisper).
  • Interactive whiteboards: The SVG-based WhiteboardCanvas maps draw actions (wb_draw_text, wb_draw_latex, wb_draw_shape, wb_draw_chart, etc.) using a normalized 1000 x 562 coordinate system. Real-time syntax highlighting for code blocks is powered by Shiki (one-dark-pro).
  • Sandboxed Widgets: The WidgetRenderer utilizes a secure iframe sandbox that interfaces with custom interactive HTML5 simulations or games using postMessages scoped to window.location.origin.

Export Options and Offline Support

  • Exports: Supports downloading lesson slides as PowerPoint (.pptx) via a customized pptxgenjs script, self-contained HTML pages, or full archive ZIP packages.
  • Caching: Employs an IndexedDB offline database (PLPro_Classrooms via Dexie) with a 24-hour TTL caching mechanism to prevent redundant API loads.
graph TB
  subgraph "Study Arena System"
    Engine["PlaybackEngine (Zustand)"]
    Canvas["WhiteboardCanvas (SVG + Shiki)"]
    TTS["Speech Synthesis Queue"]
    DB["Dexie (IndexedDB Caching)"]
  end
  subgraph "Backend API / Routes"
    Router["server/routes/ai-classroom.ts"]
    Gen["generator.ts (Multi-provider LLM)"]
  end
  Engine --> Canvas
  Engine --> TTS
  Engine --> DB
  Router --> Gen
  Engine -.-> Router
Loading

Diagram sources

  • playback-engine.ts
  • WhiteboardCanvas.tsx
  • ai-classroom.ts
  • generator.ts

Section sources

  • OPENMAIC_CHEAT_SHEET.md
  • playback-engine.ts
  • WhiteboardCanvas.tsx

Dependency Analysis

  • Backend depends on:
    • OpenAI SDK for chat completions and JSON response parsing.
    • Express routes for API exposure.
    • Session middleware for authentication.
  • Frontend depends on:
    • HTTP client for REST messaging.
    • WebSocket hook for real-time chat.
    • Local storage for AI tutor conversation persistence.
graph LR
A["client/src/pages/ai-tutor.tsx"] --> B["server/routes.ts"]
B --> C["server/lib/openai.ts"]
D["client/src/lib/chat-api.ts"] --> E["server/index.ts"]
F["client/src/hooks/use-chat-ws.ts"] --> E
Loading

Diagram sources

  • client/src/pages/ai-tutor.tsx
  • server/routes.ts
  • server/lib/openai.ts
  • client/src/lib/chat-api.ts
  • client/src/hooks/use-chat-ws.ts
  • server/index.ts

Section sources

  • server/routes.ts
  • server/lib/openai.ts
  • server/index.ts
  • client/src/pages/ai-tutor.tsx
  • client/src/lib/chat-api.ts
  • client/src/hooks/use-chat-ws.ts

Performance Considerations

  • Model selection: gpt-4o balances quality and cost for most tasks.
  • Prompt engineering: system prompts guide tone, domain expertise, and output format.
  • Response handling: enforce JSON output format and bounds checking to avoid downstream errors.
  • Fallback mechanisms: return sensible defaults when AI services are unavailable.
  • Cost optimization: minimize token usage by trimming context and using concise prompts.
  • Storage and caching: leverage local storage for UI state and Cassandra for durable message history.

[No sources needed since this section provides general guidance]

Troubleshooting Guide

Common issues and resolutions:

  • Missing OpenAI API key: The integration warns when the key is not set. Ensure the environment variable is configured.
  • AI chat errors: The endpoint catches and surfaces errors; verify network connectivity and API limits.
  • Evaluation failures: The evaluation function returns fallback values; inspect logs for underlying causes.
  • Session errors: Ensure session middleware is configured and cookies are accepted by the browser.
  • WebSocket reconnection: The chat hook implements exponential backoff; unauthorized closures prevent reconnection.

Section sources

  • server/lib/openai.ts
  • server/routes.ts
  • client/src/hooks/use-chat-ws.ts

Conclusion

PersonalLearningPro’s AI-powered features combine OpenAI-driven chat, evaluation, study planning, and performance analysis with a robust real-time messaging framework. The system emphasizes prompt engineering, structured JSON responses, and resilient fallbacks to ensure reliability. MessagePal complements these capabilities with scalable, role-based communication channels.

[No sources needed since this section summarizes without analyzing specific files]

Appendices

API Integration Patterns

  • AI Chat: POST /api/ai-chat with messages array; returns assistant content.
  • AI Evaluation: POST /api/evaluate with answerId; returns score, confidence, and feedback.
  • Messaging REST: Fetch channels, DMs, and messages; upload files.
  • WebSocket: Connect to real-time chat with typed events and presence.

Section sources

  • server/routes.ts
  • client/src/lib/chat-api.ts
  • client/src/hooks/use-chat-ws.ts

Prompt Engineering Strategies

  • System prompts define persona, domain expertise, and output expectations.
  • JSON response format requests improve parsing reliability.
  • Bounds checking ensures numeric values remain within expected ranges.

Section sources

  • server/lib/openai.ts
  • server/lib/openai.ts
  • server/lib/openai.ts
  • server/lib/openai.ts

Environment Configuration

  • OPENAI_API_KEY: Enables AI features.
  • SESSION_SECRET: Secures sessions in production.

Section sources

  • .env.example

Clone this wiki locally