FitBuddy - AI Personal Trainer

FitBuddy is an AI-powered personal trainer that provides real-time video and audio feedback on exercise posture and form. The agent can see users through their camera and offer helpful, encouraging guidance during workouts.

Features

• Real-time Video Analysis: See user posture and provide immediate feedback
• Voice Interaction: Natural conversation with American accent and friendly tone
• Exercise Guidance: Offer helpful feedback on posture and form
• Multimedia Sessions: Video streaming with bidirectional audio
• Session Management: Configurable session limits and timeouts

Architecture

• Framework: Built with vision-agents for multimedia AI agents
• Web Server: FastAPI for HTTP API endpoints
• Video Streaming: Stream.io for edge video transport
• AI Processing: OpenAI Realtime for language understanding
• Audio Processing: Deepgram for speech-to-text and text-to-speech
• Language: Python 3.12+ with dependency management via uv

Prerequisites

• Python 3.12 or higher
• uv (Python package manager)
• API keys for required services (see Configuration section)

Installation

1. Clone the repository:

   git clone <repository-url>
   cd fitbuddy

2. Install dependencies:

   uv sync

3. Activate the virtual environment:

   uv shell

Configuration

1. Copy environment template:

   cp .env.example .env

2. Edit .env file with your API keys:

   OPENAI_API_KEY=your_openai_api_key
   STREAM_API_KEY=your_stream_api_key_here
   STREAM_API_SECRET=your_stream_api_secret_here
   DEEPGRAM_API_KEY=your_deepgram_api_key_here

3. Optional configuration (default values shown):

   MAX_CONCURRENT_SESSIONS=5
   MAX_SESSIONS_PER_CALL=1
   MAX_SESSION_DURATION_SECONDS=3600
   AGENT_IDLE_TIMEOUT=120

Instructions

The instructions for the AI agent are in the personal_trainer.md file.

Usage

Running the Server

Start the FitBuddy in development mode:

uv run main.py run

If you want to run the API server (with API):

uv run main.py serve

The server will start and be ready to handle incoming calls and sessions.

How It Works

1. Session Start: When a user joins, the AI personal trainer greets them
2. Exercise Monitoring: The agent watches the user's posture through video
3. Real-time Feedback: Provides voice guidance on form and technique
4. Interactive Dialogue: Users can ask questions and receive responses
5. Session End: Automatically disconnects after timeout or when session completes

Agent Behavior

The AI personal trainer:

• Speaks with a friendly, encouraging tone
• Uses an American accent
• Focuses on posture and exercise form
• Provides constructive feedback
• Maintains engaging conversation

API Configuration

Session Limits

• MAX_CONCURRENT_SESSIONS: Maximum total concurrent agent sessions (default: 5)
• MAX_SESSIONS_PER_CALL: Agents per individual call (default: 1)
• MAX_SESSION_DURATION_SECONDS: Maximum session length in seconds (default: 3600)
• AGENT_IDLE_TIMEOUT: Disconnect after inactivity in seconds (default: 120)

Event Handling

The system tracks:

• User interruptions during agent speech
• Turn completion and speaking duration
• Session metrics and logs

Development

Project Structure

fitbuddy/
├── main.py              # Main application server
├── personal_trainer.md  # Agent instructions
├── .env.example         # Environment template
├── pyproject.toml       # Project dependencies
└── README.md           # This file

Dependencies

Key packages:

• vision-agents[deepgram,getstream,openai] - Core agent framework
• fastapi - Web server
• python-dotenv - Environment management
• deepgram - Audio processing
• openai - AI language model

Troubleshooting

Common Issues

1. API Key Errors: Ensure all required API keys are properly set in .env
2. Connection Issues: Check network connectivity and API service status
3. Audio/Video Problems: Verify camera and microphone permissions
4. Session Timeouts: Adjust timeout values in configuration if needed