Antigravity Racket Coach - AI-Driven Badminton Rally Coach

An AI-driven sports analytics coach built for the AMD Developer Hackathon: ACT II. This project combines Track 1 (hybrid token-efficient routing agent) and Track 3 (open product track) to analyze broadcast-style badminton singles matches.

By utilizing computer vision (YOLOv8 + ByteTrack) and audio signal processing, it tracks player positioning, extracts racquet hits, renders court heatmaps, computes tactical movement statistics, and uses a complexity-based router to escalate high-complexity rallies to a Gemma 2 model hosted on Fireworks AI.

---

🚀 Key Features

1. AI Player Tracking: Uses YOLOv8 with ByteTrack to track player coordinates, with linear gap interpolation to handle brief camera occlusions.
2. Interactive Court Calibration: Provides a 4-corner draggable canvas on the frontend, allowing users to calibrate the coordinate system dynamically to build a custom perspective homography matrix mapping pixels to real-world meters ($6.1\text{m} \times 13.4\text{m}$).
3. Audio-Visual Hit Detection: Cross-references audio onset transients (shuttlecock racquet hits detected via Librosa) with visual motion spikes (frame differencing in OpenCV) to attribute shots to a specific player.
4. Comprehensive Court Statistics: Computes cumulative distance, speeds, left/right court biases, and forecourt/rearcourt occupancy ratios.
5. Cost-Containment Complexity Router: Evaluates rally movement variance and hits. Low-complexity rallies (Score $< 4.0$) are handled locally via preloaded analytical templates. High-complexity rallies (Score $\ge 4.0$) are routed to Gemma via Fireworks AI, logging tokens and latencies.
6. Real-time Video Overlay: Overlays bounding boxes, directional trails, and fade-out hit triggers synced to video playback time.
7. Interactive 2D Heatmap Canvas: Renders a tennis-style badminton court grid showing player spatial density.
8. Token-Efficiency Dashboard: Displays routing history, average latencies, token consumption, and estimates total API costs saved by local routing.

---

🛠 Tech Stack

• Backend: FastAPI, Uvicorn, PyTorch (ROCm-compatible), Ultralytics (YOLOv8), OpenCV, Librosa, MoviePy, SciPy
• Frontend: React, Vite, Lucide Icons, Vanilla CSS (Premium Dark Theme, Glassmorphism, 100% Tailwind-free)
• Deployment: Docker, Docker Compose

---

📁 Repository Structure

├── backend/
│   ├── clips/                      # Preloaded videos and calibrations
│   │   ├── demo_rally_1.mp4
│   │   └── demo_rally_1_calibration.json
│   ├── pipeline/
│   │   ├── track_players.py        # Player tracking (YOLO + ByteTrack)
│   │   ├── court_calibration.py    # Perspective Homography mapping
│   │   ├── hit_detection.py        # Audio-visual hit alignment
│   │   ├── build_rally_data.py     # Unified JSON assembler
│   │   ├── stats.py                # Statistics & 2D heatmap compiler
│   │   └── router.py               # Token-efficiency routing decisions
│   ├── scripts/
│   │   ├── test_pipeline.py        # Run-anywhere command line pipeline validator
│   │   └── trim_video.py           # Extracts and trims raw clips
│   ├── api.py                      # FastAPI server endpoints
│   ├── gemma_client.py             # Fireworks AI client (Gemma Chat)
│   ├── requirements.txt            # Python dependencies
│   └── Dockerfile                  # Slim python container with GPU/CPU toggles
├── frontend/
│   ├── src/
│   │   ├── App.jsx                 # Single-page React application
│   │   ├── App.css                 # Custom CSS stylesheets (Vanilla CSS)
│   │   └── main.jsx                # Vite entrypoint
│   ├── nginx.conf                  # Nginx server reverse-proxy settings
│   └── Dockerfile                  # Multi-stage React compiler + Nginx server
├── docker-compose.yml              # Combined services orchestrator
└── README.md

---

⚙️ Setup & Installation

1. Environment Setup

Create a .env file inside the backend/ directory (or set them in your system environment) with the following values:

FIREWORKS_API_KEY=your_fireworks_api_key_here
GEMMA_MODEL=accounts/fireworks/models/gemma2-9b-it

Note: If no API key is provided, the backend falls back gracefully to a detailed manual stats report instead of raising a 500 error.

---

🐳 Running with Docker (Recommended)

To run the entire containerized application locally:

# From the root directory:
docker-compose up --build

The services will build and start:

• Frontend App: Accessible at http://localhost:3000
• FastAPI API: Accessible at http://localhost:8000

AMD Developer Cloud / ROCm GPU Execution

By default, the backend container builds with standard CPU libraries. To enable AMD ROCm GPU acceleration inside the Docker container:

1. Open backend/Dockerfile and uncomment the ROCm PyTorch installation line:

   RUN pip install --no-cache-dir torch torchvision --index-url https://download.pytorch.org/whl/rocm6.0 && pip install --no-cache-dir -r requirements.txt

2. Re-run docker-compose up --build.

---

💻 Running Locally for Development

1. Run the Backend

Ensure you have ffmpeg installed on your host system (e.g. brew install ffmpeg).

cd backend
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

# Run the API server
python3 api.py

2. Run the Frontend

cd frontend
npm install
npm run dev

The Vite dev server will host the frontend at http://localhost:5173.

---

🧪 Pipeline CLI Validation

You can run the entire analysis pipeline on a video file directly from the terminal without launching the servers:

cd backend
source .venv/bin/activate
python scripts/test_pipeline.py clips/demo_rally_1.mp4

This runs tracking, calibration, hit detection, stats, and routing, printing a comprehensive text summary of the results to the terminal.