CourtVision: Basketball Analytics Reimagined

🎥 Sample Output

Click to play the full video on Youtube:

CourtVision (formerly Basketball-CV) is an all-in-one computer vision toolkit for basketball analytics, transforming raw video footage into actionable data and visualizations. Whether you’re a coach, analyst, researcher, or enthusiast, CourtVision helps you:

• Automatically detect and track game objects (players, ball, hoops, scoreboard).
• Classify players into teams via jersey-color zero-shot classification.
• Compute and visualize possession, passes, interceptions, speed, and distance.
• Generate an overhead tactical map using homography and pose estimation.
• Annotate and export richly detailed video outputs for presentations or further analysis.

From the court to your screen—see the game like never before.

---

🚀 Key Features

• Object Detection & Tracking
  • YOLOv8x for high-accuracy detection of 7 classes: Ball, Clock, Hoop, Overlay, Player, Referee, Scoreboard.
  • ByteTrack maintains consistent player IDs across frames for longitudinal analysis.
• Team Assignment
  • CLIP-based zero-shot jersey-color classification to label team membership.
  • Cache assignments and reset every 50 frames to handle jersey occlusions.
• Ball Possession & Event Detection
  • Ball acquisition determined by containment ratio and distance heuristics.
  • Pass and interception detection by comparing successive holders and team IDs (65–70% accuracy).
• Tactical Overhead View
  • Pose model keypoints validate court landmarks.
  • Homography maps player foot positions into a 2D tactical court inset.
  • Suppression of invalid keypoints ensures robustness.
• Player Metrics
  • Cumulative distance traveled and smoothed speed (km/h) per player.
  • Interpolation fills missing detections for continuous tracking.
• Visualization Modules (drawers/)
  • BallTracksDrawer, PlayerTracksDrawer, TacticalViewDrawer, SpeedAndDistanceDrawer, PassInterceptionsDrawer, TeamBallControlDrawer.
  • Modular design lets you pick & choose overlays.
• Stub Caching
  • Save intermediate results (detections, tracks, assignments) to disk, skipping reprocessing on reruns.

---

📁 Project Structure

project_root/
├── models/                         # Trained model weights (YOLO, CLIP)
├── utils/                          # Core helpers: video IO, geometry, stubs
│   ├── video_utils.py              # read_video, save_video
│   ├── geometry_utils.py           # bbox center, width, distance
│   └── stub_utils.py               # save_stub, read_stub
├── ball_tracker.py                 # YOLO-based ball detection + interpolation
├── player_tracker.py               # YOLO + ByteTrack for players
├── team_assigner.py                # CLIP-based jersey classification
├── tactical_view_converter.py      # Pose validation & homography mapping
├── drawers/                        # Visualization components
│   ├── ball_tracks_drawer.py
│   ├── player_tracks_drawer.py
│   ├── tactical_view_drawer.py
│   ├── speed_distance_drawer.py
│   ├── pass_interceptions_drawer.py
│   └── team_ball_control_drawer.py
├── stubs/                          # Cached pickle files
├── input_videos/                   # Raw game footage
├── output_videos/                  # Annotated exports
└── README.md                       # You are here

---

🏗 Development Roadmap

This section outlines every major step in building CourtVision:

1. YOLO Architecture: Selected YOLOv8x for state-of-the-art detection speed and accuracy.
2. Ultralytics Integration: Leveraged their API for streamlined model loading and inference.
3. Initial Testing: Ran base YOLOv8x on basketball footage—detected players & ball out-of-the-box.
4. Transfer Learning: Fine-tuned YOLOv8x on custom-labeled basketball dataset via RoboFlow.
5. RoboFlow: Annotated images, trained models in Google Colab (free GPU), exported weights offline.
6. Detection Classes: Defined 7 classes: Ball, Clock, Hoop, Overlay, Player, Referee, Scoreboard.
7. Tracking Layer: Added ByteTrack to assign persistent IDs to detected players.
8. Video Processing: Built video I/O utilities (cv2.VideoCapture, VideoWriter).
9. Supervision Library: Simplified conversion between Ultralytics detections and trackers.
10. Stub Caching: Cached detections, tracks, and team assignments to accelerate development loops.
11. Data Interpolation: Used pandas.interpolate() and bfill() to fill detection gaps for smooth tracking.
12. Team Classification: Employed CLIP zero-shot classification for jersey-color-based team labeling.
13. Cache Reset Strategy: Cleared jersey assignment cache every 50 frames to handle occlusions & new players.
14. Ball Acquisition Logic: Calculated containment ratio and minimum distance to determine possessor.
15. Possession Visualization: Drew translucent triangles to indicate current ball controller.
16. Pass & Interception Detection: Identified events by comparing successive possession holders & team IDs.
17. Pose Model for Court Keypoints: Detected court landmarks to establish homography reference points.
18. Homography & Tactical View: Transformed camera view to overhead court diagram inset in video frames.
19. Player Position Mapping: Projected player foot positions into tactical view coordinates.
20. Distance Calculation: Converted pixel movements to meters using court scale and Euclidean distance.
21. Speed Estimation: Computed km/h over sliding windows of frames for smooth speed values.
22. Final Annotation: Combined trackers, metrics, and tactical view overlays into annotated video outputs.

---

🌟 Get Started

1. Clone this repo and install dependencies.
2. Download or train your YOLO & CLIP models into models/.
3. Prepare a folder under input_videos/ with your footage.
4. Environment Create a virtual environment.

python3 -m venv .venv
source .venv/bin/activate

5. Install Install dependencies.

pip install opencv-python numpy pandas ultralytics supervision transformers torch Pillow

6. Run Run the main script.

python main.py

7. Explore the annotated videos in output_videos/ and visual modules in drawers/.

📺 Demo

---

📄 License

MIT License © Manan Gulati