Hoop MSSL

Introduction

"Hoop-MSSL: Multitask self-supervised representation learning on basketball spatiotemporal data" is an article that documents the structure of a transformer based model that learns about the game of basketball through movement data of individual players. This paper proposes a BERT-like encoder-decoder structured model which learns on 3 different tasks and proves that the encoder is highly generalizable to other decoders/tasks.

Data

• a batch is a set of possessions
• each possession has trajectory of 11 agents (players and the ball)
• each trajectory is a sequence of 121 timesteps which stores (x, y, v: speed)
• data = (N x 11 x 121 x 3) where N is the batch size

Model Structure

1. Augmentation: disorder augmentation and random masking (80%) to produce two views of each possession (32x11x121x3) -> (64x11x121x3)
2. Projection: project the 3 features to a larger dimension space of 128 (64x11x121x3) -> (64x11x121x128)
3. Positional Encoding
4. Temporal Transformer: explores the relationship between different timesteps within individual possessions
5. Spatial Transformer: explores the relationship between different agents within individual possessions
6. Decoders: calculates the loss of the model using three tasks
   1. Motion Reconstruction: MLP 128->3(x, y, v)
      Loss: MSELoss
   2. Player Role Identification: 1d pooling (collapse on timestep dimension), MLP (64,11,121,128) -> (64x11x2)(offense, defense)
      Loss: Binary Cross Entropy
   3. Contrastive learning: 2d pooling (collapse on agent and timestep dimension), MLP (64x11x121x128) -> (64x128)
      Loss: NTXent (measures the similarity between the two views of the same possession)
7. Custom decoders: takes in contextualized vector and peforms a task
1. Player height
2. Player Position
3. All-star apperance

Final Notes

• In the paper, the researchers categorized decoders into player-level, play-level, and action-level. Because of the difficulty in labeling actions like pick-and-role and plays like pistol and 5-out, I only trained player-level decoders.
• I have tested the code with small fractions of the data but didn't train the model on the whole dataset as it will take up too much credits