training.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Training and datasets\n",
    "\n",
    "Finally, we also release our datasets and models. \n",
    "\n",
    "## Datasets\n",
    "\n",
    "### Links:\n",
    "\n",
    "| Model                  | Description                               | Link                                                                        |\n",
    "|------------------------|-------------------------------------------|-----------------------------------------------------------------------------|\n",
    "| 11_vs_11_selfplay_last | EDG agent                                 | https://storage.googleapis.com/narya-bucket-1/models/11_vs_11_selfplay_last |\n",
    "| deep_homo_model.h5     | Direct Homography estimation Architecture | https://storage.googleapis.com/narya-bucket-1/models/deep_homo_model.h5     |\n",
    "| deep_homo_model_1.h5   | Direct Homography estimation Weights      | https://storage.googleapis.com/narya-bucket-1/models/deep_homo_model_1.h5   |\n",
    "| keypoint_detector.h5   | Keypoints detection Weights               | https://storage.googleapis.com/narya-bucket-1/models/keypoint_detector.h5   |\n",
    "| player_reid.pth        | Player Embedding Weights                  | https://storage.googleapis.com/narya-bucket-1/models/player_reid.pth        |\n",
    "| player_tracker.params  | Player & Ball detection Weights           | https://storage.googleapis.com/narya-bucket-1/models/player_tracker.params  |\n",
    "\n",
    "The datasets can be downloaded at: \n",
    "\n",
    "| Dataset                | Description                                                             | Link                                                                         |\n",
    "|------------------------|-------------------------------------------------------------------------|------------------------------------------------------------------------------|\n",
    "| homography_dataset.zip | Homography Dataset (image,homography)                                   | https://storage.googleapis.com/narya-bucket-1/dataset/homography_dataset.zip |\n",
    "| keypoints_dataset.zip  | Keypoint Dataset (image,list of mask)                                   | https://storage.googleapis.com/narya-bucket-1/dataset/keypoints_dataset.zip  |\n",
    "| tracking_dataset.zip   | Tracking Dataset in VOC format (image, bounding boxes for players/ball) | https://storage.googleapis.com/narya-bucket-1/dataset/tracking_dataset.zip   |\n",
    "\n",
    "### Overview \n",
    "\n",
    "*Homography Dataset*: The homography dataset is made of pair of images,matrix in a .jpg,.npy format. The matrix is the homography associated with the image. They are normalized, meaning that homography[2,2] == 1.\n",
    "\n",
    "*Keypoints Dataset*: We give here pair of images,xml file. The .xml files are made of the coordinates of each available keypoints on the image. We built utils function to read these files, and do so automaticaly in our Dataset class. \n",
    "\n",
    "*Tracking Dataset*: Pair of images,xml files in a VOC format.\n",
    "\n",
    "## Training \n",
    "\n",
    "Finally, we give here a quick tour of our training scripts. \n",
    "\n",
    "We start by creating a model: \n",
    "\n",
    "```python3\n",
    "full_model = KeypointDetectorModel(\n",
    "    backbone=opt.backbone, num_classes=29, input_shape=(320, 320),\n",
    ")\n",
    "\n",
    "if opt.weights is not None:\n",
    "    full_model.load_weights(opt.weights)\n",
    "```\n",
    "\n",
    "We then create a loss function and an optimizer:\n",
    "\n",
    "```python3\n",
    "# define optomizer\n",
    "optim = keras.optimizers.Adam(opt.lr)\n",
    "# define loss function\n",
    "dice_loss = sm.losses.DiceLoss()\n",
    "focal_loss = sm.losses.CategoricalFocalLoss()\n",
    "total_loss = dice_loss + (1 * focal_loss)\n",
    "\n",
    "metrics = [sm.metrics.IOUScore(threshold=0.5), sm.metrics.FScore(threshold=0.5)]\n",
    "\n",
    "# compile keras model with defined optimozer, loss and metrics\n",
    "model.compile(optim, total_loss, metrics)\n",
    "\n",
    "callbacks = [\n",
    "    keras.callbacks.ModelCheckpoint(\n",
    "        name_model, save_weights_only=True, save_best_only=True, mode=\"min\"\n",
    "    ),\n",
    "    keras.callbacks.ReduceLROnPlateau(\n",
    "        patience=10, verbose=1, cooldown=10, min_lr=0.00000001\n",
    "    ),\n",
    "]\n",
    "\n",
    "model.summary()\n",
    "```\n",
    "\n",
    "We can easily build a Dataset and a Dataloader (handling batches):\n",
    "\n",
    "```python3\n",
    "x_train_dir = os.path.join(opt.data_dir, opt.x_train_dir)\n",
    "kp_train_dir = os.path.join(opt.data_dir, opt.y_train_dir)\n",
    "x_test_dir = os.path.join(opt.data_dir, opt.x_test_dir)\n",
    "kp_test_dir = os.path.join(opt.data_dir, opt.y_test_dir)\n",
    "\n",
    "full_dataset = KeyPointDatasetBuilder(\n",
    "    img_train_dir=x_train_dir,\n",
    "    img_test_dir=x_test_dir,\n",
    "    mask_train_dir=kp_train_dir,\n",
    "    mask_test_dir=kp_test_dir,\n",
    "    batch_size=opt.batch_size,\n",
    "    preprocess_input=preprocessing_fn,\n",
    ")\n",
    "\n",
    "train_dataloader, valid_dataloader = full_dataset._get_dataloader()\n",
    "```\n",
    "\n",
    "Finally, easily launch a training with:\n",
    "\n",
    "```python3\n",
    "model.fit_generator(\n",
    "    train_dataloader,\n",
    "    steps_per_epoch=len(train_dataloader),\n",
    "    epochs=opt.epochs,\n",
    "    callbacks=callbacks,\n",
    "    validation_data=valid_dataloader,\n",
    "    validation_steps=len(valid_dataloader),\n",
    ")\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}