authors/marcos-dv/educational/NFL_Team_Rating.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SWWWN49eoi4Z"
      },
      "source": [
        "# NFL Team Rating\n",
        "[![NFL_Team_Rating.ipynb](https://img.shields.io/badge/github-%23121011.svg?logo=github)](https://github.com/ampl/colab.ampl.com/blob/master/authors/marcos-dv/educational/NFL_Team_Rating.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ampl/colab.ampl.com/blob/master/authors/marcos-dv/educational/NFL_Team_Rating.ipynb) [![Kaggle](https://kaggle.com/static/images/open-in-kaggle.svg)](https://kaggle.com/kernels/welcome?src=https://github.com/ampl/colab.ampl.com/blob/master/authors/marcos-dv/educational/NFL_Team_Rating.ipynb) [![Gradient](https://assets.paperspace.io/img/gradient-badge.svg)](https://console.paperspace.com/github/ampl/colab.ampl.com/blob/master/authors/marcos-dv/educational/NFL_Team_Rating.ipynb) [![Open In SageMaker Studio Lab](https://studiolab.sagemaker.aws/studiolab.svg)](https://studiolab.sagemaker.aws/import/github/ampl/colab.ampl.com/blob/master/authors/marcos-dv/educational/NFL_Team_Rating.ipynb) [![Hits](https://h.ampl.com/https://github.com/ampl/colab.ampl.com/blob/master/authors/marcos-dv/educational/NFL_Team_Rating.ipynb)](https://colab.ampl.com)\n",
        "\n",
        "Description: NFL Team Rating problem from the Analytical Decision Modeling course at the Arizona State University.\n",
        "\n",
        "Tags: educational, quadratic, amplpy, gurobi\n",
        "\n",
        "Notebook author: Yimin Wang <<yimin_wang@asu.edu>>, Marcos Dominguez Velad <<marcos@ampl.com>>\n",
        "\n",
        "Model author: Yimin Wang <<yimin_wang@asu.edu>>\n",
        "\n",
        "References:\n",
        "1. Analytical Decision Modeling course at W. P. Carey School of Business. Syllabus:\n",
        "  https://catalog.apps.asu.edu/catalog/classes/classlist?keywords=86683&searchType=all&term=2237&collapse=Y"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yTw30bfQoi4a"
      },
      "source": [
        "## Objective and Prerequisites\n",
        "\n",
        "This NFL team rating problem shows you how to determine the optimal rating of sporting teams based on their past performances. We use the root mean squared error (RMSE) as a criteria to determine what set of ratings is most accurate. The objectives of the sports rating problem are:\n",
        "\n",
        "* Find the best possible set of ratings for all teams,\n",
        "* Encorporate possible home team advantages,\n",
        "* Minimize the errors when predicting outcomes of future matches, and\n",
        "* Ensure that the ratings are consistent and identifiable.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "62q-NwBMoi4c"
      },
      "source": [
        "---\n",
        "## Problem Description\n",
        "\n",
        "![picture](https://drive.google.com/uc?id=1zpCeYLlNZePdlL_S9mi9nlrkUSWYticg)\n",
        "\n",
        "We consider the NFL games from 32 teams. The teams are indexed 1 to 32, for example, team 1 is Arizon, team 2 is Atlanta, and so on.\n",
        "\n",
        "|\tTeam Number\t|\tTeam name\t|\tTeam Number\t|\tTeam name\t|\n",
        "|--- | --- | --- | --- |\n",
        "|\t1 \t|\tArizona Cardinals\t|\t17 \t|\tMiami Dolphins\t|\n",
        "|\t2 \t|\tAtlanta Falcons\t|\t18 \t|\tMinnesota Vikings\t|\n",
        "|\t3 \t|\tBaltimore Ravens\t|\t19 \t|\tNew England Patriots\t|\n",
        "|\t4 \t|\tBuffalo Bills\t|\t20 \t|\tNew Orleans Saints\t|\n",
        "|\t5 \t|\tCarolina Panthers\t|\t21 \t|\tNew York Giants\t|\n",
        "|\t6 \t|\tChicago Bears\t|\t22 \t|\tNew York Jets\t|\n",
        "|\t7 \t|\tCincinnati Bengals\t|\t23 \t|\tOakland Raiders\t|\n",
        "|\t8 \t|\tCleveland Browns\t|\t24 \t|\tPhiladelphia Eagles\t|\n",
        "|\t9 \t|\tDallas Cowboys\t|\t25 \t|\tPittsburgh Steelers\t|\n",
        "|\t10 \t|\tDenver Broncos\t|\t26 \t|\tSt. Louis Rams\t|\n",
        "|\t11 \t|\tDetroit Lions\t|\t27 \t|\tSan Diego Chargers\t|\n",
        "|\t12 \t|\tGreen Bay Packers\t|\t28 \t|\tSan Francisco 49ers\t|\n",
        "|\t13 \t|\tHouston Texans\t|\t29 \t|\tSeattle Seahawks\t|\n",
        "|\t14 \t|\tIndianapolis Colts\t|\t30 \t|\tTampa Bay Buccaneers\t|\n",
        "|\t15 \t|\tJacksonville Jaguars\t|\t31 \t|\tTennessee Titans\t|\n",
        "|\t16 \t|\tKansas City Chiefs\t|\t32 \t|\tWashington Redskins\t|\n",
        "\n",
        "Table below illustrates some of the results of the 256 regular season NFL games from the 2015 season. The first game is team 10 Denver versus team 3 Baltimore, played at Denver. Denver won the game by a score of 49 to 27, and the point spread (home team score minus vistor team score) is calculated as the difference between the home team score and the visiting team score. In the above example, the point spread is 49-27=22.\n",
        "\n",
        "\n",
        "|\tWeek\t|\tMatch\t|\tHome team index\t|\tVisiting team index\t|\tHome team score\t|\tVisiting team score\t|\tPoint spread\t|\n",
        "| --- | --- | --- | --- | --- | --- | --- |\n",
        "|\t1\t|\t1\t|\t10 \t|\t3 \t|\t49 \t|\t27 \t|\t22 \t|\n",
        "|\t1\t|\t2\t|\t5 \t|\t29 \t|\t7 \t|\t12 \t|\t-5 \t|\n",
        "|\t1\t|\t3\t|\t15 \t|\t16 \t|\t2 \t|\t28 \t|\t-26 \t|\n",
        "|\t1\t|\t4\t|\t22 \t|\t30 \t|\t18 \t|\t17 \t|\t1 \t|\n",
        "|\t1\t|\t5\t|\t26 \t|\t1 \t|\t27 \t|\t24 \t|\t3 \t|\n",
        "|\t1\t|\t6\t|\t9 \t|\t21 \t|\t36 \t|\t31 \t|\t5 \t|\n",
        "|\t1\t|\t7\t|\t28 \t|\t12 \t|\t34 \t|\t28 \t|\t6 \t|\n",
        "|\t1\t|\t8\t|\t20 \t|\t2 \t|\t23 \t|\t17 \t|\t6 \t|\n",
        "|\t1\t|\t9\t|\t14 \t|\t23 \t|\t21 \t|\t17 \t|\t4 \t|\n",
        "|\t1\t|\t10\t|\t6 \t|\t7 \t|\t24 \t|\t21 \t|\t3 \t|\n",
        "|\t1\t|\t11\t|\t11 \t|\t18 \t|\t34 \t|\t24 \t|\t10 \t|\n",
        "|\t1\t|\t12\t|\t4 \t|\t19 \t|\t21 \t|\t23 \t|\t-2 \t|\n",
        "|\t1\t|\t13\t|\t25 \t|\t31 \t|\t9 \t|\t16 \t|\t-7 \t|\n",
        "|\t...\t|\t...\t|\t... \t|\t... \t|\t...\t|\t... \t|\t... \t|\n",
        "|\t...\t|\t...\t|\t... \t|\t... \t|\t...\t|\t... \t|\t... \t|\n",
        "|\t...\t|\t...\t|\t... \t|\t... \t|\t...\t|\t... \t|\t... \t|\n",
        "|\t17\t|\t256\t|\t17 \t|\t22 \t|\t7 \t|\t20 \t|\t-13 \t|\n",
        "\n",
        "We would like to find a best rating system that most accurately predicts the matches."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MYpg58_8oi4e"
      },
      "source": [
        "## Model Formulation\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "\n",
        "### Indices\n",
        "\n",
        "$i,j \\in \\{1..32\\}$: Index to represent teams\n",
        "\n",
        "$t \\in \\{1..256\\}$: Index to represent different games\n",
        "\n",
        "### Parameters\n",
        "\n",
        "$p_{ijt}$: point spread between team $i$ and team $j$ in game $t$\n",
        "\n",
        "### Decision Variables\n",
        "\n",
        "$x_{i}$: Ratings of team $i$, $i \\in \\{1..32\\}$\n",
        "\n",
        "$y$: Home team advantage\n",
        "\n",
        "### Objective Function\n",
        "\n",
        "- **Prediction Accuracy**. We want to minimize the prediction error.\n",
        "\n",
        "\n",
        "\\begin{equation}\n",
        "\\text{Min}_{x_{i},y} \\quad \\frac{\\sqrt{\\sum_{t \\in \\{1..256\\}} (x_i + y - x_j - p_{ijt})^2}}{256}\n",
        "\\tag{0}\n",
        "\\end{equation}\n",
        "\n",
        "### Constraints\n",
        "\n",
        "\\begin{equation}\n",
        "\\sum_i x_i = 32*85 \\ \\ \\ \\text{(fix average rating to be 85)}\n",
        "\\tag{1}\n",
        "\\end{equation}\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LrdbknV1oi4f"
      },
      "source": [
        "---\n",
        "\n",
        "## Python Implementation\n",
        "\n",
        "We now import the AMPL Python Module and other Python libraries."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2tPyWxhSLLYl",
        "outputId": "0ae0876f-1eb5-4d34-a524-86b61fa5f916"
      },
      "outputs": [],
      "source": [
        "# Install dependencies\n",
        "%pip install -q amplpy"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85,
          "referenced_widgets": [
            "a6e234808964470692b42e39cc3e28fe",
            "cd5610257054498a8fac67b099383d7d",
            "ed42846dfe914971b77c266371bf9ff6",
            "74968e1bf66b4f868e6b3b0390fe1e3d",
            "4971a28d89274cadac6a678b721666e6",
            "0b63ce492bcb432f92f99f2d70261f26",
            "cf98cae131e64d8c900a900ada200f06",
            "71916f0f6883405a99c0727f74fc675d",
            "ebda6bb9c85940d28d428e930265f144",
            "b867fda15a7e421d85c618f38aba38dd",
            "b7c841ecb90e4b66af31c410f09160b5",
            "3d192916823b446485dac899441f869e",
            "9e029c37ed8b461f9483dd7ced76050b"
          ]
        },
        "id": "vZuT7rzcoi4g",
        "outputId": "4ef0a09e-12de-4a16-9fbd-23f2d5e35e50"
      },
      "outputs": [],
      "source": [
        "# Google Colab & Kaggle integration\n",
        "from amplpy import AMPL, ampl_notebook\n",
        "\n",
        "ampl = ampl_notebook(\n",
        "    modules=[\"gurobi\"],  # modules to install\n",
        "    license_uuid=\"default\",  # license to use\n",
        ")  # instantiate AMPL object and register magics"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nzcjnZRRBynN"
      },
      "source": [
        "### Set up the inputs"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "dF93AcQwk1vy"
      },
      "outputs": [],
      "source": [
        "#####################################################\n",
        "#                    Model Formulation\n",
        "#####################################################\n",
        "\n",
        "# from 1 to 32\n",
        "team = [*range(1, 33)]\n",
        "# from 0 to 255\n",
        "game = [*range(1, 257)]\n",
        "\n",
        "team_label = [\n",
        "    \"Arizona Cardinals\",\n",
        "    \"Atlanta Falcons\",\n",
        "    \"Baltimore Ravens\",\n",
        "    \"Buffalo Bills\",\n",
        "    \"Carolina Panthers\",\n",
        "    \"Chicago Bears\",\n",
        "    \"Cincinnati Bengals\",\n",
        "    \"Cleveland Browns\",\n",
        "    \"Dallas Cowboys\",\n",
        "    \"Denver Broncos\",\n",
        "    \"Detroit Lions\",\n",
        "    \"Green Bay Packers\",\n",
        "    \"Houston Texans\",\n",
        "    \"Indianapolis Colts\",\n",
        "    \"Jacksonville Jaguars\",\n",
        "    \"Kansas City Chiefs\",\n",
        "    \"Miami Dolphins\",\n",
        "    \"Minnesota Vikings\",\n",
        "    \"New England Patriots\",\n",
        "    \"New Orleans Saints\",\n",
        "    \"New York Giants\",\n",
        "    \"New York Jets\",\n",
        "    \"Oakland Raiders\",\n",
        "    \"Philadelphia Eagles\",\n",
        "    \"Pittsburgh Steelers\",\n",
        "    \"St. Louis Rams\",\n",
        "    \"San Diego Chargers\",\n",
        "    \"San Francisco 49ers\",\n",
        "    \"Seattle Seahawks\",\n",
        "    \"Tampa Bay Buccaneers\",\n",
        "    \"Tennessee Titans\",\n",
        "    \"Washington Redskins\",\n",
        "]\n",
        "\n",
        "# past game performance\n",
        "p = [\n",
        "    [10, 3, 22],\n",
        "    [5, 29, -5],\n",
        "    [15, 16, -26],\n",
        "    [22, 30, 1],\n",
        "    [26, 1, 3],\n",
        "    [9, 21, 5],\n",
        "    [28, 12, 6],\n",
        "    [20, 2, 6],\n",
        "    [14, 23, 4],\n",
        "    [6, 7, 3],\n",
        "    [11, 18, 10],\n",
        "    [4, 19, -2],\n",
        "    [25, 31, -7],\n",
        "    [8, 17, -13],\n",
        "    [32, 24, -6],\n",
        "    [27, 13, -3],\n",
        "    [19, 22, 3],\n",
        "    [1, 11, 4],\n",
        "    [29, 28, 26],\n",
        "    [30, 20, -2],\n",
        "    [13, 31, 6],\n",
        "    [21, 10, -18],\n",
        "    [2, 26, 7],\n",
        "    [6, 18, 1],\n",
        "    [16, 9, 1],\n",
        "    [14, 17, -4],\n",
        "    [3, 8, 8],\n",
        "    [23, 15, 10],\n",
        "    [24, 27, -3],\n",
        "    [4, 5, 1],\n",
        "    [12, 32, 18],\n",
        "    [7, 25, 10],\n",
        "    [24, 16, -10],\n",
        "    [5, 21, 38],\n",
        "    [22, 4, 7],\n",
        "    [25, 6, -17],\n",
        "    [29, 15, 28],\n",
        "    [18, 8, -4],\n",
        "    [17, 2, 4],\n",
        "    [7, 12, 4],\n",
        "    [32, 11, -7],\n",
        "    [28, 14, -20],\n",
        "    [3, 13, 21],\n",
        "    [19, 30, 20],\n",
        "    [20, 1, 24],\n",
        "    [31, 27, 3],\n",
        "    [9, 26, 24],\n",
        "    [10, 23, 16],\n",
        "    [26, 28, -24],\n",
        "    [2, 19, -7],\n",
        "    [16, 21, 24],\n",
        "    [4, 3, 3],\n",
        "    [13, 29, -3],\n",
        "    [18, 25, 7],\n",
        "    [23, 32, -10],\n",
        "    [30, 1, -3],\n",
        "    [8, 7, 11],\n",
        "    [27, 9, 9],\n",
        "    [10, 24, 32],\n",
        "    [11, 6, 8],\n",
        "    [15, 14, -34],\n",
        "    [31, 22, 25],\n",
        "    [20, 17, 21],\n",
        "    [8, 4, 13],\n",
        "    [1, 5, 16],\n",
        "    [12, 11, 13],\n",
        "    [14, 29, 6],\n",
        "    [23, 27, 10],\n",
        "    [9, 10, -3],\n",
        "    [28, 13, 31],\n",
        "    [6, 20, -8],\n",
        "    [21, 24, -15],\n",
        "    [17, 3, -3],\n",
        "    [31, 16, -9],\n",
        "    [26, 15, 14],\n",
        "    [7, 19, 7],\n",
        "    [2, 22, -2],\n",
        "    [6, 21, 6],\n",
        "    [3, 12, -2],\n",
        "    [18, 5, -25],\n",
        "    [28, 1, 12],\n",
        "    [19, 20, 3],\n",
        "    [22, 25, -13],\n",
        "    [8, 11, -14],\n",
        "    [16, 23, 17],\n",
        "    [13, 26, -25],\n",
        "    [29, 31, 7],\n",
        "    [10, 15, 16],\n",
        "    [4, 7, -3],\n",
        "    [30, 24, -11],\n",
        "    [9, 32, 15],\n",
        "    [27, 14, 10],\n",
        "    [1, 29, -12],\n",
        "    [5, 26, 15],\n",
        "    [2, 30, 8],\n",
        "    [25, 3, 3],\n",
        "    [22, 19, 3],\n",
        "    [11, 7, -3],\n",
        "    [12, 8, 18],\n",
        "    [14, 10, 6],\n",
        "    [32, 6, 4],\n",
        "    [15, 27, -18],\n",
        "    [17, 4, -2],\n",
        "    [31, 28, -14],\n",
        "    [16, 13, 1],\n",
        "    [24, 9, -14],\n",
        "    [21, 18, 16],\n",
        "    [30, 5, -18],\n",
        "    [18, 12, -13],\n",
        "    [23, 25, 3],\n",
        "    [1, 2, 14],\n",
        "    [19, 17, 10],\n",
        "    [11, 9, 1],\n",
        "    [20, 4, 18],\n",
        "    [15, 28, -32],\n",
        "    [10, 32, 24],\n",
        "    [7, 22, 40],\n",
        "    [24, 21, -8],\n",
        "    [16, 8, 6],\n",
        "    [26, 29, -5],\n",
        "    [17, 7, 2],\n",
        "    [26, 31, -7],\n",
        "    [5, 2, 24],\n",
        "    [13, 14, -3],\n",
        "    [32, 27, 6],\n",
        "    [22, 20, 6],\n",
        "    [8, 3, 6],\n",
        "    [19, 25, 24],\n",
        "    [9, 18, 4],\n",
        "    [4, 16, -10],\n",
        "    [23, 24, -29],\n",
        "    [29, 30, 3],\n",
        "    [12, 6, -7],\n",
        "    [18, 32, 7],\n",
        "    [14, 26, -30],\n",
        "    [21, 23, 4],\n",
        "    [1, 13, 3],\n",
        "    [28, 5, -1],\n",
        "    [27, 10, -8],\n",
        "    [6, 11, -2],\n",
        "    [31, 15, -2],\n",
        "    [2, 29, -23],\n",
        "    [12, 24, -14],\n",
        "    [25, 4, 13],\n",
        "    [3, 7, 3],\n",
        "    [20, 9, 32],\n",
        "    [30, 17, 3],\n",
        "    [31, 14, -3],\n",
        "    [29, 18, 21],\n",
        "    [6, 3, 3],\n",
        "    [21, 12, 14],\n",
        "    [13, 23, -5],\n",
        "    [30, 2, 13],\n",
        "    [15, 1, -13],\n",
        "    [25, 11, 10],\n",
        "    [20, 28, 3],\n",
        "    [17, 27, 4],\n",
        "    [10, 16, 10],\n",
        "    [24, 32, 8],\n",
        "    [4, 22, 23],\n",
        "    [7, 8, 21],\n",
        "    [5, 19, 4],\n",
        "    [2, 20, -4],\n",
        "    [13, 15, -7],\n",
        "    [11, 30, -3],\n",
        "    [16, 27, -3],\n",
        "    [12, 18, 0],\n",
        "    [21, 9, -3],\n",
        "    [8, 25, -16],\n",
        "    [26, 6, 21],\n",
        "    [19, 10, 3],\n",
        "    [1, 14, 29],\n",
        "    [23, 31, -4],\n",
        "    [17, 5, -4],\n",
        "    [3, 22, 16],\n",
        "    [32, 28, -21],\n",
        "    [3, 25, 2],\n",
        "    [11, 12, 30],\n",
        "    [9, 23, 7],\n",
        "    [4, 2, -3],\n",
        "    [27, 7, -7],\n",
        "    [24, 1, 3],\n",
        "    [18, 6, 3],\n",
        "    [32, 21, -7],\n",
        "    [14, 31, 8],\n",
        "    [22, 17, -20],\n",
        "    [28, 26, 10],\n",
        "    [8, 15, -4],\n",
        "    [5, 30, 21],\n",
        "    [16, 10, -7],\n",
        "    [13, 19, -3],\n",
        "    [29, 20, 27],\n",
        "    [15, 13, 7],\n",
        "    [25, 17, -6],\n",
        "    [7, 14, 14],\n",
        "    [1, 26, 20],\n",
        "    [27, 21, 23],\n",
        "    [28, 29, 2],\n",
        "    [10, 31, 23],\n",
        "    [3, 18, 3],\n",
        "    [30, 4, 21],\n",
        "    [20, 5, 18],\n",
        "    [32, 16, -35],\n",
        "    [12, 2, 1],\n",
        "    [19, 8, 1],\n",
        "    [22, 23, 10],\n",
        "    [24, 11, 14],\n",
        "    [6, 9, 17],\n",
        "    [10, 27, -7],\n",
        "    [8, 6, -7],\n",
        "    [31, 1, -3],\n",
        "    [17, 19, 4],\n",
        "    [18, 24, 18],\n",
        "    [5, 22, 10],\n",
        "    [15, 4, -7],\n",
        "    [30, 28, -19],\n",
        "    [14, 13, 22],\n",
        "    [25, 7, 10],\n",
        "    [21, 29, -23],\n",
        "    [23, 16, -25],\n",
        "    [26, 20, 11],\n",
        "    [9, 12, -1],\n",
        "    [2, 32, 1],\n",
        "    [11, 3, -2],\n",
        "    [16, 14, -16],\n",
        "    [11, 21, -3],\n",
        "    [13, 10, -24],\n",
        "    [26, 30, 10],\n",
        "    [7, 18, 28],\n",
        "    [29, 1, -7],\n",
        "    [22, 8, 11],\n",
        "    [4, 17, 19],\n",
        "    [15, 31, -4],\n",
        "    [32, 9, -1],\n",
        "    [5, 20, 4],\n",
        "    [27, 23, 13],\n",
        "    [3, 19, -34],\n",
        "    [12, 25, -7],\n",
        "    [24, 6, 43],\n",
        "    [28, 2, 10],\n",
        "    [29, 26, 18],\n",
        "    [1, 28, -3],\n",
        "    [9, 24, -2],\n",
        "    [6, 12, -5],\n",
        "    [31, 13, 6],\n",
        "    [27, 16, 3],\n",
        "    [20, 30, 25],\n",
        "    [21, 32, 14],\n",
        "    [14, 15, 20],\n",
        "    [7, 3, 17],\n",
        "    [18, 11, 1],\n",
        "    [19, 4, 14],\n",
        "    [2, 5, -1],\n",
        "    [25, 8, 13],\n",
        "    [23, 10, -20],\n",
        "    [17, 22, -13],\n",
        "]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8S7TM77vXXhY"
      },
      "source": [
        "### Compute the index set to facilitate setting up the model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "A7TvJBeHXVuc"
      },
      "outputs": [],
      "source": [
        "# Computing the index set (ijt)\n",
        "\n",
        "# Valid set of tuples\n",
        "A = []\n",
        "for t in game:\n",
        "    # record team pairs and match sequence\n",
        "    i = p[t - 1][0]\n",
        "    j = p[t - 1][1]\n",
        "    tp = i, j, t\n",
        "    A.append(tp)\n",
        "\n",
        "# print(np.matrix(A))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3kjqP68u4RyN"
      },
      "source": [
        "### Setup decisions, objective, and constraints"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "fImti2AV73Jt"
      },
      "outputs": [],
      "source": [
        "%%ampl_eval\n",
        "\n",
        "# Parameters\n",
        "param num_teams;\n",
        "param num_games;\n",
        "\n",
        "# Sets\n",
        "set teams := 1..num_teams;\n",
        "set games := 1..num_games;\n",
        "# data for point spread is a subset of teams x teams x games\n",
        "set games_data within {teams, teams, games};\n",
        "\n",
        "# Point spread between team i and team j in game t\n",
        "param p{games_data};\n",
        "\n",
        "# Build decision variables: team ratings and home team advantage\n",
        "var x{teams} >= 0;\n",
        "var y >= 0;"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "e04vv__b8EIR"
      },
      "outputs": [],
      "source": [
        "%%ampl_eval\n",
        "# Objective function: Minimize SSE\n",
        "minimize sse:\n",
        "    sum{(i,j,t) in games_data} (x[i] + y -x[j] - p[i,j,t])^2;"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "id": "_nlvX4Il8Ohe"
      },
      "outputs": [],
      "source": [
        "%%ampl_eval\n",
        "#Constraints\n",
        "# Fix average rating to be at 85\n",
        "s.t. fix_average_rating:\n",
        "    sum{i in teams} x[i] = 85*num_teams;"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "id": "IS9xZpAJeAUZ"
      },
      "outputs": [],
      "source": [
        "# Load data into ampl\n",
        "ampl.param[\"num_teams\"] = 32\n",
        "ampl.param[\"num_games\"] = 256\n",
        "ampl.set[\"games_data\"] = A\n",
        "ampl.param[\"p\"] = {(i, j, t): p[t - 1][2] for (i, j, t) in A}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hLWgN1XK4WAo"
      },
      "source": [
        "### Solve the model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "id": "ATwVQP0aDRIf",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "3591fbc2-8c03-4a07-eedf-835306ebd6f5"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Gurobi 10.0.1: Gurobi 10.0.1: optimal solution; objective 30972.98811\n",
            "0 simplex iterations\n",
            "6 barrier iterations\n"
          ]
        }
      ],
      "source": [
        "# Run optimization engine\n",
        "ampl.option[\"solver\"] = \"gurobi\"\n",
        "ampl.solve()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "I-fYhYVE4Ynh"
      },
      "source": [
        "### Examine outputs - The minimum SSE"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "id": "_4zHdHMXQQgQ",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a35fa719-abc0-4bb1-aebb-f7dcbeb7bdcb"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "The minimum sum of squared errors are  30972.99\n"
          ]
        }
      ],
      "source": [
        "# check the SSE\n",
        "print(\n",
        "    \"The minimum sum of squared errors are \",\n",
        "    round(ampl.get_objective(\"sse\").value(), 2),\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bDnhtdcYmsTB"
      },
      "source": [
        "### Check the optimal team ratings"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "id": "3vd_Uq4R4fkP",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "4df73a91-7609-4c84-fa8d-127990610bba"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\u001b[1m Home team advantage is \u001b[0m ( 3.11 ) \n",
            "\n",
            "\u001b[1m Optimal team ratings\n",
            "------------------------------------------\n",
            "\n",
            "\u001b[1m 1   Arizona Cardinals \u001b[0m: 91.45 \n",
            "\u001b[1m 2   Atlanta Falcons \u001b[0m: 82.24 \n",
            "\u001b[1m 3   Baltimore Ravens \u001b[0m: 81.47 \n",
            "\u001b[1m 4   Buffalo Bills \u001b[0m: 81.79 \n",
            "\u001b[1m 5   Carolina Panthers \u001b[0m: 94.2 \n",
            "\u001b[1m 6   Chicago Bears \u001b[0m: 80.87 \n",
            "\u001b[1m 7   Cincinnati Bengals \u001b[0m: 90.35 \n",
            "\u001b[1m 8   Cleveland Browns \u001b[0m: 77.31 \n",
            "\u001b[1m 9   Dallas Cowboys \u001b[0m: 84.31 \n",
            "\u001b[1m 10   Denver Broncos \u001b[0m: 96.37 \n",
            "\u001b[1m 11   Detroit Lions \u001b[0m: 83.36 \n",
            "\u001b[1m 12   Green Bay Packers \u001b[0m: 81.89 \n",
            "\u001b[1m 13   Houston Texans \u001b[0m: 77.42 \n",
            "\u001b[1m 14   Indianapolis Colts \u001b[0m: 89.04 \n",
            "\u001b[1m 15   Jacksonville Jaguars \u001b[0m: 73.9 \n",
            "\u001b[1m 16   Kansas City Chiefs \u001b[0m: 91.08 \n",
            "\u001b[1m 17   Miami Dolphins \u001b[0m: 84.16 \n",
            "\u001b[1m 18   Minnesota Vikings \u001b[0m: 78.38 \n",
            "\u001b[1m 19   New England Patriots \u001b[0m: 90.89 \n",
            "\u001b[1m 20   New Orleans Saints \u001b[0m: 93.77 \n",
            "\u001b[1m 21   New York Giants \u001b[0m: 79.6 \n",
            "\u001b[1m 22   New York Jets \u001b[0m: 78.91 \n",
            "\u001b[1m 23   Oakland Raiders \u001b[0m: 77.01 \n",
            "\u001b[1m 24   Philadelphia Eagles \u001b[0m: 86.85 \n",
            "\u001b[1m 25   Pittsburgh Steelers \u001b[0m: 83.05 \n",
            "\u001b[1m 26   St. Louis Rams \u001b[0m: 87.22 \n",
            "\u001b[1m 27   San Diego Chargers \u001b[0m: 87.66 \n",
            "\u001b[1m 28   San Francisco 49ers \u001b[0m: 95.13 \n",
            "\u001b[1m 29   Seattle Seahawks \u001b[0m: 98.04 \n",
            "\u001b[1m 30   Tampa Bay Buccaneers \u001b[0m: 82.33 \n",
            "\u001b[1m 31   Tennessee Titans \u001b[0m: 84.23 \n",
            "\u001b[1m 32   Washington Redskins \u001b[0m: 75.72 \n",
            "------------------------------------------\n",
            "\u001b[1m Average team ratings: \u001b[0m 85.0 \n"
          ]
        }
      ],
      "source": [
        "# print optimal ratings by team\n",
        "\n",
        "print(\n",
        "    \"\\033[1m Home team advantage is \\033[0m (\", round(ampl.var[\"y\"].value(), 2), \") \\n\"\n",
        ")\n",
        "\n",
        "print(\"\\033[1m Optimal team ratings\")\n",
        "print(\"------------------------------------------\\n\")\n",
        "# loop through all destinations\n",
        "\n",
        "x = ampl.var[\"x\"]\n",
        "\n",
        "average_rating = 0\n",
        "team_count = 0\n",
        "for i in team:\n",
        "    print(\n",
        "        \"\\033[1m\",\n",
        "        i,\n",
        "        \" \",\n",
        "        team_label[i - 1],\n",
        "        \"\\033[0m:\",\n",
        "        round(x[i].value(), 2),\n",
        "        \"\\n\",\n",
        "        end=\"\",\n",
        "    )\n",
        "    average_rating += x[i].value()\n",
        "    team_count += 1\n",
        "\n",
        "print(\"------------------------------------------\")\n",
        "print(\n",
        "    \"\\033[1m Average team ratings: \\033[0m\", round(average_rating / team_count, 2), \"\"\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_YfEEAFTTYnV"
      },
      "source": [
        "#Conclusion\n",
        "\n",
        "The NFL team rating problem shows that an nonlinear optimization model can be used to predict game match performances. The above example illustrates that teams can be rated in such a way to minimize the prediction errors for game outcomes.\n",
        "\n",
        "A key take away of the above example is that the team ratings should be controlled at a standard level (85 in this case). The reason is that without such a standard level, there exists an infinite set of optimal solutions that achieve the same RMSE. The standard level, however, can be set at any number.\n",
        "\n",
        "One potential issue with the above team rating optimization approach is that the ratings can over fit historical performance. An exponential weighted performance approach can partially overcome this issue but putting more weight on more recent matches and less weight on distant matchings. However, given that the regular season is not very long, such an issue is unlikely to be a significant concern."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XiMLngKYoi4l"
      },
      "source": [
        "##  References\n",
        "\n",
        "[1] AMPL python reference. https://amplpy.readthedocs.io/en/latest/reference.html\n",
        "\n",
        "[2] This notebook is developed by Yimin Wang (yimin_wang@asu.edu). Marcos Dominguez (marcos@ampl.com) also contributed to this notebook content."
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.1"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "a6e234808964470692b42e39cc3e28fe": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "VBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "VBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "VBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_cd5610257054498a8fac67b099383d7d",
              "IPY_MODEL_ed42846dfe914971b77c266371bf9ff6",
              "IPY_MODEL_74968e1bf66b4f868e6b3b0390fe1e3d",
              "IPY_MODEL_4971a28d89274cadac6a678b721666e6"
            ],
            "layout": "IPY_MODEL_0b63ce492bcb432f92f99f2d70261f26"
          }
        },
        "cd5610257054498a8fac67b099383d7d": {
          "model_module": "@jupyter-widgets/output",
          "model_name": "OutputModel",
          "model_module_version": "1.0.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/output",
            "_model_module_version": "1.0.0",
            "_model_name": "OutputModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/output",
            "_view_module_version": "1.0.0",
            "_view_name": "OutputView",
            "layout": "IPY_MODEL_b7c841ecb90e4b66af31c410f09160b5",
            "msg_id": "",
            "outputs": [
              {
                "output_type": "stream",
                "name": "stdout",
                "text": [
                  "AMPL License UUID (you can use a free https://ampl.com/ce license):\n"
                ]
              }
            ]
          }
        },
        "ed42846dfe914971b77c266371bf9ff6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_cf98cae131e64d8c900a900ada200f06"
            ],
            "layout": "IPY_MODEL_71916f0f6883405a99c0727f74fc675d"
          }
        },
        "74968e1bf66b4f868e6b3b0390fe1e3d": {
          "model_module": "@jupyter-widgets/output",
          "model_name": "OutputModel",
          "model_module_version": "1.0.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/output",
            "_model_module_version": "1.0.0",
            "_model_name": "OutputModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/output",
            "_view_module_version": "1.0.0",
            "_view_name": "OutputView",
            "layout": "IPY_MODEL_3d192916823b446485dac899441f869e",
            "msg_id": "",
            "outputs": []
          }
        },
        "4971a28d89274cadac6a678b721666e6": {
          "model_module": "@jupyter-widgets/output",
          "model_name": "OutputModel",
          "model_module_version": "1.0.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/output",
            "_model_module_version": "1.0.0",
            "_model_name": "OutputModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/output",
            "_view_module_version": "1.0.0",
            "_view_name": "OutputView",
            "layout": "IPY_MODEL_9e029c37ed8b461f9483dd7ced76050b",
            "msg_id": "",
            "outputs": [
              {
                "output_type": "stream",
                "name": "stdout",
                "text": [
                  "Licensed to AMPL Community Edition License for the AMPL Model Colaboratory (https://colab.ampl.com).\n"
                ]
              }
            ]
          }
        },
        "0b63ce492bcb432f92f99f2d70261f26": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "cf98cae131e64d8c900a900ada200f06": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "TextModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "TextModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "TextView",
            "continuous_update": true,
            "description": "License UUID:",
            "description_tooltip": null,
            "disabled": false,
            "layout": "IPY_MODEL_ebda6bb9c85940d28d428e930265f144",
            "placeholder": "​",
            "style": "IPY_MODEL_b867fda15a7e421d85c618f38aba38dd",
            "value": ""
          }
        },
        "71916f0f6883405a99c0727f74fc675d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ebda6bb9c85940d28d428e930265f144": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b867fda15a7e421d85c618f38aba38dd": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": "initial"
          }
        },
        "b7c841ecb90e4b66af31c410f09160b5": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3d192916823b446485dac899441f869e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9e029c37ed8b461f9483dd7ced76050b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}