hyperparameter_naming.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "hyperparameter_naming.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyPZ3dt+T/1UVthZDY3zTj6O",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/taiwotman/TaiwotmanGoogleColab/blob/main/hyperparameter_naming.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WpqB5hG774sg"
      },
      "source": [
        "# Hyperparameter Naming a.k.a Sofia's Data Science Problem\n",
        "\n",
        "Author: Taiwo O. Adetiloye, PhD | [Website](https://taiwotman.github.io)\n",
        "\n",
        "Date: August 28, 2020\n",
        "\n",
        "Reference: https://gist.github.com/taiwotman/0826dc50cf57832a204b3d1648c21072"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mVhfgu1W_jTu"
      },
      "source": [
        "**Problem statement**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dOHivNeb8Qiq"
      },
      "source": [
        "Sofia, a data scientist, wants to train a machine learning model that accepts a set of tunable hyperparameters. The hyperparameters are specified in a Python dictionary as follows:\n",
        "\n",
        "```\n",
        "H = {\n",
        "   “learning_rate”: 0.1,\n",
        "   “dropout_in”: 0.5,\n",
        "   “dropout_out”: 0.3,\n",
        "   “use_cutout”: True,\n",
        "   “use_skip”: False,\n",
        "   “momentum”: 0.9\n",
        "}\n",
        "\n",
        "```\n",
        "\n",
        "\n",
        "Sofia wants to nicely serialize the hyperparameters into a string such that she can easily remember her experiments. To this end, she decides to code a function get_experiment_name(hyperparameters_dict, prefix_length=-1) that returns a string. A prefix_length specifies the length of the hyperparameter prefix that will be reported in the experiment name string. In the case that two hyperparameters have the same prefix of length prefix_length, the function returns the minimum length prefix that can uniquely identify the hyperparameter.\n",
        "\n",
        "A few example usages are:\n",
        "```\n",
        "get_experiment_name(H) -> “learningrate_0.1_dropoutin_0.5_dropoutout_0.3_usecutout_True_useskip_False_momentum_0.9”\n",
        "\n",
        "get_experiment_name(H, 3) -> “lea_0.1_dropouti_0.5_dropouto_0.3_usec_True_uses_False_mom_0.9”\n",
        "\n",
        "get_experiment_name(H, 1) -> “l_0.1_dropouti_0.5_dropouto_0.3_usec_True_uses_False_m_0.9”\n",
        "```\n",
        "\n",
        "\n",
        "Complete the following function and discuss/analyze corner-cases i.e. possible problems with the current formulation:\n",
        "\n",
        "```\n",
        "def get_experiment_name(hyperparameters_dict, prefix_length=-1):\n",
        "    experiment_name = “”\n",
        "    return experiment_name\n",
        "```"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uSIdk-xq8pkT"
      },
      "source": [
        "**Solution**\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "C_MmiaxM8uPb"
      },
      "source": [
        "###\n",
        "## Author: Taiwo O. Adetiloye\n",
        "## Date: Aug 28, 2020\n",
        "## Solution 1\n",
        "###\n",
        "  \n",
        "import collections\n",
        "def get_experiment_name(hyperparameters_dict, prefix_length=-1):\n",
        "    H = hyperparameters_dict\n",
        "   \n",
        "    experiment_name  = \"\"\n",
        "\n",
        "    # define default value of experiment \n",
        "\n",
        "    for key, value in H.items():\n",
        "            prefix_determinator = key.replace(\"_\",\"\") + \"_\"+ str(value) +\"_\"\n",
        "            experiment_name =  experiment_name + prefix_determinator\n",
        " \n",
        "    if (prefix_length  == -1) or  (prefix_length is None):\n",
        "       \n",
        "       # return default value\n",
        "       get_length = len(experiment_name)    # get length of experiment name\n",
        "       return experiment_name[:get_length-1]\n",
        "        \n",
        "    else:\n",
        "       H_key_list  = list(H.keys()) #get key lists\n",
        "\n",
        "       # get prefix\n",
        "       prefix_determinator1 = [ H_key_list[i].split(\"_\") [0] for i in range(len(H_key_list))]\n",
        "       \n",
        "       #get same key prefix\n",
        "       uniq = [item for item, count in collections.Counter(prefix_determinator1).items() if count > 1]\n",
        "       uniq_prefix = []\n",
        "\n",
        "       for key in H.keys(): \n",
        "           for i in uniq:\n",
        "                if key.startswith(i):\n",
        "                   split_key =  key.split(\"_\")\n",
        "                   str1 = split_key[0]\n",
        "                   str2 = split_key[1][:1]\n",
        "                   concat_key = str1 +  str2\n",
        "                   uniq_prefix.append(concat_key)\n",
        "                   H_key_list.remove(key)\n",
        "\n",
        "       H_key_list_prefix = [i[:prefix_length] for i in H_key_list]\n",
        "\n",
        "       # Replace experiment name strings with new values \n",
        "       experiment_name = experiment_name\\\n",
        "       .replace(\"learningrate\",H_key_list_prefix[0])\\\n",
        "       .replace(\"momentum\",H_key_list_prefix[1])\\\n",
        "       .replace(\"dropoutin\",uniq_prefix[0])\\\n",
        "       .replace(\"dropoutout\",uniq_prefix[1])\\\n",
        "       .replace(\"usecutout\",uniq_prefix[2])\\\n",
        "       .replace(\"useskip\",uniq_prefix[3])\n",
        "       \n",
        "       get_length = len(experiment_name)    # get length of experiment name\n",
        "       return experiment_name[:get_length-1]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MmTTPcWd89Gq"
      },
      "source": [
        "**Test cases**\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AMMFUtDK9BgD"
      },
      "source": [
        " H = {\n",
        "    \"learning_rate\": 0.1,\n",
        "    \"dropout_in\": 0.5,\n",
        "    \"dropout_out\": 0.3,\n",
        "    \"use_cutout\": True,\n",
        "    \"use_skip\": False,\n",
        "    \"momentum\": 0.9\n",
        "    }\n",
        "\n",
        "#Test case 1\n",
        "assert(get_experiment_name(H)) == \"learningrate_0.1_dropoutin_0.5_dropoutout_0.3_usecutout_True_useskip_False_momentum_0.9\" \n",
        "\n",
        "#Test case 2 \n",
        "assert(get_experiment_name(H,1)) == \"l_0.1_dropouti_0.5_dropouto_0.3_usec_True_uses_False_m_0.9\"\n",
        "\n",
        "#Test case 3\n",
        "assert(get_experiment_name(H,3)) == \"lea_0.1_dropouti_0.5_dropouto_0.3_usec_True_uses_False_mom_0.9\"\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zLRlzr1GBU5p"
      },
      "source": [
        "**Edge cases**\n",
        "\n",
        "1. If the prefix_length is greater that the length of the key prefix, \n",
        "then the formulation would be distorted\n",
        "2.  The dictionary is not ideal for situation where the\n",
        "hyperparameters has to be changed since dictionary keys are immutable.\n",
        "3. This is compatible with Python3 using collections to get unique items\n"
      ]
    }
  ]
}