black_box_optimization_package_recommender.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "black-box-optimization-package-recommender.ipynb",
      "provenance": [],
      "authorship_tag": "ABX9TyPvACeYoLZ9aFH4gqkgh16A",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/microprediction/humpday/blob/main/black_box_optimization_package_recommender.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dOItaUcAIxs8"
      },
      "source": [
        "!pip install humpday[full]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Uninstall ones you don't want. For example:"
      ],
      "metadata": {
        "id": "uZyl4uqtpm2c"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip uninstall freelunch -y"
      ],
      "metadata": {
        "id": "p2HN3-InpheV"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "If you uninstall, Runtime->Restart runtime"
      ],
      "metadata": {
        "id": "QX5RCfR5qCdO"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GGfPDj6ERPU5",
        "outputId": "0c4475f9-0c9f-4e7d-feb8-a04cf3151bcf",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "from humpday import recommend\n",
        "import time \n",
        "import math\n",
        "from pprint import pprint"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Bayesian-Optimization turned off pending https://github.com/fmfn/BayesianOptimization/issues/300\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V1yKlwYgJLuQ"
      },
      "source": [
        "# A script to recommend a black box optimizer\n",
        "There are lots of package out there. This will generate a shortlist of Python global derivative free optimizers. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "e2Ff1ZyVJGBR",
        "outputId": "9c31540e-b85e-48e3-f360-3829075b5af7"
      },
      "source": [
        "def my_objective(u):\n",
        "   # Swap this out for your own. \n",
        "   # Domain must be [0,1]^n\n",
        "   time.sleep(0.01)\n",
        "   return u[0]*math.sin(u[1])\n",
        "\n",
        "my_objective(u=[0.2,0.5])\n"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.0958851077208406"
            ]
          },
          "metadata": {},
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x3myV_06JO09"
      },
      "source": [
        "## Run the recommender\n",
        "This will puke a list of optimizers, informed by ongoing battles reported in the [Optimizer Elo Ratings](https://microprediction.github.io/optimizer-elo-ratings/html_leaderboards/all_d08_n80.html)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dWwUwleVJIQT",
        "outputId": "f4e29792-1074-4473-cecf-5037a42ec757"
      },
      "source": [
        "recommendations = recommend(my_objective, n_dim=4, n_trials=130)\n",
        "pprint(recommendations[:7])\n",
        "# Elo rating,  relative computation time, name "
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[(1814.3755906106635, 0.0029230117797851562, 'scipy_powell_cube'),\n",
            " (1802.5838664883092, 0.020003080368041992, 'shgo_nelder_sobol_cube'),\n",
            " (1731.6287534816827, 0.028772830963134766, 'pymoo_pattern_cube'),\n",
            " (1687.2937489303822, 0.024258852005004883, 'pymoo_nelder_cube'),\n",
            " (1678.704250303725, 0.017965078353881836, 'shgo_powell_sobol_cube'),\n",
            " (1655.2005888973442, 0.018489837646484375, 'shgo_slsqp_sobol_cube'),\n",
            " (1626.2464422864057, 0.0031270980834960938, 'scipy_lbfgsb_cube')]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7IP3OxYNO4mo"
      },
      "source": [
        "Any of these can be called from the humpday package. For example:"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from humpday.optimizers.scipycube import scipy_powell_cube"
      ],
      "metadata": {
        "id": "ECMxDhDum8H2"
      },
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ncULVsRVPLUs"
      },
      "source": [
        "# Points race\n",
        "Here's something else you can do that is a lot more time intensive, but probably very helpful. The 'points race' utility will see how all the recommended optimizers perform on your objective function (or set of objective functions). It reports a running total of points, where 3 points are assigned for first place, 2 for second and 1 for third.\n",
        "\n",
        "(I warn you that using the feeble resources on colab for this might require patience)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Hd0gHMSJQ7d9"
      },
      "source": [
        "def my_other_objective(u):\n",
        "   # Swap this out for your own. \n",
        "   # Domain must be [0,1]^n\n",
        "   time.sleep(0.01)\n",
        "   return u[0]*u[0]*math.cos(u[1])"
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-HK2wVpvQv12",
        "outputId": "3c3f07c5-016d-494e-c338-7ea65ffed26d"
      },
      "source": [
        "from humpday import points_race\n",
        "points_race(objectives=[my_objective, my_other_objective],n_dim=5, n_trials=100)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Optimizing the my_objective function ...\n"
          ]
        }
      ]
    }
  ]
}