notebooks/python/04. Batching.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# OPTaaS Batching\n",
    "\n",
    "### <span style=\"color:red\">Note:</span> To run this notebook, you need an API Key. You can get one <a href=\"mailto:info@mindfoundry.ai\">here</a>.\n",
    "\n",
    "OPTaaS can facilitate parallel computation, where you generate a batch of configurations, pass them to a number of workers to calculate the results, and then store the results to get the next batch of configurations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Connect to OPTaaS using your API Key"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mindfoundry.optaas.client.client import OPTaaSClient\n",
    "\n",
    "client = OPTaaSClient('https://optaas.mindfoundry.ai', '<Your OPTaaS API key>')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create your Task"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mindfoundry.optaas.client.parameter import FloatParameter\n",
    "\n",
    "task = client.create_task(\n",
    "    title='Batching Example',\n",
    "    parameters=[\n",
    "        FloatParameter('x', minimum=0, maximum=1),\n",
    "        FloatParameter('y', minimum=0.1, maximum=2)\n",
    "    ]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set up your workers\n",
    "This is just a simple example of how you would pass a Configuration to a worker and get a Result back. Your process will of course likely be more complex!\n",
    "\n",
    "The number of workers will depend on how much processing power you have available. In order to get the best quality configurations from OPTaaS, we recommend using no more than 10."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from multiprocessing import Pool\n",
    "from time import sleep\n",
    "from mindfoundry.optaas.client.result import Result\n",
    "\n",
    "number_of_workers = 4\n",
    "\n",
    "def spin_off_workers_and_get_results(configurations):\n",
    "    with Pool(number_of_workers) as pool:\n",
    "        return pool.map(get_result, configurations)\n",
    "\n",
    "def get_result(configuration):\n",
    "    x = configuration.values['x']\n",
    "    y = configuration.values['y']\n",
    "    sleep(1)  # Calculating score...\n",
    "    score = (x * y) - (x / y)\n",
    "    return Result(configuration=configuration, score=score)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generate the first batch of configurations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'type': 'default', 'values': {'x': 0.5, 'y': 1.05}}, { 'type': 'exploration',\n",
       "   'values': {'x': 0.5515729001609302, 'y': 0.9539107654962646}}, { 'type': 'exploration',\n",
       "   'values': {'x': 0.4024718947116285, 'y': 0.6400426939913346}}, { 'type': 'exploration',\n",
       "   'values': {'x': 0.8954369174051902, 'y': 0.2679197390779888}}]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "configurations = task.generate_configurations(number_of_workers)\n",
    "display(configurations)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Record results to get the next batch of configurations\n",
    "\n",
    "The next batch will be the same size as the number of results you record."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Scores: [0.04880952380952386, 0.32982698432350455, -3.5595532282188787, 0.19394652746118213]\n",
      "\n",
      "Next configurations: [{'x': 0.6177680922768689, 'y': 0.450418600138205}, {'x': 0.08624546027300994, 'y': 1.810425236962426}, {'x': 0.2448349778331208, 'y': 0.3929881953445803}, {'x': 0.11298400991785473, 'y': 0.44078687252193205}]\n",
      "\n",
      "Scores: [-1.0932879041023202, 0.10850272429629561, -0.5267912228698785, -0.20652157717292371]\n",
      "\n",
      "Next configurations: [{'x': 0.12110706920922909, 'y': 0.9073183154180738}, {'x': 0.5814744623833986, 'y': 1.9081733977000617}, {'x': 0.07067751248062448, 'y': 1.4424042554392127}, {'x': 0.1581821251140786, 'y': 1.4039505211339234}]\n",
      "\n",
      "Scores: [-0.023595376669542353, 0.8048257865186552, 0.05294575000372527, 0.10941057509098914]\n",
      "\n",
      "Next configurations: [{'x': 0.99999998, 'y': 1.99999998}, {'x': 0.99999998, 'y': 1.99999998}, {'x': 0.99999998, 'y': 1.99999998}, {'x': 0.99999998, 'y': 1.99999998}]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "number_of_batches = 3\n",
    "\n",
    "for _ in range(number_of_batches):\n",
    "    results = spin_off_workers_and_get_results(configurations)\n",
    "    print(f\"Scores: {[result.score for result in results]}\\n\")\n",
    "\n",
    "    configurations = task.record_results(results)\n",
    "    print(f\"Next configurations: {[c.values for c in configurations]}\\n\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  },
  "nav_menu": {},
  "toc": {
   "navigate_menu": true,
   "number_sections": false,
   "sideBar": true,
   "threshold": 6.0,
   "toc_cell": false,
   "toc_section_display": "block",
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}