07. Meta-SGD and Reptile Algorithms/7.7 Sine wave Regression Using Reptile.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Sine wave Regression Using Reptile"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "In the last section, we saw how reptile works. Now, we will understand reptile better by coding them from scratch. Let us say we have a collection of tasks and the goal of each task is to regress the output of the sine wave given some input. So what do we mean by that?\n",
    "\n",
    "Let us say y = amplitude*sin(x+phase). The goal of our algorithm is to learn to regress the value of y given the x. The value of amplitude is chosen randomly between 0.1 and 5.0 and the value of phase is chosen randomly between 0 and $\\pi$. So for each of the tasks, we sample only 10 data points and train the network. i.e for each of the tasks, we sample only 10 (x,y) pairs. let us get to the code and see in detail.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First we import all the necessary libraries,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generate Data Points"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we define a function called sample_points for generating (x,y) pairs. It takes the parameter k as an input\n",
    "which implies number of (x,y) pairs we want to sample. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def sample_points(k):\n",
    "    \n",
    "    num_points = 100\n",
    "    \n",
    "    #amplitude\n",
    "    amplitude = np.random.uniform(low=0.1, high=5.0)\n",
    "    \n",
    "    #phase\n",
    "    phase = np.random.uniform(low=0, high=np.pi)\n",
    "\n",
    "    x = np.linspace(-5, 5, num_points)\n",
    "\n",
    "    #y = a*sin(x+b)\n",
    "    y = amplitude * np.sin(x + phase)\n",
    "    \n",
    "    #sample k data points\n",
    "    sample = np.random.choice(np.arange(num_points), size=k)\n",
    "    \n",
    "    return (x[sample], y[sample])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above function returns output as follows,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 1.96969697 -2.17171717  3.18181818  1.06060606  2.67676768]\n",
      "[-0.97122995 -0.19398046 -1.1409783   0.07730279 -1.29343206]\n"
     ]
    }
   ],
   "source": [
    "x, y = sample_points(5)\n",
    "print x\n",
    "print y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Two Layered Neural Network"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Like MAML, reptile also compatible with any algorithms that can be trained with gradient descent. So we use a simple two layered neural network with 64 hidden units. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, let's reset the tensorflow graph,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "tf.reset_default_graph()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Initialize network parameters,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "num_hidden = 64\n",
    "num_classes = 1\n",
    "num_feature = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we define the placeholders for our input and output,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X = tf.placeholder(tf.float32, shape=[None, num_feature])\n",
    "Y = tf.placeholder(tf.float32, shape=[None, num_classes])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Randomly initialize our model parameters, "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "w1 = tf.Variable(tf.random_uniform([num_feature, num_hidden]))\n",
    "b1 = tf.Variable(tf.random_uniform([num_hidden]))\n",
    "\n",
    "w2 = tf.Variable(tf.random_uniform([num_hidden, num_classes]))\n",
    "b2 = tf.Variable(tf.random_uniform([num_classes]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Perform feedforward operation to predict the output Yhat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#layer 1\n",
    "z1 = tf.matmul(X, w1) + b1\n",
    "a1 = tf.nn.tanh(z1)\n",
    "\n",
    "#output layer\n",
    "z2 = tf.matmul(a1, w2) + b2\n",
    "Yhat = tf.nn.tanh(z2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We use mean squared error as our loss function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "loss_function = tf.reduce_mean(tf.square(Yhat - Y))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Minimize the loss using Adam Optimizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "optimizer = tf.train.AdamOptimizer(1e-2).minimize(loss_function)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Initialize tensorflow variables,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "init = tf.global_variables_initializer()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reptile\n",
    "\n",
    "\n",
    "Now we will see how can we find the optimal parameters of our neural network with reptile. \n",
    "\n",
    "First we initialize necessary variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#number of epochs i.e training iterations\n",
    "num_epochs = 100\n",
    "\n",
    "\n",
    "#number of samples i.e number of shots\n",
    "num_samples = 50  \n",
    "\n",
    "#number of tasks\n",
    "num_tasks = 2\n",
    "\n",
    "#number of times we want to perform optimization\n",
    "num_iterations = 10\n",
    "\n",
    "\n",
    "#mini btach size\n",
    "mini_batch = 10  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 0: Loss 4.162981987\n",
      "\n",
      "Updated Model Parameter Theta\n",
      "\n",
      "Sampling Next Batch of Tasks \n",
      "\n",
      "---------------------------------\n",
      "\n",
      "Epoch 10: Loss 1.52488529682\n",
      "\n",
      "Updated Model Parameter Theta\n",
      "\n",
      "Sampling Next Batch of Tasks \n",
      "\n",
      "---------------------------------\n",
      "\n",
      "Epoch 20: Loss 1.74668705463\n",
      "\n",
      "Updated Model Parameter Theta\n",
      "\n",
      "Sampling Next Batch of Tasks \n",
      "\n",
      "---------------------------------\n",
      "\n",
      "Epoch 30: Loss 0.955383658409\n",
      "\n",
      "Updated Model Parameter Theta\n",
      "\n",
      "Sampling Next Batch of Tasks \n",
      "\n",
      "---------------------------------\n",
      "\n",
      "Epoch 40: Loss 9.85152721405\n",
      "\n",
      "Updated Model Parameter Theta\n",
      "\n",
      "Sampling Next Batch of Tasks \n",
      "\n",
      "---------------------------------\n",
      "\n",
      "Epoch 50: Loss 3.62764883041\n",
      "\n",
      "Updated Model Parameter Theta\n",
      "\n",
      "Sampling Next Batch of Tasks \n",
      "\n",
      "---------------------------------\n",
      "\n",
      "Epoch 60: Loss 5.67536497116\n",
      "\n",
      "Updated Model Parameter Theta\n",
      "\n",
      "Sampling Next Batch of Tasks \n",
      "\n",
      "---------------------------------\n",
      "\n",
      "Epoch 70: Loss 1.39854609966\n",
      "\n",
      "Updated Model Parameter Theta\n",
      "\n",
      "Sampling Next Batch of Tasks \n",
      "\n",
      "---------------------------------\n",
      "\n",
      "Epoch 80: Loss 6.68237304688\n",
      "\n",
      "Updated Model Parameter Theta\n",
      "\n",
      "Sampling Next Batch of Tasks \n",
      "\n",
      "---------------------------------\n",
      "\n",
      "Epoch 90: Loss 7.83017683029\n",
      "\n",
      "Updated Model Parameter Theta\n",
      "\n",
      "Sampling Next Batch of Tasks \n",
      "\n",
      "---------------------------------\n",
      "\n"
     ]
    }
   ],
   "source": [
    "#start the tensorflow session\n",
    "with tf.Session() as sess:\n",
    "    \n",
    "    sess.run(init)\n",
    "    \n",
    "    for e in range(num_epochs):\n",
    "        \n",
    "        #for each task in batch of tasks\n",
    "        for task in range(num_tasks):\n",
    "\n",
    "            #get the initial parameters of the model\n",
    "            old_w1, old_b1, old_w2, old_b2 = sess.run([w1, b1, w2, b2,])\n",
    "\n",
    "            #sample x and y\n",
    "            x_sample, y_sample = sample_points(num_samples)\n",
    "\n",
    "\n",
    "            #for some k number of iterations perform optimization on the task\n",
    "            for k in range(num_iterations):\n",
    "\n",
    "                #get the minibatch x and y\n",
    "                for i in range(0, num_samples, mini_batch):\n",
    "\n",
    "                    #sample mini batch of examples \n",
    "                    x_minibatch = x_sample[i:i+mini_batch]\n",
    "                    y_minibatch = y_sample[i:i+mini_batch]\n",
    "\n",
    "\n",
    "                    train = sess.run(optimizer, feed_dict={X: x_minibatch.reshape(mini_batch,1), \n",
    "                                                           Y: y_minibatch.reshape(mini_batch,1)})\n",
    "\n",
    "            #get the updated model parameters after several iterations of optimization\n",
    "            new_w1, new_b1, new_w2, new_b2 = sess.run([w1, b1, w2, b2])\n",
    "\n",
    "            #Now we perform meta update\n",
    "\n",
    "            #i.e theta = theta + epsilon * (theta_star - theta)\n",
    "\n",
    "            epsilon = 0.1\n",
    "\n",
    "            updated_w1 = old_w1 + epsilon * (new_w1 - old_w1) \n",
    "            updated_b1 = old_b1 + epsilon * (new_b1 - old_b1) \n",
    "\n",
    "            updated_w2 = old_w2 + epsilon * (new_w2 - old_w2) \n",
    "            updated_b2 = old_b2 + epsilon * (new_b2 - old_b2) \n",
    "\n",
    "\n",
    "            #update the model parameter with new parameters\n",
    "            w1.load(updated_w1, sess)\n",
    "            b1.load(updated_b1, sess)\n",
    "\n",
    "            w2.load(updated_w2, sess)\n",
    "            b2.load(updated_b2, sess)\n",
    "\n",
    "        if e%10 == 0:\n",
    "            loss = sess.run(loss_function, feed_dict={X: x_sample.reshape(num_samples,1), Y: y_sample.reshape(num_samples,1)})\n",
    "\n",
    "            print \"Epoch {}: Loss {}\\n\".format(e,loss)             \n",
    "            print 'Updated Model Parameter Theta\\n'\n",
    "            print 'Sampling Next Batch of Tasks \\n'\n",
    "            print '---------------------------------\\n'"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}