Skip to content

Commit

Permalink
Add files via upload
Browse files Browse the repository at this point in the history
  • Loading branch information
@llSourcell
llSourcell committed Oct 4, 2017
1 parent a3abea0 commit d1349e6
Showing 1 changed file with 373 additions and 0 deletions.
373 changes: 373 additions & 0 deletions The Future of Deep Learning Research.ipynb
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,373 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# The Future of Deep Learning Research\n",
"\n",
"Topics\n",
"- How does Backpropagation work?\n",
"- What are the most popular deep learning algorithms today?\n",
"- 7 Research Directions I've handpicked\n",
"\n",
"In a recent AI conference, Geoffrey Hinton [remarked](https://www.axios.com/ai-pioneer-advocates-starting-over-2485537027.html) that he was “deeply suspicious” of back-propagation, and said “My view is throw it all away and start again.”\n",
"\n",
"![alt text](https://assets.rbl.ms/11015654/960x540.png \"Logo Title Text 1\")\n",
"\n",
"## The billion dollar question - how does the brain learn so well from sparse, unlabeled data?\n",
"\n",
"### Let's first understand how backpropagation works\n",
"\n",
"![alt text](https://i.ytimg.com/vi/An5z8lR8asY/maxresdefault.jpg \"Logo Title Text 1\")\n",
"\n",
"In 1986 Hinton released [this](http://www.cs.toronto.edu/~hinton/absps/naturebp.pdf) paper detailing a new optimization strategy for neural networks called 'backpropagation'. This paper is the reason the current Deep Learning boom is possible. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![alt text](https://www.otexts.org/sites/default/files/fpp/images/nnet2.png \"Logo Title Text 1\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"import numpy as np\n",
"\n",
"#nonlinearity\n",
"def nonlin(x,deriv=False):\n",
"\tif(deriv==True):\n",
"\t return x*(1-x)\n",
"\n",
"\treturn 1/(1+np.exp(-x))\n",
" \n",
"#input data\n",
"X = np.array([[0,0,1],\n",
" [0,1,1],\n",
" [1,0,1],\n",
" [1,1,1]])\n",
" \n",
"#output data\n",
"y = np.array([[0],\n",
"\t\t\t[1],\n",
"\t\t\t[1],\n",
"\t\t\t[0]])\n",
"\n",
"np.random.seed(1)\n",
"\n",
"# randomly initialize our weights with mean 0\n",
"syn0 = 2*np.random.random((3,4)) - 1\n",
"syn1 = 2*np.random.random((4,1)) - 1"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![alt text](https://deeplearning4j.org/img/weighted_input_RBM.png \"Logo Title Text 1\")\n",
"![alt text](https://algebra1course.files.wordpress.com/2013/02/slide10.jpg \"Logo Title Text 1\")\n",
"![alt text](https://imgur.com/a/9krNv \"Logo Title Text 1\")\n",
"\n",
"3 concepts behind Backpropagtion (From Calculus)\n",
"\n",
"1. Derivative\n",
"![alt text](https://i.imgur.com/eRF9pXu.jpg \"Logo Title Text 1\")\n",
"\n",
"2. Partial Derivative\n",
"\n",
"![alt text](https://i.imgur.com/Rergqbt.jpg \"Logo Title Text 1\")\n",
"\n",
"3. Chain Rule\n",
"\n",
"![alt text](https://i.imgur.com/HFmGQyH.jpg \"Logo Title Text 1\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"for j in xrange(60000):\n",
"\n",
"\t# Feed forward through layers 0, 1, and 2\n",
" k0 = X\n",
" k1 = nonlin(np.dot(k0,syn0))\n",
" k2 = nonlin(np.dot(k1,syn1))\n",
"\n",
" # how much did we miss the target value?\n",
" k2_error = y - k2\n",
"\n",
" if (j% 10000) == 0:\n",
" print \"Error:\" + str(np.mean(np.abs(k2_error)))\n",
" \n",
" # in what direction is the target value?\n",
" # were we really sure? if so, don't change too much.\n",
" k2_delta = k2_error*nonlin(k2,deriv=True)\n",
"\n",
" # how much did each k1 value contribute to the k2 error (according to the weights)?\n",
" k1_error = k2_delta.dot(syn1.T)\n",
" \n",
" # in what direction is the target k1?\n",
" # were we really sure? if so, don't change too much.\n",
" k1_delta = k1_error * nonlin(k1,deriv=True)\n",
"\n",
" syn1 += k1.T.dot(k2_delta)\n",
" syn0 += k0.T.dot(k1_delta)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# This is the method of choice for all labeled deep learning models\n",
"![alt text](http://www.asimovinstitute.org/wp-content/uploads/2016/09/neuralnetworks.png \"Logo Title Text 1\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# How do artificial & biological neural nets compare?\n",
"\n",
"Artificial Neural Networks are inspired by the hierarchial structure of brains neural network\n",
"\n",
"![alt text](https://appliedgo.net/media/perceptron/neuron.png \"Logo Title Text 1\")\n",
"\n",
"The brain has \n",
"-100 billion neurons \n",
"-- Each neuron has\n",
" - A cell body w/ connections\n",
" - numerous dendrites \n",
" - A single axon \n",
"- Parallel chaining (each neurons connected to 10,000+ others)\n",
"- Great at connecting different concepts\n",
"\n",
"Computers have\n",
"- Not neurons, but transistors made in silicon!\n",
"- Serially chained (each connected to 2-3 others (logic gates))\n",
"- Great at storage and recall\n",
"\n",
"Some key differences\n",
"- All sensory or motor systems in the brain are recurrent\n",
"- Sensory systems tend to have lots of lateral inhibition (neurons inhibiting other neurons in the same layer)\n",
"- There is no such thing as a fully connected layer in the brain, connectivity is usually sparse (though not random).\n",
"- brains are born pre-wired to learn without supervision.\n",
"- The Brain is low power. Alpha GO consumed the power of 1202 CPUs and 176 GPUs, not to train, but just to run. Brain’s power consumption is ~20W.\n",
"\n",
"![alt text](https://images.gr-assets.com/books/1348246481l/5080355.jpg\n",
" \"Logo Title Text 1\")\n",
"\n",
"\"the brain is not a blank slate of neuronal layers \n",
"waiting to be pieced together and wired-up; \n",
"we are born with brains already structured \n",
"for unsupervised learning in a dozen cognitive \n",
"domains, some of which already work pretty well \n",
"without any learning at all.\" - Steven Pinker\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Where are we today in unsupervised learning?\n",
"\n",
"\n",
"## For classification\n",
"\n",
"- Clustering (k-means, dimensionality reduction, anomaly detection)\n",
"\n",
"![alt text](https://ds055uzetaobb.cloudfront.net/image_optimizer/ff1732816ba08239c0d3b200c3a9708070885705.jpg\n",
" \"Logo Title Text 1\")\n",
"\n",
"- Autoencoders\n",
"![alt text](https://image.slidesharecdn.com/deeplearningfromanoviceperspective-150811155203-lva1-app6891/95/deep-learning-from-a-novice-perspective-16-638.jpg?cb=1439308391\n",
" \"Logo Title Text 1\")\n",
"\n",
"## For Generation\n",
"\n",
"- Generative Adversarial Networks\n",
"\n",
"![alt text](http://www.kdnuggets.com/wp-content/uploads/generative-adversarial-network.png\n",
" \"Logo Title Text 1\")\n",
"- Variational Autoencoders\n",
"\n",
"![alt text](http://fastforwardlabs.github.io/blog-images/miriam/imgs_code/vae.4.png\n",
" \"Logo Title Text 1\")\n",
"\n",
"- Differentiable Neural Computer\n",
"\n",
"https://en.wikipedia.org/wiki/Differentiable_neural_computer\n",
"\n",
"![alt text](https://storage.googleapis.com/deepmind-live-cms/images/dnc_figure1.width-1500_C612yWA.png\n",
" \"Logo Title Text 1\")\n",
"\n",
"![alt text](https://storage.googleapis.com/deepmind-live-cms/images/dnc_figure2.width-1500_1bhcgxm.png\n",
" \"Logo Title Text 1\")\n",
"\n",
"- The controller receives external inputs and, based on these, interacts with the memory using read and write operations known as 'heads'. \n",
"- To help the controller navigate the memory, DNC stores 'temporal links' to keep track of the order things were written in, and records the current 'usage' level of each memory location.\n",
"- DNCs were demonstrated, for example, how a DNC can be trained to navigate a variety of rapid transit systems, and then apply what it learned to get around on the London Underground. A neural network without memory would typically have to learn about each different transit system from scratch.\n",
"\n",
"\n",
"Basically, Many of the best unsupervised methods still require backprop (GANs, autoencoders, language models, word embeddings, etc. \n",
"\n",
"So many GANs (https://deephunt.in/the-gan-zoo-79597dc8c347)\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 7 Research Directions\n",
"\n",
"## Thesis - Unsupervised learning and reinforcement learning must be the primary modes of learning, because labels mean little to a child growing up.\n",
"\n",
"### 1 Bayesian Deep Learning (smarter backprop)\n",
"\n",
"![alt text](https://pbs.twimg.com/media/CdV2NH_W0AAfpGg.jpg\n",
" \"Logo Title Text 1\")\n",
"\n",
"- Deep learning struggles to model uncertainty.\n",
"- Lets use Smarter weight initialization via Bayes Thereom\n",
"- So in a bayesian setting the weights of your neural network are now random variables (sampled from a distribution\n",
"- The parameters of this distribution are tuned via backpropagation.\n",
"\n",
"### 2 Spike-Timing-Dependent Plasticity \n",
"\n",
"STDP is a rule that encourages neurons to 'pay more attention' to inputs that predict excitation. Suppose you usually only bring an umbrella if you have reasons to think it will rain (weather report, you see rain outside, etc.). Then you notice that if you see your neighbor carrying an umbrella, even though you haven't seen any rain in the forecast, but sure enough, a few minutes later you see an updated forecast (or it starts raining). This happens a few times, and you get the idea: Your neighbor seems to be getting this information (whether it is going to rain) before your current sources. So in the future, you pay more attention to what your neighbor is doing.\n",
"\n",
"![alt text](http://ars.els-cdn.com/content/image/1-s2.0-S0925231214017007-gr1.jpg\n",
" \"Logo Title Text 1\")\n",
"\n",
"- Suppose we have two neurons, A and B. \n",
"- A synapses onto B ( A->B ). \n",
"- The STDP rule states that if A fires and B fires after a short delay, the synapse will be potentiated (i.e. B will increase the 'weight' assigned to inputs from A in the future).\n",
"- The magnitude of the weight increase is inversely proportional to the delay between A firing and B firing. \n",
"- if A fires and then B fires ten seconds later, the weight change will be essentially zero. - - But if A fires and B fires ten milliseconds later, the weight update will be more substantial.\n",
"- The reverse also applies. If B fires first, then A, then the synapse will weaken, and the size of the change is again inversely proportional to the delay.\n",
"\n",
"TL;DR - You cannot properly backpropagate for weight updates in a graph based network since it's an asynchronous system(there are no layers with activations at fixed times), so you are trusting neurons faster than you at the task. \n",
"\n",
"\n",
"### 3 Self Organizing Maps\n",
"\n",
"![alt text](https://image.slidesharecdn.com/somchuc-110117091410-phpapp01/95/sefl-organizing-map-7-728.jpg?cb=1295255891\n",
" \"Logo Title Text 1\")\n",
" \n",
"![alt text](https://i.imgur.com/88tmp7q.png \"Logo Title Text 1\")\n",
"\n",
"### 4 Synthetic Gradients https://iamtrask.github.io/2017/03/21/synthetic-gradients/\n",
"\n",
"![alt text](https://storage.googleapis.com/deepmind-live-cms-alt/documents/3-10_18kmHY7.gif\n",
"\"Logo Title Text 1\")\n",
"\n",
"![alt text](https://iamtrask.github.io/img/synthetic_grads_paper.png\n",
"\"Logo Title Text 1\")\n",
"\n",
"- The first layer forward propagates into the Synthetic Gradient generator (M i+1), which then returns a gradient. \n",
"- This gradient is used instead of the real gradient (which would take a full forward propagation and backpropagation to compute). \n",
"- The weights are then updated as normal, pretending that this Synthetic Gradient is the real gradient. \n",
"\n",
"Synthetic Gradient genenerators are nothing other than a neural network that is trained to take the output of a layer and predict the gradient that will likely happen at that layer.\n",
"\n",
"The whole point of this technique was to allow individual neural networks to train without waiting on each other to finish forward and backpropagating.\n",
"\n",
"- Individual layers make a \"best guess\" for what they think the data will say\n",
"- then update their weights according to this guess. \n",
"- This \"best guess\" is called a Synthetic Gradient.\n",
"- The data is only used to help update each layer's \"guesser\" or Synthetic Gradient generator. \n",
"- This allows for (most of the time), individual layers to learn in isolation, which increases the speed of training.\n",
"\n",
"### 5 Evolutionary Strategies (https://blog.openai.com/evolution-strategies/)\n",
"\n",
"1. Create a random, initial brain for the bird (this is the neural network, with 300 neurons in our case)\n",
"2. At every epoch, create a batch of modifications to the bird’s brain (also called “mutations”)\n",
"3. Play the game using each modified brain and calculate the final reward\n",
"4. Update the brain by pushing it towards the mutated brains, proportionate to their relative success in the batch (the more reward a brain has been able to collect during a game, the more it contributes to the update)\n",
"5. Repeat steps 2-4 until a local maximum for rewards is reached.\n",
"\n",
"Code: \n",
"https://gist.github.com/karpathy/77fbb6a8dac5395f1b73e7a89300318d\n",
"\n",
"- Mutation, selection, crossover via a fitness function \n",
"- ES only requires the forward pass of the policy and does not require backpropagation (or value function estimation), which makes the code shorter and between 2-3 times faster in practice.\n",
"- RL is a “guess and check” on actions, while ES is a “guess and check” on parameters. \n",
"\n",
"### 6 Moar Reinforcement Learning\n",
"\n",
"![alt text](https://i.imgur.com/ytKyctO.jpg\n",
"\"Logo Title Text 1\")\n",
"\n",
"![alt text](https://adeshpande3.github.io/assets/Cover6th.png\n",
"\"Logo Title Text 1\")\n",
"\n",
"\n",
"### 7 Better hardware. \n",
"\n",
"- neuromorphic chips\n",
"- TPUs\n",
"- Wiring up transistors in parallel like the brain! \n",
"![alt text](http://4.bp.blogspot.com/-QBBjkC58sxo/ThYEopD5XPI/AAAAAAAAL9M/jcab9XA7eyY/s1600/SyNAPSE2goals.jpg\n",
"\"Logo Title Text 1\")."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# My Conclusion? I agree with Andrej Karpathy\n",
"\n",
"## Let's create multi-agent simulated enviroments that heavily rely on reinforcement learning + evolutionary strategies\n",
"\n",
"![alt text](https://i.imgur.com/KnFCWQS.png\n",
"\"Logo Title Text 1\")\n",
"\n",
"![alt text](https://i.imgur.com/spo1Ife.png\n",
"\"Logo Title Text 1\")\n",
"\n",
"It's comes down to the exploration-exploitation trade-off. You need exploitation to refine deep learning techniques but without exploration (other technqiues) you will never get the paradigm shift we need to go beyond classifying cat pictures and beating humans in artificial games"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.0"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

0 comments on commit d1349e6

Please sign in to comment.