01. Introduction to Deep Learning/1.02 Biological and Artificial Neurons.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Biological and Artificial Neurons"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before going ahead, first, we will explore what are neurons and how neurons in our brain\n",
    "actually work, and then we will learn about artificial neurons.\n",
    "\n",
    "A neuron can be defined as the basic computational unit of the human brain. Neurons are\n",
    "the fundamental units of our brain and nervous system. Our brain encompasses\n",
    "approximately 100 billion neurons. Each and every neuron is connected to one another\n",
    "through a structure called a synapse, which is accountable for receiving input from the\n",
    "external environment, sensory organs for sending motor instructions to our muscles, and\n",
    "for performing other activities.\n",
    "\n",
    "A neuron can also receive inputs from the other neurons through a branchlike structure\n",
    "called a dendrite. These inputs are strengthened or weakened; that is, they are weighted\n",
    "according to their importance and then they are summed together in the cell body called\n",
    "the soma. From the cell body, these summed inputs are processed and move through the\n",
    "axons and are sent to the other neurons.\n",
    "\n",
    "The basic single biological neuron is shown in the following diagram:\n",
    "\n",
    "![images](images/3.png)\n",
    "\n",
    "Now, let's see how artificial neurons work. Let's suppose we have three inputs $x_1$, $x_2$, and $x_3$\n",
    "to predict output $y$. These inputs are multiplied by weights $w_1$, $w_2$, and $w_3$ are\n",
    "summed together as follows: \n",
    "\n",
    "\n",
    "$$x_{1} \\cdot w_{1}+x_{2} \\cdot w_{2}+x_{3} \\cdot w_{3}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But why are we multiplying these inputs by weights? Because all of the inputs are not\n",
    "equally important in calculating the output $y$. Let's say that $x_2$ is more important in\n",
    "calculating the output compared to the other two inputs. Then, we assign a higher value to $w_2$\n",
    "than the other two weights. So, upon multiplying weights with inputs, $x_2$ will have a\n",
    "higher value than the other two inputs. In simple terms, weights are used for strengthening\n",
    "the inputs. After multiplying inputs with the weights, we sum them together and we add a\n",
    "value called bias, $b$ : \n",
    "\n",
    "\n",
    "\n",
    "$$ z=\\left(x_{1} \\cdot w_{1}+x_{2} \\cdot w_{2}+x_{3} \\cdot w_{3}\\right)+b$$ \n",
    "\n",
    "\n",
    "If you look at the preceding equation closely, it may look familiar? Doesn't $z$ look like the\n",
    "equation of linear regression? Isn't it just the equation of a straight line? We know that the\n",
    "equation of a straight line is given as: \n",
    "\n",
    "$$ z=m x+b$$\n",
    "\n",
    "\n",
    "\n",
    "Here $m$ is the weights (coefficients), $x$ is the input, and $b$ is the bias (intercept).\n",
    "\n",
    "\n",
    "Well, yes. Then, what is the difference between neurons and linear regression? In neurons,\n",
    "we introduce non-linearity to the result, $z$, by applying a function $f(\\cdot)$ called the activation\n",
    "or transfer function. Thus, our output becomes:\n",
    "\n",
    "\n",
    "$$y=f(z)$$\n",
    "\n",
    "\n",
    "A single artificial neuron is shown in the following diagram:\n",
    "\n",
    "![images](images/4.jpg)\n",
    "\n",
    "\n",
    "So, a neuron takes the input, x, multiples it by weights, w, and adds bias, b, forms $z$, and\n",
    "then we apply the activation function on $z$ and get the output, $y$. \n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:anaconda]",
   "language": "python",
   "name": "conda-env-anaconda-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}