Skip to content

Commit

Permalink
Update ANN_intro_and_ Entropy.rst
Browse files Browse the repository at this point in the history
  • Loading branch information
Sahar Niknam committed Oct 6, 2018
1 parent ee93cae commit 3104e9d
Showing 1 changed file with 9 additions and 18 deletions.
27 changes: 9 additions & 18 deletions docs/ANN_intro_and_ Entropy.rst
Original file line number Diff line number Diff line change
@@ -1,26 +1,17 @@
**- What are artificial neural networks?**
- What are artificial neural networks?
- What is entropy?
- How is entropy useful for understanding artificial neural networks?

**- What is entropy?**

**- How is entropy useful for understanding artificial neural networks?**

## What are artificial neural networks?

What are artificial neural networks?
====================================
Making some sorts of artificial lives, capable of acting humanly-rational, has been a long lasting dream of mankind. We started with mechanical bodies, working solely based on laws of physics, which were mostly fun creatures rather than intelligent ones. The big leap took place as we stepped in the programmable computers era; when the focus shifted to those features of human skills which were a bit more brainy. So the results became more serious and successful. Codes started beating us in some aspects of intelligence which involve memory and speed, especially when they were tested using well, and formally, structured problems. But their Achilles Heel was the tasks that need a bit of intuition and our banal common sense. So, while codes were instructed to outperform us at solving elegant logical problems at which our brains are miserably weak, they failed to carry out some simple trivial tasks that we are able to do, even without consciously thinking about them. It was like we made an intangible creature who is actually intelligent, but in a different direction perpendicular to the direction of our intelligence. Thus, we thought if we really want something that act similar to us, we need to structure it just like ourselves. And that was the very reason for all the efforts that finally led to the realization of artificial neural networks (ANNs).

Unlike the conventional codes, which are instructed what to do, in a step-by-step way and by a human supervisor, neural networks learn stuff by observing data. It is almost the same as the way our brain learns doing intuitive tasks that we have no clear idea of exactly how and exactly when we have learned doing them; for example, a trivial task like recognizing a fire hydrant in a picture of a random street. And that was the way we chose to tackle the common sense problem of AI. So, what are these neural networks?

###### Perceptron
Let’s start with the perceptron, which is a mathematical model of a single neuron and the plainest version of an artificial neural network: a network with one single-node layer. However, from a practical point of view, a perceptron is only a humble classifier that divide input data into two categories: the ones that cause our artificial neuron fires, and the ones that does not. The procedure is like this: the perceptron takes one or multiple real numbers as input, sums over a weighted version of them, adds a constant value, bias, to the result, and then uses it as the net input to its activation function. That is the function that calculates if the perceptron is going to be activated with the inputs or not. The perceptron uses Heaviside step function as its activation function. So the output of this function is the the perceptron output.
What is entropy?
================

'''
figure 1 will be placed here.
Thanks for your patience :blush:
'''

In math language, a perceptron is a simple equation:

'''
equation 1 will be placed here.
Thanks for your patience :blush:
'''
How is entropy useful for understanding artificial neural networks?
===================================================================

0 comments on commit 3104e9d

Please sign in to comment.