Skip to content
This repository has been archived by the owner on Jan 24, 2024. It is now read-only.

Commit

Permalink
make the formula consistent with the image
Browse files Browse the repository at this point in the history
  • Loading branch information
dayhaha committed Jan 2, 2017
1 parent ba01384 commit 1d618e6
Show file tree
Hide file tree
Showing 3 changed files with 9 additions and 6 deletions.
15 changes: 9 additions & 6 deletions recognize_digits/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,17 +39,19 @@ MNIST数据库的图片为28X28的二维图像,为了进行计算,我们一
比如说,如果图片的数字标签为3,则$y_i$组成的向量为[0,0,0,1,0,0,0,0,0,0]。输入层的数据x传到softmax层,在激活操作之
前,会乘以相应的权重w,并加上偏置变量b,具体如下:
$$ net_i = \sum_j W_{i,j}x_j + b_i $$
然后再用softmax函数进行激活:
$$ predict = softmax(net) $$
net是激活前的值(仅仅进行了矩阵乘法),然后再用softmax函数进行激活:
其中softmax函数的定义如下:
$$ softmax(x_i) = \frac{exp(x_i)}{\sum_j exp(x_j)} $$
$$ softmax(x_i) = \frac{e^{x_i}}{\sum_j e^{x_j}} $$
即:
$$ y_i = \frac{e^{net_i}}{\sum_j e^{net_j}} $$
神经网络的训练采用backpropagation的形式,其一般会定义一个损失函数(也称目标函数),训练的目的是为了减小目标函数的
值。在分类问题中,我们一般采用交叉熵代价损失函数(cross entropy),其形式如下:
值。在分类问题中,我们一般采用交叉熵代价损失函数(cross entropy)。

以下为softmax回归的网络图:
<p align="center">
<img src="image/softmax_regression.png"><br/>
图2. softmax回归网络结构图<br/>
注:图中得到net需再加上b偏置量,输出层p就是预测的结果predict
注:图中权重用黑线表示,偏置用红线表示,+1代表偏置参数的系数为1
</p>


Expand All @@ -64,11 +66,12 @@ $$ H_1 = activation(W_1X + b_1) $$
经过第二层网络,可以得到:
$$ H_2 = activation(W_2H_1 + b_2) $$
最后,再经过输出层:
$$ P = softmax(W_3H_2 + b_3) $$
$$ Y = softmax(W_3H_2 + b_3) $$
得到的P即为最后的预测结果向量。
<p align="center">
<img src="image/MLP.png"><br/>
图3. 多层感知器网络结构图<br/>
注:图中权重用黑线表示,偏置用红线表示,+1代表偏置参数的系数为1
</p>

#### 常见激活函数介绍
Expand Down
Binary file modified recognize_digits/image/MLP.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified recognize_digits/image/softmax_regression.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 1d618e6

Please sign in to comment.