# ShomyLiu/stat-learn

Switch branches/tags
Nothing to show
Fetching latest commit…
Cannot retrieve the latest commit at this time.
 .. Failed to load latest commit information. README.md kmeans.py rbf.py run.py tf_kmeans.py tf_rbf.py tf_run.py

http://shomy.top/2017/02/26/rbf-network/

## 实现

### 数据

def getData(n=100, d=5, method='sum'):
'''
生成数据
'''
X = np.random.uniform(1., 3.0,(n,d))
if method == 'sum':
y = np.sum(X, axis=1)
else:
y = 2*X
return X,y

### RBFNet

\begin{align}h(\mathbf{x}) &= \sum\limits_{m=1}^M\beta_mRBF(\mathbf{x}, \mu_m) \ &=\sum\limits_{m=1}^M\beta_m * \exp(-\gamma_m * ||\mathbf{x}-\mathbf{c}||^2) \end{align}

• 计算出每个簇内的发散平均程度，即距离中心的平均距离: $\sigma = \frac{1}{m}\sum||\mathbf{x}-\mu_m||$
• $\gamma_m = \frac{1}{2\sigma^2}$

• 使用Linear Regression就可以直接解出$\beta$:$\beta = (\mathbf{Z^TZ})^{-1}\mathbf{Z}^T\mathbf{y}$，其中$\mathbf{Z}$是所有训练数据的rbf的结果，$\mathbf{y}$是所有训练数据的y。
• 按照梯度下降的方式来训练参数(只在TensorFlow版本实现)

### 结果

CPU版本：

1. 使用$y=sum(x)$的3000条数据，逼近结果如下:
$python run.py X: [ 2.76801352 2.09847071 1.26193386 1.24845461 1.40333726] real : 8.78020996497 prediction: 8.71215929411 ****************************** X: [ 1.00621598 1.81564319 1.59844746 2.52523288 1.27452441] real : 8.22006392064 prediction: 8.24858828932 ****************************** X: [ 2.90835704 2.22760245 1.80146849 1.37866698 1.21111461] real : 9.52720958234 prediction: 9.40142232908 ****************************** X: [ 1.93636418 2.54609154 1.80642139 2.33747451 1.25772174] real : 9.88407335723 prediction: 10.1683670115 ****************************** X: [ 2.80741503 1.35740491 2.26844426 2.62627921 1.73952771] real : 10.7990711232 prediction: 10.8892856428 ****************************** python run.py 9.02s user 0.50s system 106% cpu 8.947 total 1. 使用$y=2x$的3000条数据，逼近效果如下: $ python run.py
X: [ 2.64225429  1.16385274  2.69488719  2.58292883  2.31537922]
real      : [ 5.28450858  2.32770548  5.38977438  5.16585766  4.63075843]
prediction: [ 5.11075705  2.36091391  5.29326006  5.02440494  4.63639069]
******************************
X: [ 2.79241969  2.69607337  2.07437327  2.73446838  1.19582012]
real      : [ 5.58483938  5.39214674  4.14874654  5.46893677  2.39164025]
prediction: [ 5.21027141  5.08792887  4.06967137  5.14665798  2.53483961]
******************************
X: [ 2.60551718  2.97657156  2.80778824  2.47291552  2.84918172]
real      : [ 5.21103436  5.95314312  5.61557648  4.94583105  5.69836345]
prediction: [ 4.84442944  5.35447225  5.0674437   4.57218657  5.18815626]
******************************
X: [ 2.44505957  1.70038561  1.99612367  1.1245514   1.18954801]
real      : [ 4.89011913  3.40077121  3.99224733  2.2491028   2.37909603]
prediction: [ 4.8029322   3.39746125  3.94098729  2.30900805  2.52864718]
******************************
X: [ 1.85397812  2.83347041  1.31929703  1.35131915  2.29165357]
real      : [ 3.70795624  5.66694081  2.63859405  2.7026383   4.58330715]
prediction: [ 3.66509186  5.63858479  2.65371075  2.73222885  4.53157419]
******************************
python run.py  6.84s user 0.58s system 114% cpu 4.751 total

GPU版本结果:

1. 使用$y=sum(x)$的3000数据:

X: [ 1.62451696  1.02659273  2.81655526  1.12427032  2.43579245]
real      : [ 9.02772713]
prediction: [ 8.7921114]
******************************
X: [ 2.53124356  1.34387374  1.38431156  1.20624399  2.59180689]
real      : [ 9.05747986]
prediction: [ 8.99416161]
******************************
X: [ 1.70616233  2.51688766  1.07848394  1.27796292  1.98958588]
real      : [ 8.56908321]
prediction: [ 8.56313705]
******************************
X: [ 2.03224301  2.65368748  1.41390944  2.04034257  1.04631364]
real      : [ 9.18649578]
prediction: [ 9.23960781]
******************************
X: [ 2.16495895  1.28095651  1.34778249  2.75269151  1.90516138]
real      : [ 9.45155144]
prediction: [ 9.43926144]
******************************
python tf_run.py  3.53s user 0.85s system 166% cpu 2.630 total

2. 使用$y=2x$的3000条训练数据:

X: [ 1.06009257  1.647856    1.8015914   1.13952839  2.74504972]
real      : [ 2.12018514  3.29571199  3.60318279  2.27905679  5.49009943]
prediction: [ 2.36333179  3.43519783  3.55674648  2.42750287  5.14753294]
******************************
X: [ 2.94805002  1.19944155  1.2205795   2.25700951  2.18170953]
real      : [ 5.89610004  2.3988831   2.44115901  4.51401901  4.36341906]
prediction: [ 5.56341791  2.54713631  2.6267767   4.27490234  4.30334949]
******************************
X: [ 1.93198025  1.48437035  2.22421408  1.9887476   2.1166265 ]
real      : [ 3.8639605   2.9687407   4.44842815  3.97749519  4.233253  ]
prediction: [ 4.06275845  2.60474968  4.85847521  4.15260172  4.46441317]
******************************
X: [ 1.86214089  2.13927388  2.89492273  2.43535662  1.00828433]
real      : [ 3.72428179  4.27854776  5.78984547  4.87071323  2.01656866]
prediction: [ 3.73180866  4.23187256  5.57253933  4.6572423   2.15151691]
******************************
X: [ 1.35404587  2.86987972  2.46327639  2.51390696  1.88503158]
real      : [ 2.70809174  5.73975945  4.92655277  5.02781391  3.77006316]
prediction: [ 2.71249247  5.52877426  4.94359541  4.8839283   3.81015444]
******************************
python tf_run.py  4.70s user 0.86s system 165% cpu 3.363 total


1. Linear Regression方式:

X: [ 2.71186709  1.1967051   1.31952727  2.1717248   2.57859206]
real      : [ 9.97841644]
prediction: [ 9.96531868]
******************************
X: [ 2.73147416  1.85858846  1.61863422  2.991575    2.25436425]
real      : [ 11.45463562]
prediction: [ 11.45114994]
******************************
X: [ 1.70037389  1.58699119  1.53201306  2.21282601  1.39819622]
real      : [ 8.43039989]
prediction: [ 8.39311886]
******************************
X: [ 2.01777792  1.22136164  1.52503204  2.9318645   1.15536594]
real      : [ 8.85140228]
prediction: [ 8.84553528]
******************************
X: [ 2.70246601  2.19741511  1.7523278   2.72755265  1.09670925]
real      : [ 10.47647095]
prediction: [ 10.4424572]
******************************
2. 梯度下降训练方式:

X: [ 2.3670311   2.37218928  1.41229916  2.26609325  1.49073756]
real      : [ 9.90835094]
prediction: [ 10.1823082]
******************************
X: [ 2.30584574  1.68435013  1.76450276  2.61775756  2.94909477]
real      : [ 11.32155132]
prediction: [ 11.11611843]
******************************
X: [ 2.48293662  1.83565235  1.58969617  1.05468404  2.01141191]
real      : [ 8.97438145]
prediction: [ 9.28915977]
******************************
X: [ 2.92746592  1.39118993  2.33665395  1.11171508  2.27592731]
real      : [ 10.04295254]
prediction: [ 9.89380741]
******************************
X: [ 1.08498883  1.48899996  1.32145357  2.53989029  2.54879212]
real      : [ 8.98412514]
prediction: [ 9.06929016]


Numpy版KMeans+RBFNet 与 TensorFlow版KMeans+RBFNet的完整代码: RBFNet With K-Means