Implementing Simple Radial Basis Function Layer #2261

geekypathak21 · 2020-03-06T20:37:07Z

Hey @zoq I am trying to add simple RBF network in mlpack. Task needed to be completed are as follows. It's not completed yet I will try to complete it. Please suggest if you have any inputs for this.

Add Documentation for RBF
Add Implementation of RBF
Add Some Activation Functions
Tests for new Neural Network

rcurtin · 2020-03-12T14:54:28Z

This code touches the ANN code, and there was recently a big refactoring of the ANN code in #2259, so be sure to merge the master branch into your branch here to make sure that nothing will fail if this PR is merged. 👍 (I'm pasting this message into all possibly relevant PRs.)

favre49

There are a lot of style issues, please use the style checks and fix them all

src/mlpack/methods/ann/activation_functions/gaussian_function.hpp

src/mlpack/methods/ann/layer/radial_basis_function.hpp

src/mlpack/methods/ann/layer/base_layer.hpp

src/mlpack/methods/ann/layer/radial_basis_function.hpp

src/mlpack/methods/ann/layer/radial_basis_function_impl.hpp

mlpack-bot · 2020-04-17T18:53:39Z

This issue has been automatically marked as stale because it has not had any recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions! 👍

geekypathak21 · 2020-04-17T19:00:27Z

keep open

src/mlpack/methods/ann/layer/layer_types.hpp

src/mlpack/methods/ann/layer/radial_basis_function_impl.hpp

saksham189 · 2020-05-15T04:47:48Z

Also could you give me a link/reference to an implementation which you are referring to implement this. In most implementation I see the centres and scaling factors are being trained as well.

src/mlpack/methods/ann/layer/radial_basis_function_impl.hpp

src/mlpack/methods/ann/activation_functions/gaussian_function.hpp

saksham189 · 2020-05-19T06:21:24Z

Alright, so I do not think that we need the Gradient method. Since mlpack already has an implementation of kmeans I think it would be fine to pass the centres through the constructor.

geekypathak21 · 2020-05-19T06:23:50Z

I will try and get back to you @saksham189

src/mlpack/methods/ann/layer/radial_basis_function.hpp

saksham189 · 2020-06-12T06:29:20Z

Hey @himanshupathak21061998 I think the work here is almost complete.
Can you see why the static code analysis check is failing and maybe clean up the commit history?

saksham189 · 2020-06-12T06:36:12Z

src/mlpack/methods/ann/layer/radial_basis_function_impl.hpp

+         typename Activation>
+RBF<InputDataType, OutputDataType, Activation>::RBF() :
+    inSize(0),
+    outSize(0)


can you initialise betas and sigmas to 0 here. I think that should fix the static code analysis check.

saksham189 · 2020-06-13T09:33:13Z

Hey Himanshu did you try running the test locally with random seed?

geekypathak21 · 2020-06-13T15:03:20Z

Yup, @saksham189 already did see the comment above

I have tested with random seed locally.
When without setting beta parameter

classificationError = 0.174   for centroids = 10
classificationError = 0.13   for centroids = 20
classificationError = 0.092   for centroids = 30
classificationError = 0.092   for centroids = 40
classificationError = 0.092   for centroids = 50
classificationError = 0.078   for centroids = 60
classificationError = 0.068   for centroids = 70
classificationError = 0.088   for centroids = 80
classificationError = 0.072   for centroids = 90
classificationError = 0.07   for centroids = 100
classificationError = 0.074   for centroids = 110
classificationError = 0.09   for centroids = 120
classificationError = 0.084   for centroids = 130
classificationError = 0.06   for centroids = 140
classificationError = 0.094   for centroids = 150

When setting betas=4.1

classificationError = 0.192   for centroids = 10
classificationError = 0.124   for centroids = 20
classificationError = 0.07   for centroids = 30
classificationError = 0.082   for centroids = 40
classificationError = 0.032   for centroids = 50
classificationError = 0.068   for centroids = 60
classificationError = 0.038   for centroids = 70
classificationError = 0.042   for centroids = 80
classificationError = 0.046   for centroids = 90
classificationError = 0.02   for centroids = 100
classificationError = 0.03   for centroids = 110
classificationError = 0.026   for centroids = 120
classificationError = 0.032   for centroids = 130
classificationError = 0.02   for centroids = 140
classificationError = 0.026   for centroids = 150

saksham189 · 2020-06-13T16:18:47Z

alright great! can you run the current test locally multiple times and see if it continues to pass?
(just to make sure)

geekypathak21 · 2020-06-13T21:12:10Z

alright great! can you run the current test locally multiple times and see if it continues to pass?
(just to make sure)

Sorry for the late reply actually I have to rebuild the project which took some time I ran the tests 20 times and with no failure. I will start working on SVM from tommorow.

saksham189

I think this looks good to merge if you just make the 2 minor fixes :)

src/mlpack/methods/ann/activation_functions/gaussian_function.hpp

src/mlpack/tests/feedforward_network_test.cpp

src/mlpack/methods/ann/activation_functions/gaussian_function.hpp

Co-authored-by: saksham189 <7020962+saksham189@users.noreply.github.com>

saksham189

Thanks for the work on this!

mlpack-bot

Second approval provided automatically after 24 hours. 👍

saksham189 · 2020-06-16T15:05:42Z

Thanks again @himanshupathak21061998 !

geekypathak21 · 2020-06-16T16:03:51Z

Thanks for merging @saksham189

mlpack-bot bot added s: needs review s: unanswered s: unlabeled labels Mar 6, 2020

zoq added c: methods t: added feature and removed s: unanswered s: unlabeled labels Mar 6, 2020

geekypathak21 force-pushed the add-rbf branch from b905b7c to 8f9832b Compare March 12, 2020 22:41

favre49 reviewed Mar 15, 2020

View reviewed changes

geekypathak21 force-pushed the add-rbf branch from 34bf078 to 83f67d6 Compare March 15, 2020 13:53

mlpack-bot bot added the s: stale label Apr 17, 2020

mlpack-bot bot removed the s: stale label Apr 17, 2020

ShikharJ added the s: keep open label Apr 17, 2020

geekypathak21 force-pushed the add-rbf branch 2 times, most recently from fb61043 to 3def59d Compare May 12, 2020 16:39

geekypathak21 commented May 13, 2020

View reviewed changes

src/mlpack/methods/ann/layer/layer_types.hpp Show resolved Hide resolved

saksham189 reviewed May 13, 2020

View reviewed changes

src/mlpack/methods/ann/layer/radial_basis_function_impl.hpp Outdated Show resolved Hide resolved

geekypathak21 force-pushed the add-rbf branch 2 times, most recently from c6a8f87 to 4ed29cf Compare May 14, 2020 19:57

saksham189 reviewed May 15, 2020

View reviewed changes

src/mlpack/methods/ann/layer/radial_basis_function_impl.hpp Outdated Show resolved Hide resolved

geekypathak21 commented May 16, 2020

View reviewed changes

src/mlpack/methods/ann/layer/radial_basis_function_impl.hpp Show resolved Hide resolved

saksham189 reviewed May 17, 2020

View reviewed changes

src/mlpack/methods/ann/layer/radial_basis_function_impl.hpp Show resolved Hide resolved

geekypathak21 commented May 17, 2020

View reviewed changes

src/mlpack/methods/ann/activation_functions/gaussian_function.hpp Show resolved Hide resolved

saksham189 reviewed May 20, 2020

View reviewed changes

src/mlpack/methods/ann/layer/radial_basis_function.hpp Outdated Show resolved Hide resolved

geekypathak21 added 10 commits June 12, 2020 03:28

Adding definition of RBF class

2354b0f

Implementing forward function for rbf

9e85f09

Adding changes in RBF implementation

179cee2

Adding more changes in implementation

184d396

Adding suggested changes

c6499e0

Following the api of other layers

968d38d

Adding Gradient function

267ff5f

Changing implementation and removing gradient function

64c7bd7

Adding some changes in implementation

f2fe947

Changes in test

48f68b9

saksham189 reviewed Jun 12, 2020

View reviewed changes

geekypathak21 force-pushed the add-rbf branch 2 times, most recently from 47b544c to 151fc51 Compare June 12, 2020 19:52

Changing implementation and removing gradient function

641fb2c

geekypathak21 force-pushed the add-rbf branch from 151fc51 to 641fb2c Compare June 12, 2020 21:11

saksham189 reviewed Jun 14, 2020

View reviewed changes

src/mlpack/methods/ann/activation_functions/gaussian_function.hpp Outdated Show resolved Hide resolved

src/mlpack/tests/feedforward_network_test.cpp Outdated Show resolved Hide resolved

Some style fix

462adbf

saksham189 reviewed Jun 15, 2020

View reviewed changes

src/mlpack/methods/ann/activation_functions/gaussian_function.hpp Outdated Show resolved Hide resolved

Update src/mlpack/methods/ann/activation_functions/gaussian_function.hpp

c93c0aa

Co-authored-by: saksham189 <7020962+saksham189@users.noreply.github.com>

saksham189 approved these changes Jun 15, 2020

View reviewed changes

mlpack-bot bot approved these changes Jun 16, 2020

View reviewed changes

mlpack-bot bot removed the s: needs review label Jun 16, 2020

saksham189 merged commit 398a276 into mlpack:master Jun 16, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implementing Simple Radial Basis Function Layer #2261

Implementing Simple Radial Basis Function Layer #2261

geekypathak21 commented Mar 6, 2020 •

edited by saksham189

rcurtin commented Mar 12, 2020

favre49 left a comment

mlpack-bot bot commented Apr 17, 2020

geekypathak21 commented Apr 17, 2020

saksham189 commented May 15, 2020

saksham189 commented May 19, 2020

geekypathak21 commented May 19, 2020

saksham189 commented Jun 12, 2020 •

edited

saksham189 Jun 12, 2020

saksham189 commented Jun 13, 2020

geekypathak21 commented Jun 13, 2020

saksham189 commented Jun 13, 2020 •

edited

geekypathak21 commented Jun 13, 2020

saksham189 left a comment

saksham189 left a comment

mlpack-bot bot left a comment

saksham189 commented Jun 16, 2020

geekypathak21 commented Jun 16, 2020

Implementing Simple Radial Basis Function Layer #2261

Implementing Simple Radial Basis Function Layer #2261

Conversation

geekypathak21 commented Mar 6, 2020 • edited by saksham189

rcurtin commented Mar 12, 2020

favre49 left a comment

Choose a reason for hiding this comment

mlpack-bot bot commented Apr 17, 2020

geekypathak21 commented Apr 17, 2020

saksham189 commented May 15, 2020

saksham189 commented May 19, 2020

geekypathak21 commented May 19, 2020

saksham189 commented Jun 12, 2020 • edited

saksham189 Jun 12, 2020

Choose a reason for hiding this comment

saksham189 commented Jun 13, 2020

geekypathak21 commented Jun 13, 2020

saksham189 commented Jun 13, 2020 • edited

geekypathak21 commented Jun 13, 2020

saksham189 left a comment

Choose a reason for hiding this comment

saksham189 left a comment

Choose a reason for hiding this comment

mlpack-bot bot left a comment

Choose a reason for hiding this comment

saksham189 commented Jun 16, 2020

geekypathak21 commented Jun 16, 2020

geekypathak21 commented Mar 6, 2020 •

edited by saksham189

saksham189 commented Jun 12, 2020 •

edited

saksham189 commented Jun 13, 2020 •

edited