Add Noise layer into tfjs

[WenheLI]

This PR is about tensorflow/tfjs#1326

I added three noise layers described at Keras documentation
https://keras.io/layers/noise/

Thanks

---

This change is 

[googlebot]

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here (e.g. I signed it!) and we'll verify it.

---

What to do if you already signed the CLA

Individual signers

• It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

• Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
• The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
• The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

[WenheLI]

I signed it!

[googlebot]

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

[caisq]

Thanks for the PR, @WenheLI . The biggest comment I have right now is: please add unit tests by creating a new file called src/layers/noise_test.ts.

Also see some small comments below.

Please address the comments and I'll review the PR again.

Reviewable status: 0 of 1 approvals obtained (waiting on @WenheLI)

---

src/layers/noise.ts, line 25 at r1 (raw file):

/**

style: Doc string should go above the class, not below it. Follow the pattern in the other examples. Same elsewhere.

---

src/layers/noise.ts, line 47 at r1 (raw file):

style nit: No multiple empty lines. Only one is fine. The linter should have caught that. Same elsewhere.

[WenheLI]

Thanks @caisq . I will work on the unit test soon. Just one thing I notice that namely most layers are exported by export_layers.ts, do I need to add my layers into this file?

[caisq]

@WenheLI Good question! I forgot to comment on that. Please follow the patterns of other layers (e.g., Dense in core.ts) in order to export the factory method of the new layer type under the tf.layers.* namespace.

[WenheLI]

Hi, I just added some symbolic tests on the noise layers. Do I need to add more tests to this unit? Since I found it might be hard to test these layers with actual values, I might need some references on it to go further?

[caisq]

Reviewable status: 0 of 1 approvals obtained (waiting on @WenheLI)

---

src/layers/noise.ts, line 25 at r3 (raw file):

Standard Deviation

Style nit: all comments should end in a period. Same elsewhere.

---

src/layers/noise.ts, line 30 at r3 (raw file):

Gaussian noise.

It's help to add a comment here that the additive Gaussian Noise is applied only during
training. I know the Python doc string doesn't mention that. So it would be good to
fix that as well (in keras and tensorflow repos, as separate PRs, that is).

---

src/layers/noise.ts, line 77 at r3 (raw file):

(

Remove superfluous pair of parentheses.

---

src/layers/noise.ts, line 77 at r3 (raw file):

  (

Style: Line continuation indent is 4 chars. Same elsewhere.

---

src/layers/noise.ts, line 93 at r3 (raw file):

 * Apply multiplicative 1-centered Gaussian noise.

 *

Like I commented above, it's helpful to mention that the multiplicative noise is
applied only during training.

---

src/layers/noise.ts, line 141 at r3 (raw file):

1.0

nit: JavaScript / TypeScript don't distinguish float and int. All numbers are float. So remove the decimal points and the trailing zero. Same elsewhere.

---

src/layers/noise.ts, line 169 at r3 (raw file):

 *

nit: Remove extra line.

---

src/layers/noise.ts, line 223 at r3 (raw file):

rate = this.rate

Why include this additional argument? The value of this.rate is always used anyway.

---

src/layers/noise.ts, line 223 at r3 (raw file):

input = inputs,

No need to have a default value here.

---

src/layers/noise_test.ts, line 12 at r3 (raw file):

* Unit Tests for Noise Layers

Apart from these symbolic tests, we need tests based on concrete tensor values.

Follow the examples for other layers such as Dense.

[WenheLI]

@caisq Thanks for your comments. I have fixed most of your comments. Just one more thing remains to be unfinished that is the unit tests for concrete values.

Since all the three noise layers will get random results, I am thinking test if their outputs will be a normal distribution that requires a function jarqueBeraNormalityTest in @tensorflow/tfjs-core/src/ops/rand_util which is not exported. Do I need to manually export it?
Or I should give these layers a fixed seed and test if the output matches our expected value given the fixed seed?

Thanks!

[caisq]

@WenheLI Good question about how to test the new layers with concrete values. As you pointed out, the outputs will be random (during training, that is). I suggest cover the following aspects:

1. That the call() method executes without any error when the kwarg training is set to false, and that the output
   tensor should be equal to the input tensor.
2. That the call() method executes without any error when the kwarg training is set to true, and that the output has the same dtype and shape as the input. In addition, the output should be unequal to the input. Note that we currently don't have a method like expectTensorsUnequal. But you can do that kind of assertion by calculating the maximum of the absolute difference between the input and output, e.g., output.sub(input).abs().max() and assert the result is greater than 0.
3. That two consecutive calls tocall() give different results, if kwarg training is true.

Thanks.

Reviewable status: 0 of 1 approvals obtained (waiting on @WenheLI)

[WenheLI]

@caisq Thanks for your suggestions! I have added the unit test. Let me know if there is any further problem!

[caisq]

Thanks a lot, @WenheLI . Let's wait for the tests to pass. Then we can merge the PR.

Reviewable status: 0 of 1 approvals obtained (waiting on @WenheLI)