Subject: Add swish activation function support (#1804)

[Triforcey]

Body:
FEATURE

Created Swish activation function class and unit tests

To see the logs from the Cloud Build CI, please join either our discussion or announcement mailing list.

---

This change is 

[googlebot]

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

---

What to do if you already signed the CLA

Individual signers

• It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

• Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
• The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
• The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

[googlebot]

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

[caisq]

Reviewable status: 0 of 1 approvals obtained (waiting on @caisq and @Triforcey)

---

tfjs-layers/src/activations.ts, line 224 at r2 (raw file):

alpha = 1

Is this necessary? Can you provide links to related document or code? I haven't seen the Swish activation with this parameter before. Also it appears to be not used anywhere or exposed to the public API.

[Triforcey]

Yes, that variable is a part of the function's definition. It is referenced as β here: https://en.wikipedia.org/wiki/Swish_function. I still need to expose this to the API as a trainable parameter. I'm going to do that. I would appreciate any pointers you may have on how to go about doing that.

[caisq]

Reviewable status: complete! 1 of 1 approvals obtained (waiting on @Triforcey)

---

tfjs-layers/src/activations.ts, line 224 at r2 (raw file):

Previously, caisq (Shanqing Cai) wrote…

alpha = 1

Is this necessary? Can you provide links to related document or code? I haven't seen the Swish activation with this parameter before. Also it appears to be not used anywhere or exposed to the public API.

@Triforcey OK. I'n fine with leaving this argument in for the time being. However, for exposing it to public API, we generally strive to be consistent with our Python counterpart (tf.keras). In the case of swish, it seems that the Python counter part doesn't expose this as an argument or a trainable variable yet. See:

1. https://www.tensorflow.org/api_docs/python/tf/keras/activations/swish
2. https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/ops/nn_impl.py#L551

[Triforcey]

Okay, sounds good. Thank you for your assistance.

[Triforcey]

@caisq the branch is ready to be merged. 😄

[caisq]

@rthadur @pyu10055 feel free to merge the PR when you're comfortable.