Add Adadelta optimizer

[Lewuathe]

Adadelta is a optimization algorithm to overcome the sensitivity of learning rate selection of Adagrad.

see: http://www.matthewzeiler.com/wp-content/uploads/2017/07/googleTR2012.pdf

---

This change is 

[nsthorat]

Nice change! Will merge after a few tiny fixes, but the implementation looks correct :)

---

Reviewed 3 of 3 files at r1.
Review status: all files reviewed at latest revision, 2 unresolved discussions, some commit checks failed.

---

src/graph/session_test.ts, line 455 at r1 (raw file):

  });

  it('Safe mode math, math scope train does not throw', () => {

I totally missed this in the last reviews but the string names of these are not correct, can you rename this to "adelta"

---

src/graph/optimizers/adadelta_optimizer.ts, line 51 at r1 (raw file):

  }

remove blank line

---

Comments from Reviewable

[Lewuathe]

Review status: all files reviewed at latest revision, 2 unresolved discussions, some commit checks failed.

---

src/graph/session_test.ts, line 455 at r1 (raw file):

Previously, nsthorat (Nikhil Thorat) wrote…

I totally missed this in the last reviews but the string names of these are not correct, can you rename this to "adelta"

Sure will update.

---

Comments from Reviewable

[Lewuathe]

Review status: 1 of 3 files reviewed at latest revision, 2 unresolved discussions, some commit checks failed.

---

src/graph/optimizers/adadelta_optimizer.ts, line 51 at r1 (raw file):

Previously, nsthorat (Nikhil Thorat) wrote…

remove blank line

Do you think we can set no-consecutive-blank-lines of tslint setting?
I found several issues when I applied this setting to current codebase.

---

Comments from Reviewable

[nsthorat]

Reviewed 2 of 2 files at r2.
Review status: all files reviewed at latest revision, 1 unresolved discussion, some commit checks failed.

---

src/graph/optimizers/adadelta_optimizer.ts, line 51 at r1 (raw file):

Previously, Lewuathe (Kai Sasaki) wrote…

Do you think we can set no-consecutive-blank-lines of tslint setting?
I found several issues when I applied this setting to current codebase.

Yup, we should do that. I'll make the change at some point soon, but you're welcome to make that change and send me a PR with the fixes if you'd like, no worries if you don't want to though :)

---

Comments from Reviewable

[nsthorat]

---

Review status: all files reviewed at latest revision, all discussions resolved, some commit checks failed.

---

Comments from Reviewable