Fix 1) tanh underflow error, 2) scoping bug and add 3) debug mode for math

[dsmilkov]

• Fix tanh overflow error where tanh(-100) was NaN.
• Fix scope bug where scope's return result of type NDArray[] was not kept in the parent scope
• Add debug mode for math, where it throws an error the first time a NaN entry is found.

---

This change is 

[nsthorat]

---

Reviewed 3 of 4 files at r1.
Review status: 3 of 4 files reviewed at latest revision, 2 unresolved discussions.

---

src/math/math.ts, line 67 at r1 (raw file):

   * and checked for NaNs. This significantly impacts performance.
   */
  enableDebugMode() {

I wonder if we should just call this enableNaNDetection or something more than just "enableDebugMode", unless you can think of some other things debug mode will do. For any other xxx detection we could have a bit for that as well. WDYT?

---

src/math/math.ts, line 159 at r1 (raw file):

  }

  private checkForNaN(arr: NDArray): void {

can you add a unit test for this?

---

Comments from Reviewable

[nsthorat]

Reviewed 1 of 4 files at r1.
Review status: all files reviewed at latest revision, all discussions resolved, all commit checks successful.

---

Comments from Reviewable

[dsmilkov]

Review status: 3 of 4 files reviewed at latest revision, 2 unresolved discussions, some commit checks pending.

---

src/math/math.ts, line 67 at r1 (raw file):

Previously, nsthorat (Nikhil Thorat) wrote…

I wonder if we should just call this enableNaNDetection or something more than just "enableDebugMode", unless you can think of some other things debug mode will do. For any other xxx detection we could have a bit for that as well. WDYT?

I can imagine later adding other things to debug mode and not having the user call several debug methods. For example, since we are already downloading all of the values to CPU, we might want to console log [min, 5%-tile, median, 95%-tile, max] for each tensor, log the operation of the last math call (obtained using console.trace()), log the execution time (when we add gpu callbacks in webgl 2.0) etc

---

src/math/math.ts, line 159 at r1 (raw file):

Previously, nsthorat (Nikhil Thorat) wrote…

can you add a unit test for this?

Done.

---

Comments from Reviewable