Add kernel timing information to tf.profile()

[Linchenn]

This PR does the following:

• add kernelTimeMs (as discussed in this ISSUE) and extraInfo (GPU programs) information into the result of tf.profile()

To see the logs from the Cloud Build CI, please join either our discussion or announcement mailing list.

---

This change is 

[Linchenn]

Reviewable status: 0 of 1 approvals obtained

---

tfjs-core/src/engine.ts, line 649 at r1 (raw file):

        outputShapes: outputs.map(item => item.shape),
        kernelTimeMs: kernelProfile.timeMs,
        extraInfo: kernelProfile.extraInfo

Here, kernelTimeMs and extraInfo are promises. I resolve them at tf.profile(), because the runKernel is not an async function.

Another solution is to resolve them within runKernel by then():

    if (this.state.profiling) {
      Promise.all([kernelProfile.timeMs, kernelProfile.extraInfo]).then(data => {
        this.state.activeProfile.kernels.push({
          name: kernelName,
          ...
          kernelTimeMs: data[0],
          extraInfo: data[1]
        });
      });
    }

Since above code of the second solution is relatively complicate, I chose the first solution.

[annxingyuan]

Thank you Linchenn this is great! I just left a tiny question regarding one of your comments below - once you address I will approve!

Reviewable status: 0 of 1 approvals obtained (waiting on @annxingyuan, @lina128, @Linchenn, and @pyu10055)

---

tfjs-core/src/engine_test.ts, line 399 at r1 (raw file):

    expect(typeof profile.kernels[1].extraInfo).toBe('string');

    // The `kernelTimeMs` and `extraInfo` are tested in Profiler.profileKernel

sorry I'm a little confused about this comment - it looks like kernelTimeMs and extraInfo are tested in the lines above? what do you mean that they are tested in Profiler.profileKernel?

[Linchenn]

Thanks Ann!

Reviewable status: 0 of 1 approvals obtained (waiting on @annxingyuan, @lina128, and @pyu10055)

---

tfjs-core/src/engine_test.ts, line 399 at r1 (raw file):

Previously, annxingyuan (Ann Yuan) wrote…

sorry I'm a little confused about this comment - it looks like kernelTimeMs and extraInfo are tested in the lines above? what do you mean that they are tested in Profiler.profileKernel?

Thanks for pointing it out, and, the comment here is not exact enough. Changed.

The above lines only test whether the promises are resolved, while the values of kernelTimeMs and extraInfo are tested in Profiler.profileKernel's tests.

---

tfjs-core/src/engine_test.ts, line 401 at r1 (raw file):

    // The `kernelTimeMs` and `extraInfo` are tested in Profiler.profileKernel
    // and are excluded from here.
    expect(profile.kernels[0]).toEqual(jasmine.objectContaining({

I saw our codebase rarely uses jasmine.objectContaining, so do you think it is better to include kernelTimeMs and extraInfo into the object and use toEqual here? like:

    expect(profile.kernels[0]).toEqual({
      'name': 'Square',
      ...
      'outputShapes': [[3]],

      'kernelTimeMs': profile.kernels[0].kernelTimeMs,
      'extraInfo': profile.kernels[0].extraInfo

    });

[annxingyuan]

Reviewable status: complete! 1 of 1 approvals obtained (waiting on @annxingyuan, @lina128, @Linchenn, and @pyu10055)

---

tfjs-core/src/engine_test.ts, line 399 at r1 (raw file):

Previously, Linchenn wrote…

Thanks for pointing it out, and, the comment here is not exact enough. Changed.

The above lines only test whether the promises are resolved, while the values of kernelTimeMs and extraInfo are tested in Profiler.profileKernel's tests.

if you wanted to you could build a stub profiler like we do in profiler_test.ts - otherwise sure asserting that the kernelTimeMs and extraInfo are equal to itself is fine - just add a comment about why we're not asserting that they equal particular values

[Linchenn]

Reviewable status: complete! 1 of 1 approvals obtained (waiting on @annxingyuan, @lina128, and @pyu10055)

---

tfjs-core/src/engine_test.ts, line 399 at r1 (raw file):

Previously, annxingyuan (Ann Yuan) wrote…

if you wanted to you could build a stub profiler like we do in profiler_test.ts - otherwise sure asserting that the kernelTimeMs and extraInfo are equal to itself is fine - just add a comment about why we're not asserting that they equal particular values

Wise points! Thank you so much Ann.