Add block matmul

[aman-tiwari]

Description

Currently: 1.4x times speed up on a 512x512 matrix-matrix matmul.

In response to tensorflow/tfjs#582

---

For repository owners only:

Please remember to apply all applicable tags to your pull request.
Tags: FEATURE, BREAKING, BUG, PERF, DEV, DOC, SECURITY

For more info see: https://github.com/tensorflow/tfjs/blob/master/DEVELOPMENT.md

---

This change is 

[aman-tiwari]

This is the benchmarking code (removed it from the PR)

  it('benchmark matmul sq matrix', async done => {
    const backend = tf.ENV.backend as MathBackendCPU;
    const bs = [32, 48, 64, (64 / 2) + 64, 128, (128 / 2) + 128];
    const ns = [64, 128, 192, 256, 239, 398, 512];
    const RUNS = 20;
    for (const n of ns) {
      const a = tf.randomUniform([n, n]) as tf.Tensor2D;
      const b = tf.randomUniform([n, n]) as tf.Tensor2D;
      // Warmup.
      backend.matMulNaive(a, b, false, false).dataSync();

      let res: tf.Tensor = null;
      const start = now();
      for (let i = 0; i < RUNS; i++) {
        res = backend.matMulNaive(a, b, false, false);
      }
      res.dataSync();
      const naiveTime = (now() - start) / RUNS;
      console.log(`N: ${n}\t ${naiveTime.toFixed(2)}ms`);

      for (const blockSize of bs) {
        backend.blockSize = blockSize;
        const a = tf.randomUniform([n, n]) as tf.Tensor2D;
        const b = tf.randomUniform([n, n]) as tf.Tensor2D;
        // Warmup.
        backend.matMul(a, b, false, false).dataSync();

        let res: tf.Tensor = null;
        const start = now();
        for (let i = 0; i < RUNS; i++) {
          res = backend.matMul(a, b, false, false);
        }
        res.dataSync();
        const elapsed = (now() - start) / RUNS;
        const speedup = (naiveTime / elapsed).toFixed(2);
        console.log(
            `mul BS: ${blockSize}\t ${elapsed.toFixed(2)} ms\t speedup: ${
                speedup}x\t diff:${(elapsed - naiveTime).toFixed(2)}ms`);
        await tf.nextFrame();
      }
    }
    done();
  });

[dsmilkov]

Reviewed 1 of 3 files at r1, 1 of 2 files at r2, 1 of 1 files at r3, 1 of 1 files at r4.
Reviewable status: 0 of 1 approvals obtained

[aman-tiwari]

By running tests on matrices we found that the block size for the cache blocked matrix multiply was 48. We suffer a performance penalty of 0.5-1ms on small matrices but gain 100s of milliseconds on large matrices.

[dsmilkov]

Reviewed 1 of 1 files at r5.
Reviewable status: complete! 1 of 1 approvals obtained