Add benchmark for performance comparison

[taivu1998]

Stack from ghstack:

• Add benchmark for performance comparison #43221 Add benchmark for performance comparison
• Replace Conv1d with Conv2d #42867 Replace Conv1d with Conv2d

Differential Revision: D23197567

[dr-ci]

💊 CI failures summary and remediations

As of commit 0ca3f12 (more details on the Dr. CI page):

---

• 1/1 failures possibly* introduced in this PR
  • 1/1 non-CircleCI failure(s)

---

1 failure confirmed as flaky and can be ignored:

• pytorch_linux_xenial_cuda10_2_cudnn7_py3_gcc7_test

---

ci.pytorch.org: 1 failed

• Failed: pr/pytorch-linux-xenial-rocm3.5.1-py3.6

---

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group.

See how this bot performed.

This comment has been revised 16 times.

[kimishpatel]

I think you want two benchmark, no? Otherwise how are you distinguish the reported results between optimized vs unoptimized? Looking at the values reported by benchmark? I think that is fine as well.

[taivu1998]

So currently if we look at two consecutive lines, we can see the results for the two cases: optimized (Optimzed:1) and unoptimized(Optimzed:0). I think that way we can easily compare the performance for a given set of hyperparameter. Let me know if that's ok.

[kimishpatel]

Sure, thats ok.

[kimishpatel]

Looks good. Can you add the benchmark results here?

[kimishpatel]

Results are looking good. I am wondering if you can summarize the average improvement.

[taivu1998]

Average Improvement: 85%

Example Run:

Hyperparameters	Time (Unoptimized)	Time (Optimized)	Improvement
Input Channels: 32, Output Channels: 32, Kernel: 3, Batch Size: 1, Width: 32	1465880 ns	1075866 ns	27%
Input Channels: 32, Output Channels: 32, Kernel: 3, Batch Size: 1, Width: 128	2969526 ns	1904455 ns	36%
Input Channels: 32, Output Channels: 32, Kernel: 3, Batch Size: 4, Width: 64	33907082 ns	3265120 ns	90%
Input Channels: 32, Output Channels: 32, Kernel: 4, Batch Size: 3, Width: 128	69604856 ns	5152217 ns	93%
Input Channels: 32, Output Channels: 128, Kernel: 7, Batch Size: 2, Width: 64	637923970 ns	21728873 ns	97%
Input Channels: 32, Output Channels: 128, Kernel: 7, Batch Size: 4, Width: 128	1669162458 ns	106482191 ns	94%

[kimishpatel]

I mean more side-to-side comparison would have been nice. So that you can just show what is the % improvement.

[facebook-github-bot]

@kimishpatel merged this pull request in da03625.