construct only necessary elements in OffsetCalculator

[ngimel]

Per title. Elements beyond dim are never accessed because

pytorch/aten/src/ATen/cuda/detail/OffsetCalculator.cuh

Lines 49 to 51 in 646510f

	if (dim == dims) {
	break;
	}

.
On addmm instruction count per 30 repetitions 1467813 -> 1452261
add 651522 -> 633462
add_ 529331 -> 511271

add benchmarking snippet:

 timer = Timer("m1.add_(b);", setup="at::Tensor m1=torch::empty({2,2},device(at::kCUDA) ); at::Tensor b = torch::empty({2}, device(at::kCUDA));", language="c++", timer=timeit.default_timer)
 stats=timer.collect_callgrind(number=30)
 print(stats.as_standardized().stats(inclusive=False))

[facebook-github-bot]

💊 CI failures summary and remediations

As of commit 646510f (more details on the Dr. CI page):

---

• 2/2 failures possibly* introduced in this PR
  • 2/2 non-scanned failure(s)

---

ci.pytorch.org: 1 failed

• Failed: pr/pytorch-linux-bionic-rocm4.0.1-py3.6

---

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

[codecov]

Codecov Report

Merging #55107 (646510f) into master (4865195) will increase coverage by 0.34%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #55107      +/-   ##
==========================================
+ Coverage   77.08%   77.42%   +0.34%     
==========================================
  Files        1893     1893              
  Lines      186444   186444              
==========================================
+ Hits       143715   144362     +647     
+ Misses      42729    42082     -647     

[ezyang]

nice catch!

[facebook-github-bot]

@ngimel has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

[swolchok]

btw, might be worth emitting two loops here depending on whether element_sizes == nullptr to avoid a bunch of idiv instructions just to divide by 1.

[ngimel]

Good point, I'll do that.

[ngimel]

Tried it, it actually increases instruction count, and in most cases element_sizes is not nullptr.
It is silly that TI build multiplies by element sizes and stores stride in bytes, and here we copy element_sizes into array (true story,

pytorch/aten/src/ATen/native/cuda/Loops.cuh

Line 43 in 5610e82

element_sizes[i] = iter.element_size(i);

) and then divide by them, but that's outside of this fix scope.

[facebook-github-bot]

@ngimel merged this pull request in 0d47374.