make_variable consumes the Tensor if it only has one reference

[jamesr66a]

Stack from ghstack:

• make_variable consumes the Tensor if it only has one reference #22705 make_variable consumes the Tensor if it only has one reference

This significantly reduces the overhead of operator dispatch. Benchmark:

import torch, time

NITER = 10000000

def foo(x):
    for i in range(10000000):
        x = torch.neg(x)
    return x

with torch.no_grad():
    scripted = torch.jit.script(foo)
    x = torch.rand([])

    s = time.time()
    scripted(x)
    print('script time', (time.time() - s) / NITER / 1e-6, 'us')

Before

script time 1.3357892751693727 us

After

script time 1.129352855682373 us

TODO: I don't know if this is valid if the Storage is shared

Differential Revision: D16192220

[ajtulloch]

Isn't this unsafe in a multithreaded environment? Isn't it possible to have two threads invoking make_variable(tensor) concurrently on the same tensor instance?

[jamesr66a]

Wouldn't two threads have to move() the same instance into the call to make_variable? Or is there another sequencing you're thinking of?

[jamesr66a]

Or more directly: i believe this is safe because data is taken by value

[ajtulloch]

Ah yeah, you're right - taking by value makes this safe. The interleaving I was thinking of two threads invoking make_variable(t); one threads passes the use_count check tries to move, and the other thread fails the use_count check and tries to copy from the moved-from value, but that can't happen since neither could pass the use_count check.

[zdevito]

NGNT

[facebook-github-bot]

@jamesr66a merged this pull request in 815e73b.