Make random number generator state a TH type

[timharley]

State of the random number generator is moved from static data to a first-class type within TH: a THGenerator. These are then available within torch as a torch.Generator object. A default generator added in to the torch namespace when loading the package and used implicitly for any random functions.

• Behaviour of the random number stream is identical [see below for a test]. i.e. outstanding bugs are not fixed.
• The torch API is backwards compatible (all unit tests pass with no changes).
• This patch makes random number generation thread safe between different interpreters within a single process.
• Additionally, the torch API is extended, allowing you to explicitly pass a generator object to any particular random function, so you can have parallel random number streams within a single lua program.

[timharley]

To test that the random number stream is identical after these changes, I ran this test with each version of torch. The output files are binary equal for the same seed.

require 'nn'

local seed = arg[1] or 123
local output_file = arg[2] or 'test.out'

torch.manualSeed(seed)

input = torch.randn(1,20,20)
m=nn.Sequential()
m:add(nn.SpatialConvolution(1,10,5,5))
m:add(nn.Tanh())
m:add(nn.Reshape(10_16_16))
m:add(nn.Linear(10_16_16,20))
m:zeroGradParameters()
local x,dx = m:getParameters()
dx:zero()

--Forward
m:forward(input)
local go = torch.rand(20)
m:backward(input,go)

torch.save(output_file,{x=x,dx=dx,o=m.output,gi=m.gradInput})

[soumith]

hey, this is fantastic work, thank you for working on this.
none of us touched this task of moving the generator state to lua side, because it is such a pain to wrap around :)

my only question is, before we pull this in, we make sure that there's no speed bottlenecks, if you could benchmark the RNG speed (from a single number, small tensor to huge tensors) over master branch vs your branch, we'd get an idea about speed. For example, benchmark torch.rand from 2-element tensors all the way to 200k.

[timharley]

This patch is C89 compatible,

export CFLAGS="-std=c89 -pedantic"

doesn't throw up any warnings.

[timharley]

With these changes random number generation is ~5-10% slower.

All times generated from an average over 2x10^9 random number generations.

master:
Average time rand torch.rand(, 1) is: 1.3707073307037e-06
Average time rand torch.rand(, 10) is: 1.5792630553246e-06
Average time rand torch.rand(, 100) is: 2.5713230371475e-06
Average time rand torch.rand(, 1000) is: 1.2444764375687e-05
Average time rand torch.rand(, 10000) is: 0.00010921930074692
Average time rand torch.rand(, 100000) is: 0.001081223487854
Average time rand torch.rand(, 1000000) is: 0.011473865509033
Average time rand torch.randn(, 1) is: 1.6775269258022e-06
Average time rand torch.randn(, 10) is: 2.5748799443245e-06
Average time rand torch.randn(, 100) is: 1.1024998545647e-05
Average time rand torch.randn(, 1000) is: 9.1695610284805e-05
Average time rand torch.randn(, 10000) is: 0.00093235199451447
Average time rand torch.randn(, 100000) is: 0.0093500734567642
Average time rand torch.randn(, 1000000) is: 0.091872580051422
Average time for tensor:uniform() on 1 element tensor is: 1.3819939899445e-06
Average time for tensor:uniform() on 10 element tensor is: 1.4446853995323e-06
Average time for tensor:uniform() on 100 element tensor is: 2.4154500961304e-06
Average time for tensor:uniform() on 1000 element tensor is: 1.344687461853e-05
Average time for tensor:uniform() on 10000 element tensor is: 0.00012447719573975
Average time for tensor:uniform() on 100000 element tensor is: 0.0012220104932785
Average time for tensor:uniform() on 1000000 element tensor is: 0.0123568546772

generator branch
Average time rand torch.rand(, 1) is: 1.4173873603344e-06
Average time rand torch.rand(, 10) is: 1.528921353817e-06
Average time rand torch.rand(, 100) is: 2.6873714923859e-06
Average time rand torch.rand(, 1000) is: 1.387915968895e-05
Average time rand torch.rand(, 10000) is: 0.00012145735025406
Average time rand torch.rand(, 100000) is: 0.0011950484514236
Average time rand torch.rand(, 1000000) is: 0.01229336977005
Average time rand torch.randn(, 1) is: 1.5438157200813e-06
Average time rand torch.randn(, 10) is: 2.5391666531563e-06
Average time rand torch.randn(, 100) is: 1.1269260525703e-05
Average time rand torch.randn(, 1000) is: 0.00010158686041832
Average time rand torch.randn(, 10000) is: 0.00098479055166245
Average time rand torch.randn(, 100000) is: 0.0098458305597305
Average time rand torch.randn(, 1000000) is: 0.10014077544212
Average time for tensor:uniform() on 1 element tensor is: 1.5280487501621e-06
Average time for tensor:uniform() on 10 element tensor is: 1.6711731433868e-06
Average time for tensor:uniform() on 100 element tensor is: 2.9455724954605e-06
Average time for tensor:uniform() on 1000 element tensor is: 1.4268969297409e-05
Average time for tensor:uniform() on 10000 element tensor is: 0.00013465700149536
Average time for tensor:uniform() on 100000 element tensor is: 0.0012339264154434
Average time for tensor:uniform() on 1000000 element tensor is: 0.013465664386749

code:

require 'torch'
require 'sys'
require 'math'

torch.manualSeed(123)
local size =6
local count = tonumber(arg[1]) or 10
count = count * 1000000

for n = 0,size do
    local length = 10^n
    local reps  = math.ceil( count / length )
    local t = torch.Tensor(length)
    local t1 = sys.clock()
    for i = 1,reps do
        torch.rand(t, length)
    end
    local t2 = sys.clock() - t1
    print("Average time rand torch.rand(<tensor>, " .. length ..") is: " .. t2/reps)
end

for n = 0,size do
    local length = 10^n
    local reps  = math.ceil( count / length )
    local t = torch.Tensor(length)
    local t1 = sys.clock()
    for i = 1,reps do
        torch.randn(t, length)
    end
    local t2 = sys.clock() - t1
    print("Average time rand torch.randn(<tensor>, " .. length ..") is: " .. t2/reps)
end

for n = 0,size do
    local length = 10^n
    local reps  = math.ceil( count / length )
    local t = torch.Tensor(length)
    local t1 = sys.clock()
    for i = 1,reps do
        t:uniform(0, 1)
    end
    local t2 = sys.clock() - t1
    print("Average time for tensor:uniform() on " .. length .." element tensor is: " .. t2/reps)
end

[andresy]

Thanks for the thorough analysis :) I am waiting for the valgrind step then!

[timharley]

valgrind is not showing any issues from this branch across torch.test() or nn.test().

[andresy]

Really great work, thanks a lot!