Limit on input Tensor size using cunn

[Tushar-N]

Code to reproduce the error:

require 'cunn'
require 'cutorch'
cutorch.setDevice(1)
model=nn.Sequential():add(nn.Linear(300, 500)):add(nn.LogSoftMax()):cuda()
batch_size=90000
output=model:forward(torch.rand(batch_size,300):float():cuda())

The error:

/home/tushar/torch/install/share/lua/5.1/nn/Sequential.lua:44: invalid argument at /tmp/luarocks_cunn-scm-1-144/cunn/LogSoftMax.cu:249
stack traceback:
    [C]: in function 'updateOutput'
    /home/tushar/torch/install/share/lua/5.1/nn/Sequential.lua:44: in function 'forward'
    [string "_RESULT={model:forward(torch.rand(batch_size,..."]:1: in main chunk
    [C]: in function 'xpcall'
    /home/tushar/torch/install/share/lua/5.1/trepl/init.lua:630: in function 'repl'
    ...shar/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:185: in main chunk
    [C]: at 0x00406670  
                                                                      [0.2761s]

Large batch sizes work fine on CPU. Smaller batch sizes (<80k) work fine on GPU.

smth chntla on the issue:

It must be that the cuda launch parameters (number of blocks/threads) were configured without such large batch sizes in mind

[etrulls]

I'm also running into this.
(I'm trying to hunt down the magic number but I'm new to CUDA and it's not very friendly.)

[dominikgrewe]

The grid used to launch the kernel uses the batch size as the first dimension: https://github.com/torch/cunn/blob/master/LogSoftMax.cu#L238

There's a limit on the size of these grids, which depends on the version of CUDA you're using. It's 65536 for compute capability <= 2.x and 2^31-1 for newer versions (see https://en.wikipedia.org/wiki/CUDA#Version_features_and_specifications)

[etrulls]

Aha, I got that far, and I couldn't see why it wasn't working out of the box on a K40.
Turns out cunn is compiling for a compute capability of 2.0, so changing that seems to solve the problem. Thanks!

[hogwild]

I use a Geforce 750Ti. The problem is solved by changing the CUDA_NVCC_FLAGS from "-arch=sm_20" to "-arch=sm_50" in the CmakeLists.txt.

[mrharicot]

I have the same issue with SpatialSoftmax on the GPU, I am using a Titan X and cunn was compiled with CUDA 7.5 and compute capability 5.2
Any idea if this is fixable?

EDIT: I think I have fixed this by explicitly using cudnn.SpatialSoftMax