Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

when use GTX1070 and beta23, report error: CUDA_ERROR_ILLEGAL_ADDRESS #55

Open
P-Chao opened this issue May 11, 2017 · 0 comments
Open

Comments

@P-Chao
Copy link

P-Chao commented May 11, 2017

I can sucesseful run CPU version, and when use gpu cudnn5.1 win10 , another matconvnet program runs well, but when I run FCN training with 'opts.train.gpus = [1]', the error code:

I use matlab2017a, matconvnet1.0beta23, cudnn5.1, windows10, thank you very much.

More about the debug info,
when the first time enter vl_nnconv(), the param_size 1x1x4096x21 gpuarray, runs well.
when seconed time enter vl_nnconv(), the param_size 1x1x4090x4096 gpuarray, runs failed, report error:

`
cnn_train_dag: resetting GPU

ans =

CUDADevice with properties:

                  Name: 'GeForce GTX 1070'
                 Index: 1
     ComputeCapability: '6.1'
        SupportsDouble: 1
         DriverVersion: 8
        ToolkitVersion: 8
    MaxThreadsPerBlock: 1024
      MaxShmemPerBlock: 49152
    MaxThreadBlockSize: [1024 1024 64]
           MaxGridSize: [2.1475e+09 65535 65535]
             SIMDWidth: 32
           TotalMemory: 8.5899e+09
       AvailableMemory: 7.0877e+09
   MultiprocessorCount: 15
          ClockRateKHz: 1860000
           ComputeMode: 'Default'
  GPUOverlapsTransfers: 1
KernelExecutionTimeout: 1
      CanMapHostMemory: 1
       DeviceSupported: 1
        DeviceSelected: 1

train: epoch 01: 1/565:Error using vl_nnconv
An error occurred during PTX compilation of .
The information log was:

The error log was:

The CUDA error code was: CUDA_ERROR_ILLEGAL_ADDRESS.

Error in dagnn.Conv/backward (line 21)
[derInputs{1}, derParams{1}, derParams{2}] = vl_nnconv(...

Error in dagnn.Layer/backwardAdvanced (line 120)
[derInputs, derParams] = obj.backward ...

Error in dagnn.DagNN/eval (line 117)
obj.layers(l).block.backwardAdvanced(obj.layers(l)) ;

Error in cnn_train_dag>processEpoch (line 217)
net.eval(inputs, params.derOutputs, 'holdOn', s < params.numSubBatches) ;

Error in cnn_train_dag (line 87)
[net, state] = processEpoch(net, state, params, 'train') ;

Error in fcnTrain (line 98)
info = cnn_train_dag(net, imdb, getBatchWrapper(bopts), ...
`

@P-Chao P-Chao changed the title use GTX1070 and beta23, but report error: CUDA_ERROR_ILLEGAL_ADDRESS when use GTX1070 and beta23, report error: CUDA_ERROR_ILLEGAL_ADDRESS May 11, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant