Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

I'm training a model now, but why this error happened? #31

Closed
DuanHuiyu opened this issue Jan 30, 2018 · 1 comment
Closed

I'm training a model now, but why this error happened? #31

DuanHuiyu opened this issue Jan 30, 2018 · 1 comment

Comments

@DuanHuiyu
Copy link

Traceback (most recent call last):
File "/media/huiyu/Project/Python/other/saliency-salgan-2017/scripts/my_train.py", line 170, in
train()
File "/media/huiyu/Project/Python/other/saliency-salgan-2017/scripts/my_train.py", line 153, in train
model = ModelSALGAN(INPUT_SIZE[0], INPUT_SIZE[1])
File "/media/huiyu/Project/Python/other/saliency-salgan-2017/scripts/models/model_salgan.py", line 28, in init
self.predictFunction = theano.function([self.input_var], test_prediction)
File "/home/huiyu/miniconda2/lib/python2.7/site-packages/theano/compile/function.py", line 320, in function
output_keys=output_keys)
File "/home/huiyu/miniconda2/lib/python2.7/site-packages/theano/compile/pfunc.py", line 479, in pfunc
output_keys=output_keys)
File "/home/huiyu/miniconda2/lib/python2.7/site-packages/theano/compile/function_module.py", line 1777, in orig_function
defaults)
File "/home/huiyu/miniconda2/lib/python2.7/site-packages/theano/compile/function_module.py", line 1641, in create
input_storage=input_storage_lists, storage_map=storage_map)
File "/home/huiyu/miniconda2/lib/python2.7/site-packages/theano/gof/link.py", line 690, in make_thunk
storage_map=storage_map)[:3]
File "/home/huiyu/miniconda2/lib/python2.7/site-packages/theano/gof/vm.py", line 1003, in make_all
no_recycling))
File "/home/huiyu/miniconda2/lib/python2.7/site-packages/theano/sandbox/cuda/init.py", line 256, in make_thunk
compute_map, no_recycling)
File "/home/huiyu/miniconda2/lib/python2.7/site-packages/theano/gof/op.py", line 970, in make_thunk
no_recycling)
File "/home/huiyu/miniconda2/lib/python2.7/site-packages/theano/gof/op.py", line 879, in make_c_thunk
output_storage=node_output_storage)
File "/home/huiyu/miniconda2/lib/python2.7/site-packages/theano/gof/cc.py", line 1200, in make_thunk
keep_lock=keep_lock)
File "/home/huiyu/miniconda2/lib/python2.7/site-packages/theano/gof/cc.py", line 1143, in compile
keep_lock=keep_lock)
File "/home/huiyu/miniconda2/lib/python2.7/site-packages/theano/gof/cc.py", line 1595, in cthunk_factory
key=key, lnk=self, keep_lock=keep_lock)
File "/home/huiyu/miniconda2/lib/python2.7/site-packages/theano/gof/cmodule.py", line 1142, in module_from_key
module = lnk.compile_cmodule(location)
File "/home/huiyu/miniconda2/lib/python2.7/site-packages/theano/gof/cc.py", line 1506, in compile_cmodule
preargs=preargs)
File "/home/huiyu/miniconda2/lib/python2.7/site-packages/theano/sandbox/cuda/nvcc_compiler.py", line 399, in compile_str
'for cmd', ' '.join(cmd))
Exception: ('The following error happened while compiling the node', GpuDnnConv{algo='small', inplace=True}(GpuContiguous.0, GpuContiguous.0, GpuAllocEmpty.0, GpuDnnConvDesc{border_mode=(1, 1),subsample=(1, 1), conv_mode='cross', precision='float32'}.0, Constant{1.0}, Constant{0.0}), '\n', 'nvcc return status', 2, 'for cmd', 'nvcc -shared -O3 -Xlinker -rpath,/usr/local/cuda-8.0/lib64-arch=sm_61 -m64 -Xcompiler -fno-math-errno,-Wno-unused-label,-Wno-unused-variable,-Wno-write-strings,-DCUDA_NDARRAY_CUH=c72d035fdf91890f3b36710688069b2e,-DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION,-fPIC,-fvisibility=hidden -Xlinker -rpath,/home/huiyu/.theano/compiledir_Linux-4.4--generic-x86_64-with-debian-stretch-sid-x86_64-2.7.14-64/cuda_ndarray -I/home/huiyu/.theano/compiledir_Linux-4.4--generic-x86_64-with-debian-stretch-sid-x86_64-2.7.14-64/cuda_ndarray -I/usr/local/cuda-8.0/include -I/home/huiyu/miniconda2/lib/python2.7/site-packages/theano/sandbox/cuda -I/home/huiyu/miniconda2/lib/python2.7/site-packages/numpy/core/include -I/home/huiyu/miniconda2/include/python2.7 -I/home/huiyu/miniconda2/lib/python2.7/site-packages/theano/gof -o /home/huiyu/.theano/compiledir_Linux-4.4--generic-x86_64-with-debian-stretch-sid-x86_64-2.7.14-64/tmpxFjabo/b1b28962a86b2752ba91f7d762cc71c7.so mod.cu -L/home/huiyu/.theano/compiledir_Linux-4.4--generic-x86_64-with-debian-stretch-sid-x86_64-2.7.14-64/cuda_ndarray -L/home/huiyu/miniconda2/lib -lcudart -lcublas -lcuda_ndarray -lcudnn -lpython2.7', "[GpuDnnConv{algo='small', inplace=True}(<CudaNdarrayType(float32,4D)>, <CudaNdarrayType(float32, 4D)>, <CudaNdarrayType(float32, 4D)>, <CDataType{cudnnConvolutionDescriptor_t}>, Constant{1.0}, Constant{0.0})]")

@DuanHuiyu DuanHuiyu changed the title I'm trainning a model now, but why this error happened? I'm training a model now, but why this error happened? Jan 30, 2018
@DuanHuiyu
Copy link
Author

THEANO_FLAGS=mode=FAST_RUN,device=cuda0,floatX=float32,lib.cnmem=1,optimizer_including=cudnn

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant