Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

caffe crashed #2719

Closed
cysin opened this issue Jul 8, 2015 · 1 comment
Closed

caffe crashed #2719

cysin opened this issue Jul 8, 2015 · 1 comment

Comments

@cysin
Copy link

cysin commented Jul 8, 2015

I have a server running caffe and it had chances to crash with following message. I have disabled the log output so does this matter?
#0 0x00007f1869576989 in raise () from /lib64/libc.so.6
#1 0x00007f1869578098 in abort () from /lib64/libc.so.6
#2 0x00007f18567807d9 in google::logging_fail() () from ../libs/libglog.so.0
#3 0x00007f1856781e6d in google::LogMessage::Fail() () from ../libs/libglog.so.0
#4 0x00007f1856783ced in google::LogMessage::SendToLog() () from ../libs/libglog.so.0
#5 0x00007f1856781a5c in google::LogMessage::Flush() () from ../libs/libglog.so.0
#6 0x00007f185678463e in google::LogMessageFatal::~LogMessageFatal() () from ../libs/libglog.so.0
#7 0x00007f1856bba287 in void caffe::caffe_gpu_gemm(CBLAS_TRANSPOSE, CBLAS_TRANSPOSE, int, int, int, float, float const_, float const_, float, float*) () from ../lib/libcaffe.so
#8 0x00007f1856bd6d13 in caffe::InnerProductLayer::Forward_gpu(std::vectorcaffe::Blob<float_, std::allocatorcaffe::Blob<float_> > const&, std::vectorcaffe::Blob<float_, std::allocatorcaffe::Blob<float_> > const&) () from ../lib/libcaffe.so
#9 0x00007f1856b989ee in caffe::Net::ForwardFromTo(int, int) () from ../lib/libcaffe.so
#10 0x00007f1856b98ed7 in caffe::Net::ForwardPrefilled(float*) () from ../lib/libcaffe.so

@longjon
Copy link
Contributor

longjon commented Jul 13, 2015

What was the exception/error message? Can you provide steps to reproduce this with current master in DEBUG mode? What GPU were you running on? (See https://github.com/BVLC/caffe/wiki/Reporting-Bugs-and-Other-Issues.) This looks like a CUDA error, which could mean, e.g., that the GPU was out of memory; is that possible?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants