Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

marian decode error #283

Closed
520jefferson opened this issue Aug 22, 2018 · 2 comments
Closed

marian decode error #283

520jefferson opened this issue Aug 22, 2018 · 2 comments

Comments

@520jefferson
Copy link

@emjotde When i use 3 gpus to decode but i get this:

[New Thread 0x7fff92cc1700 (LWP 20374)]
[Thread 0x7fff5c6e4700 (LWP 20371) exited]
[Thread 0x7fff767fc700 (LWP 20373) exited]
[Thread 0x7fff777fe700 (LWP 20372) exited]
[Thread 0x7fff92cc1700 (LWP 20374) exited]
[New Thread 0x7fff92cc1700 (LWP 20376)]
[New Thread 0x7fff777fe700 (LWP 20375)]
[Thread 0x7fff92cc1700 (LWP 20376) exited]
[Thread 0x7fff777fe700 (LWP 20375) exited]
[2018-08-22 19:12:16] [memory] Reserving 305 MB, device gpu7
[2018-08-22 19:12:16] [memory] Reserving 305 MB, device gpu8
[2018-08-22 19:12:16] Error: invalid device ordinal - marian/src/marian/src/tensors/gpu/algorithm.cu:13
[Switching to Thread 0x7fff77fff700 (LWP 20335)]
0x00007fffd1e6f428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
54 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 0x00007fffd1e6f428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#1 0x00007fffd1e7102a in __GI_abort () at abort.c:89
#2 0x00000000007937f6 in gpuAssert(cudaError, char const*, int, bool) [clone .constprop.339] ()
#3 0x0000000000795404 in void marian::gpu::copy(std::shared_ptrmarian::Backend, float const*, float const*, float*) ()
#4 0x0000000000618d6c in void marian::TensorBase::set(float const*, float const*) ()
#5 0x0000000000616ee9 in std::_Function_handler<void (std::shared_ptrmarian::TensorBase), marian::inits::from_item(marian::io::Item const&)::{lambda(std::shared_ptrmarian::TensorBase)#2}>::_M_invoke(std::_Any_data const&, std::shared_ptrmarian::TensorBase&&) ()
#6 0x0000000000611702 in marian::ParamNode::init() ()
#7 0x00000000004a8a4c in marian::ExpressionGraph::forwardNext() ()
#8 0x00000000004b754c in marian::BeamSearch::search(std::shared_ptrmarian::ExpressionGraph, std::shared_ptrmarian::data::CorpusBatch) ()
#9 0x00000000004bac87 in marian::TranslateServicemarian::BeamSearch::run(std::vector<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > const&)::{lambda(unsigned long)#1}::operator()(unsigned long) const ()
#10 0x00000000004bb871 in std::future<std::result_of<marian::TranslateServicemarian::BeamSearch::run(std::vector<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > const&)::{lambda(unsigned long)#1}& (unsigned long&)>::type> marian::ThreadPool::enqueue<marian::TranslateServicemarian::BeamSearch::run(std::vector<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > const&)::{lambda(unsigned long)#1}&, unsigned long&>(std::result_of&&, (marian::TranslateServicemarian::BeamSearch::run(std::vector<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > const&)::{lambda(unsigned long)#1}&)...)::{lambda()#1}::operator()() const ()
#11 0x00000000004bbb4c in std::_Function_handler<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> (), std::__future_base::_Task_setter<std::unique_ptr<std::__future_base::_Result, std::__future_base::_Result_base::_Deleter>, std::_Bind_simple<std::reference_wrapper<std::future<std::result_of<marian::TranslateServicemarian::BeamSearch::run(std::vector<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > const&)::{lambda(unsigned long)#1}& (unsigned long&)>::type> marian::ThreadPool::enqueue<marian::TranslateServicemarian::BeamSearch::run(std::vector<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > const&)::{lambda(unsigned long)#1}&, unsigned long&>(std::result_of&&, (marian::TranslateServicemarian::BeamSearch::run(std::vector<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > const&)::{lambda(unsigned long)#1}&)...)::{lambda()#1}> ()>, void> >::_M_invoke(std::_Any_data const&) ()
#12 0x0000000000450bf9 in std::__future_base::_State_baseV2::_M_do_set(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>, bool) ()
#13 0x00007ffff637ba99 in __pthread_once_slow (once_control=0x7fff240166e8,
init_routine=0x7fffd27da2a0 <__once_proxy>) at pthread_once.c:116
---Type to continue, or q to quit---
#14 0x00000000004519b2 in std::_Function_handler<void (), std::future<std::result_of<marian::TranslateServicemarian::BeamSearch::run(std::vector<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > const&)::{lambda(unsigned long)#1}& (unsigned long&)>::type> marian::ThreadPool::enqueue<marian::TranslateServicemarian::BeamSearch::run(std::vector<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > const&)::{lambda(unsigned long)#1}&, unsigned long&>(std::result_of&&, (marian::TranslateServicemarian::BeamSearch::run(std::vector<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > const&)::{lambda(unsigned long)#1}&)...)::{lambda()#3}>::_M_invoke(std::_Any_data const&) ()
#15 0x000000000046b8d4 in std::thread::_Impl<std::_Bind_simple<marian::ThreadPool::ThreadPool(unsigned long, unsigned long)::{lambda()#1} ()> >::_M_run() ()
#16 0x00007fffd27db260 in ?? () from /opt/anaconda2/lib/libstdc++.so.6
#17 0x00007ffff63746ba in start_thread (arg=0x7fff77fff700) at pthread_create.c:333
#18 0x00007fffd1f4141d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
(gdb)

@520jefferson
Copy link
Author

@emjotde the gpu number i gived but it doesn't exist,it will case this error in running. But no case error in the beginning.

@emjotde
Copy link
Member

emjotde commented Aug 22, 2018

It does say Error: invalid device ordinal

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants