Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

segfault after some time of work #3297

Closed
oprudkyi opened this issue Jul 28, 2014 · 6 comments
Closed

segfault after some time of work #3297

oprudkyi opened this issue Jul 28, 2014 · 6 comments
Labels
crash needs more info no isolated repro

Comments

@oprudkyi
Copy link
Contributor 

./hphp/hhvm/hhvm --version
HipHop VM 3.3.0-dev (dbg)
Compiler: heads/master-0-g0e110b0f53d7b8903af61530cf6c5844c7a8f17d
Repo schema: bfeccaa74423bb93ef5b97cb64768a7fcd0bd1d0

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x2aab191ff940 (LWP 23145)]
0x00000000023a83f4 in std::_Hashtable<HPHP::StringData*, std::pair<HPHP::StringData* const, HPHP::Unit*>, std::allocator<std::pair<HPHP::StringData* const, HPHP::Unit*> >, std::__detail::_Select1st, HPHP::string_data_same, HPHP::string_data_hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_find_before_node (this=0x2aaab1ce7a30, __n=0, 
    __k=@0x2aab191fb558: 0x2aaaecc10700, __code=1629322656) at /root/hhvm/build/gcc49/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../include/c++/4.9.0/bits/hashtable.h:1388
1388    /root/hhvm/build/gcc49/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../include/c++/4.9.0/bits/hashtable.h: No such file or directory.
(gdb) bt
#0  0x00000000023a83f4 in std::_Hashtable<HPHP::StringData*, std::pair<HPHP::StringData* const, HPHP::Unit*>, std::allocator<std::pair<HPHP::StringData* const, HPHP::Unit*> >, std::__detail::_Select1st, HPHP::string_data_same, HPHP::string_data_hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_find_before_node (this=0x2aaab1ce7a30, __n=0, 
    __k=@0x2aab191fb558: 0x2aaaecc10700, __code=1629322656) at /root/hhvm/build/gcc49/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../include/c++/4.9.0/bits/hashtable.h:1388
#1  0x00000000023a7c3e in std::_Hashtable<HPHP::StringData*, std::pair<HPHP::StringData* const, HPHP::Unit*>, std::allocator<std::pair<HPHP::StringData* const, HPHP::Unit*> >, std::__detail::_Select1st, HPHP::string_data_same, HPHP::string_data_hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_find_node (this=0x2aaab1ce7a30, __bkt=0, 
    __key=@0x2aab191fb558: 0x2aaaecc10700, __c=1629322656) at /root/hhvm/build/gcc49/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../include/c++/4.9.0/bits/hashtable.h:590
#2  0x00000000023a79e2 in std::_Hashtable<HPHP::StringData*, std::pair<HPHP::StringData* const, HPHP::Unit*>, std::allocator<std::pair<HPHP::StringData* const, HPHP::Unit*> >, std::__detail::_Select1st, HPHP::string_data_same, HPHP::string_data_hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::find (this=0x2aaab1ce7a30, 
    __k=@0x2aab191fb558: 0x2aaaecc10700) at /root/hhvm/build/gcc49/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../include/c++/4.9.0/bits/hashtable.h:1238
#3  0x00000000023a7751 in std::unordered_map<HPHP::StringData*, HPHP::Unit*, HPHP::string_data_hash, HPHP::string_data_same, std::allocator<std::pair<HPHP::StringData* const, HPHP::Unit*> > >::find (this=0x2aaab1ce7a30, 
    __x=@0x2aab191fb558: 0x2aaaecc10700) at /root/hhvm/build/gcc49/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../include/c++/4.9.0/bits/unordered_map.h:574
#4  0x00000000023a4a0d in HPHP::lookupUnit (path=0x2aaaecc10680, currentDir=0x3dfa77b "", initial_opt=0x2aab191fb607) at /root/hhvm/hhvm/hphp/runtime/base/unit-cache.cpp:417
#5  0x0000000002232802 in HPHP::invoke_file_impl (res=..., path=..., once=true, currentDir=0x3dfa77b "") at /root/hhvm/hhvm/hphp/runtime/base/builtin-functions.cpp:736
#6  0x0000000002232901 in HPHP::invoke_file (s=..., once=true, currentDir=0x3dfa77b "") at /root/hhvm/hhvm/hphp/runtime/base/builtin-functions.cpp:752
#7  0x00000000022329d4 in HPHP::include_impl_invoke (file=..., once=true, currentDir=0x3dfa77b "") at /root/hhvm/hhvm/hphp/runtime/base/builtin-functions.cpp:763
#8  0x00000000022ff3bb in HPHP::hphp_invoke (context=0x2aaab1ce7800, cmd=..., func=false, funcParams=..., funcRet=..., reqInitFunc=..., reqInitDoc=..., error=@0x2aab191fb97f: false, errorMsg=..., once=true, warmupOnly=false, 
    richErrorMsg=false) at /root/hhvm/hhvm/hphp/runtime/base/program-functions.cpp:1736
#9  0x00000000024af75f in HPHP::HttpRequestHandler::executePHPRequest (this=0x2aaaf1214ac0, transport=0x2aaaecf91b10, reqURI=..., sourceRootInfo=..., cachableDynamicContent=false)
    at /root/hhvm/hhvm/hphp/runtime/server/http-request-handler.cpp:379
#10 0x00000000024aed17 in HPHP::HttpRequestHandler::handleRequest (this=0x2aaaf1214ac0, transport=0x2aaaecf91b10) at /root/hhvm/hhvm/hphp/runtime/server/http-request-handler.cpp:289
#11 0x0000000002518f19 in HPHP::ServerWorker<std::shared_ptr<HPHP::FastCGIJob>, HPHP::FastCGITransportTraits>::doJobImpl (this=0x2aaadb70d740, job=..., abort=false) at /root/hhvm/hhvm/hphp/runtime/server/server-worker.h:103
#12 0x000000000251883a in HPHP::ServerWorker<std::shared_ptr<HPHP::FastCGIJob>, HPHP::FastCGITransportTraits>::doJob (this=0x2aaadb70d740, job=...) at /root/hhvm/hhvm/hphp/runtime/server/server-worker.h:57
#13 0x00000000025105ce in HPHP::JobQueueWorker<std::shared_ptr<HPHP::FastCGIJob>, HPHP::Server*, true, false, HPHP::JobQueueDropVMStack>::start (this=0x2aaadb70d740) at /root/hhvm/hhvm/hphp/util/job-queue.h:466
#14 0x0000000002511a1d in HPHP::AsyncFunc<HPHP::ServerWorker<std::shared_ptr<HPHP::FastCGIJob>, HPHP::FastCGITransportTraits> >::run_ (obj=0x2aaab994d200) at /root/hhvm/hhvm/hphp/util/async-func.h:205
#15 0x00000000037633a7 in HPHP::AsyncFuncImpl::threadFuncImpl (this=0x2aaab994d200) at /root/hhvm/hhvm/hphp/util/async-func.cpp:131
#16 0x0000000003762f99 in HPHP::AsyncFuncImpl::ThreadFunc (obj=0x2aaab994d200) at /root/hhvm/hhvm/hphp/util/async-func.cpp:51
#17 0x00002aaaaf95f77d in start_thread () from /lib64/libpthread.so.0
#18 0x0000003c996d325d in clone () from /lib64/libc.so.6
#19 0x0000000000000000 in ?? ()
@LiraNuna LiraNuna added the crash label Jul 29, 2014
@LiraNuna
Copy link
Contributor

More information and/or a reproducible test case will be very much needed for us to fix this.

@oprudkyi
Copy link
Contributor Author

I can get it only at production with many requests
gdb is already running , what useful info I can get here ?
p.s. I'm not good with gdb :(

@oprudkyi
Copy link
Contributor Author

oprudkyi commented Aug 8, 2014

I've checked with fresher version
HipHop VM 3.3.0-dev (dbg)
Compiler: heads/master-0-g49a02efc3b34d7d15375f10bd21e00d4008d91c0
Repo schema: 7dca1766037ff4c62b362f3df1ef271a849a7451
Extension API: 20140702

and with gcc 4.8.3 and 4.9.1 and have got exactly same error

also, I found possible sequence to crash - run hhvm under huge load for some time , then remove load at all (switch apache to php) and restore load

@oprudkyi
Copy link
Contributor Author

oprudkyi commented Aug 8, 2014

also some additional data (with use of dgb helper)

0x000000000234d33b in std::__detail::_Equal_helper<HPHP::StringData*, std::pair<HPHP::StringData* const, HPHP::Unit*>, std::__detail::_Select1st, HPHP::string_data_same, unsigned long, true>::_S_equals (__eq=..., __extract=..., 
    __k=@0x2aab13dfb508: 0x2aab00411310, __c=1629322656, __n=0x0) at /opt/hhvm/lib/gcc/x86_64-unknown-linux-gnu/4.8.3/../../../../include/c++/4.8.3/bits/hashtable_policy.h:1166
1166        { return __c == __n->_M_hash_code && __eq(__k, __extract(__n->_M_v)); }
(gdb) bt
#0  0x000000000234d33b in std::__detail::_Equal_helper<HPHP::StringData*, std::pair<HPHP::StringData* const, HPHP::Unit*>, std::__detail::_Select1st, HPHP::string_data_same, unsigned long, true>::_S_equals (__eq=..., __extract=..., 
    __k=@0x2aab13dfb508: 0x2aab00411310, __c=1629322656, __n=0x0) at /opt/hhvm/lib/gcc/x86_64-unknown-linux-gnu/4.8.3/../../../../include/c++/4.8.3/bits/hashtable_policy.h:1166
#1  0x000000000234cf76 in std::__detail::_Hashtable_base<HPHP::StringData*, std::pair<HPHP::StringData* const, HPHP::Unit*>, std::__detail::_Select1st, HPHP::string_data_same, HPHP::string_data_hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Hashtable_traits<true, false, true> >::_M_equals (this=0x2aaabbe9da30, __k=@0x2aab13dfb508: 0x2aab00411310, __c=1629322656, __n=0x0)
    at /opt/hhvm/lib/gcc/x86_64-unknown-linux-gnu/4.8.3/../../../../include/c++/4.8.3/bits/hashtable_policy.h:1470
#2  0x000000000234c9f7 in std::_Hashtable<HPHP::StringData*, std::pair<HPHP::StringData* const, HPHP::Unit*>, std::allocator<std::pair<HPHP::StringData* const, HPHP::Unit*> >, std::__detail::_Select1st, HPHP::string_data_same, HPHP::string_data_hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_find_before_node (this=0x2aaabbe9da30, __n=0, 
    __k=@0x2aab13dfb508: 0x2aab00411310, __code=1629322656) at /opt/hhvm/lib/gcc/x86_64-unknown-linux-gnu/4.8.3/../../../../include/c++/4.8.3/bits/hashtable.h:1165
#3  0x000000000234c2e4 in std::_Hashtable<HPHP::StringData*, std::pair<HPHP::StringData* const, HPHP::Unit*>, std::allocator<std::pair<HPHP::StringData* const, HPHP::Unit*> >, std::__detail::_Select1st, HPHP::string_data_same, HPHP::string_data_hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_find_node (this=0x2aaabbe9da30, __bkt=0, 
    __key=@0x2aab13dfb508: 0x2aab00411310, __c=1629322656) at /opt/hhvm/lib/gcc/x86_64-unknown-linux-gnu/4.8.3/../../../../include/c++/4.8.3/bits/hashtable.h:604
#4  0x000000000234c082 in std::_Hashtable<HPHP::StringData*, std::pair<HPHP::StringData* const, HPHP::Unit*>, std::allocator<std::pair<HPHP::StringData* const, HPHP::Unit*> >, std::__detail::_Select1st, HPHP::string_data_same, HPHP::string_data_hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::find (this=0x2aaabbe9da30, 
    __k=@0x2aab13dfb508: 0x2aab00411310) at /opt/hhvm/lib/gcc/x86_64-unknown-linux-gnu/4.8.3/../../../../include/c++/4.8.3/bits/hashtable.h:1025
#5  0x000000000234be15 in std::unordered_map<HPHP::StringData*, HPHP::Unit*, HPHP::string_data_hash, HPHP::string_data_same, std::allocator<std::pair<HPHP::StringData* const, HPHP::Unit*> > >::find (this=0x2aaabbe9da30, 
    __x=@0x2aab13dfb508: 0x2aab00411310) at /opt/hhvm/lib/gcc/x86_64-unknown-linux-gnu/4.8.3/../../../../include/c++/4.8.3/bits/unordered_map.h:543
#6  0x00000000023490b4 in HPHP::lookupUnit (path=0x2aab00411290, currentDir=0x3df7383 "", initial_opt=0x2aab13dfb5c7) at /root/hhvm/hhvm/hphp/runtime/base/unit-cache.cpp:417
#7  0x00000000021de8f1 in HPHP::invoke_file_impl (res=..., path=..., once=true, currentDir=0x3df7383 "") at /root/hhvm/hhvm/hphp/runtime/base/builtin-functions.cpp:736
#8  0x00000000021de9f0 in HPHP::invoke_file (s=..., once=true, currentDir=0x3df7383 "") at /root/hhvm/hhvm/hphp/runtime/base/builtin-functions.cpp:752
#9  0x00000000021deac3 in HPHP::include_impl_invoke (file=..., once=true, currentDir=0x3df7383 "") at /root/hhvm/hhvm/hphp/runtime/base/builtin-functions.cpp:763
#10 0x00000000022a799a in HPHP::hphp_invoke (context=0x2aaabbe9d800, cmd="/home/t/application/t/index.php", func=false, funcParams=..., funcRet=..., reqInitFunc="", reqInitDoc="", 
    error=@0x2aab13dfb93f: false, errorMsg="Internal Server Error", once=true, warmupOnly=false, richErrorMsg=false) at /root/hhvm/hhvm/hphp/runtime/base/program-functions.cpp:1754
#11 0x000000000244e13e in HPHP::HttpRequestHandler::executePHPRequest (this=0x2aab006a0be0, transport=0x2aaad9f84e98, reqURI=..., sourceRootInfo=..., cachableDynamicContent=false)
    at /root/hhvm/hhvm/hphp/runtime/server/http-request-handler.cpp:381
#12 0x000000000244d6d1 in HPHP::HttpRequestHandler::handleRequest (this=0x2aab006a0be0, transport=0x2aaad9f84e98) at /root/hhvm/hhvm/hphp/runtime/server/http-request-handler.cpp:291
#13 0x00000000024b6417 in HPHP::ServerWorker<std::shared_ptr<HPHP::FastCGIJob>, HPHP::FastCGITransportTraits>::doJobImpl (this=0x2aab053ab300, job=std::shared_ptr (count 3, weak 0) 0x2aab24798b18, abort=false)
    at /root/hhvm/hhvm/hphp/runtime/server/server-worker.h:103
#14 0x00000000024b5d04 in HPHP::ServerWorker<std::shared_ptr<HPHP::FastCGIJob>, HPHP::FastCGITransportTraits>::doJob (this=0x2aab053ab300, job=std::shared_ptr (count 3, weak 0) 0x2aab24798b18)
    at /root/hhvm/hhvm/hphp/runtime/server/server-worker.h:57
#15 0x00000000024ae1a0 in HPHP::JobQueueWorker<std::shared_ptr<HPHP::FastCGIJob>, HPHP::Server*, true, false, HPHP::JobQueueDropVMStack>::start (this=0x2aab053ab300) at /root/hhvm/hhvm/hphp/util/job-queue.h:466
#16 0x00000000024af3d1 in HPHP::AsyncFunc<HPHP::ServerWorker<std::shared_ptr<HPHP::FastCGIJob>, HPHP::FastCGITransportTraits> >::run_ (obj=0x2aaae777bf00) at /root/hhvm/hhvm/hphp/util/async-func.h:205
#17 0x000000000375de8b in HPHP::AsyncFuncImpl::threadFuncImpl (this=0x2aaae777bf00) at /root/hhvm/hhvm/hphp/util/async-func.cpp:131
#18 0x000000000375da7d in HPHP::AsyncFuncImpl::ThreadFunc (obj=0x2aaae777bf00) at /root/hhvm/hhvm/hphp/util/async-func.cpp:51
#19 0x00002aaaaf98977d in start_thread () from /lib64/libpthread.so.0
#20 0x0000003c996d325d in clone () from /lib64/libc.so.6
#21 0x0000000000000000 in ?? ()

$1 = 1629322656
(gdb) print __n
$2 = (std::__detail::_Hash_node<std::pair<HPHP::StringData* const, HPHP::Unit*>, true> *) 0x0
(gdb) print __n->_M_hash_code
Cannot access memory at address 0x18

@fredemmott
Copy link
Contributor

I fixed a few issues that could cause crashes in the unit cache last month; these fixes should be in 3.4. Can you see if this fixed the crashes for you?

@paulbiss
Copy link
Contributor

Closing due to inactivity.

https://github.com/facebook/hhvm/wiki/Human-Timeouts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
crash needs more info no isolated repro
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@LiraNuna @fredemmott @paulbiss @oprudkyi