Added Memory Hooks #9

Closed
wants to merge 3 commits into
from

3 participants

@mikewied
couchbase member

No description provided.

Mike Wiederhold added some commits Feb 16, 2012
Mike Wiederhold Added tcmalloc/tcmalloc-minimal to build script
Change-Id: I1be0b10a5760813d5458175c54a0ea513a96fe34
37fd61b
Mike Wiederhold Added allocation hooks to engine hooks api
Change-Id: I7593d8e45ce063b45207b9c2ee0e0b7259eca2a5
843b53e
Mike Wiederhold Allocator hooks no longer cares who the allocator is
Change-Id: I4ee9ee6f5ff333edc9c32db6389eaaa1e1b09549
f34f19c
@mikewied
couchbase member

Please take a look at these when you guys have a chance. It might be a good idea to merge the second and third commits together, but I think it could go either way.

@mikewied mikewied closed this Feb 28, 2012
@dustin dustin was unassigned by mikewied Sep 18, 2014
@ns-codereview ns-codereview pushed a commit that referenced this pull request Jun 30, 2015
@daverigby daverigby Fix race in memcached_uptime access
Identified by ThreadSanitizer:

WARNING: ThreadSanitizer: data race (pid=34137)
  Write of size 4 at 0x7fa0ab35b288 by main thread:
    #0 mc_time_clock_tick() /home/daver/repos/couchbase/server/memcached/daemon/mc_time.cc:171 (memcached+0x0000000f9df3)
    #1 mc_time_clock_event_handler(int, short, void*) /home/daver/repos/couchbase/server/memcached/daemon/mc_time.cc:158 (memcached+0x0000000fa08f)
    #2 event_process_active_single_queue /home/couchbase/jenkins/workspace/cbdeps-build/label/ubuntu14.04/release/sherlock/deps/packages/build/libevent/libevent-prefix/src/libevent/event.c:1368 (libevent_core-2.0.so.5+0x00000000b7cb)
    #3 __libc_start_main /build/buildd/eglibc-2.19/csu/libc-start.c:287 (libc.so.6+0x000000021ec4)

  Previous read of size 4 at 0x7fa0ab35b288 by thread T12 (mutexes: write M45839, write M45891):
    #0 mc_time_get_current_time /home/daver/repos/couchbase/server/memcached/daemon/mc_time.cc:94 (memcached+0x0000000fa0b6)
    #1 do_item_update /home/daver/repos/couchbase/server/memcached/engines/default_engine/items.c:358 (default_engine.so+0x000000008109)
    #2 item_get /home/daver/repos/couchbase/server/memcached/engines/default_engine/items.c:821 (default_engine.so+0x000000007deb)
    #3 default_get /home/daver/repos/couchbase/server/memcached/engines/default_engine/default_engine.c:335 (default_engine.so+0x000000003ef5)
    #4 EWB_Engine::get(engine_interface*, void const*, void**, void const*, int, unsigned short) /home/daver/repos/couchbase/server/memcached/engines/ewouldblock_engine/ewouldblock_engine.cc:247 (ewouldblock_engine.so+0x000000004fc1)
    #5 bucket_get /home/daver/repos/couchbase/server/memcached/daemon/memcached.cc:123 (memcached+0x0000000e096a)
    #6 process_bin_packet(conn*) /home/daver/repos/couchbase/server/memcached/daemon/memcached.cc:5051 (memcached+0x0000000d2100)
    #7 run_event_loop /home/daver/repos/couchbase/server/memcached/daemon/connections.cc:174 (memcached+0x0000000c70c3)
    #8 thread_libevent_process(int, short, void*) /home/daver/repos/couchbase/server/memcached/daemon/thread.cc:374 (memcached+0x0000000f8fc0)
    #9 event_persist_closure /home/couchbase/jenkins/workspace/cbdeps-build/label/ubuntu14.04/release/sherlock/deps/packages/build/libevent/libevent-prefix/src/libevent/event.c:1319 (libevent_core-2.0.so.5+0x00000000b6c7)
    #10 platform_thread_wrap /home/daver/repos/couchbase/server/platform/src/cb_pthreads.c:23 (libplatform.so.0.1.0+0x000000003730)

Change-Id: I9d73a6f5553d88beb1bcc2526763d892fe1b990f
Reviewed-on: http://review.couchbase.org/52484
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
8a92da8
@ns-codereview ns-codereview pushed a commit that referenced this pull request Jun 30, 2015
@daverigby daverigby ewouldblock_engine: Fix data race on cookie_map
As identified by ThreadSanitizer:

WARNING: ThreadSanitizer: data race (pid=950)
  Write of size 8 at 0x7d100002db60 by thread T17 (mutexes: write M45854):
    #0 operator delete(void*) <null>:0 (memcached+0x00000005a6bb)
    #1 __gnu_cxx::new_allocator<...>::deallocate(...) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/ext/new_allocator.h:110 (ewouldblock_engine.so+0x000000007677)
    #2 std::_Rb_tree<>::erase(...) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/bits/stl_tree.h:890 (ewouldblock_engine.so+0x000000005890)
    #3 bucket_unknown_command /home/daver/repos/couchbase/server/memcached/daemon/memcached.cc:130 (memcached+0x0000000e8cd0)
    #4 process_bin_unknown_packet(conn*) /home/daver/repos/couchbase/server/memcached/daemon/memcached.cc:1877 (memcached+0x0000000d2223)
    #5 run_event_loop /home/daver/repos/couchbase/server/memcached/daemon/connections.cc:174 (memcached+0x0000000c70c3)
    #6 event_handler /home/daver/repos/couchbase/server/memcached/daemon/memcached.cc:6908 (memcached+0x0000000d7b24)
    #7 event_persist_closure /home/couchbase/jenkins/workspace/cbdeps-build/label/ubuntu14.04/release/sherlock/deps/packages/build/libevent/libevent-prefix/src/libevent/event.c:1319 (libevent_core-2.0.so.5+0x00000000b6c7)
    #8 platform_thread_wrap /home/daver/repos/couchbase/server/platform/src/cb_pthreads.c:23 (libplatform.so.0.1.0+0x000000003730)

  Previous read of size 8 at 0x7d100002db60 by thread T16 (mutexes: write M45851, write M45962):
    #0 std::less<void const*>::operator()(void const* const&, void const* const&) const /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/bits/stl_function.h:371 (ewouldblock_engine.so+0x000000006ac8)
    #1 EWB_Engine::should_inject_error(...) /home/daver/repos/couchbase/server/memcached/engines/ewouldblock_engine/ewouldblock_engine.cc:110 (ewouldblock_engine.so+0x000000006c31)
    #2 EWB_Engine::allocate(...) /home/daver/repos/couchbase/server/memcached/engines/ewouldblock_engine/ewouldblock_engine.cc:211 (ewouldblock_engine.so+0x000000004d33)
    #3 add_set_replace_executor(conn*, void*, ENGINE_STORE_OPERATION) /home/daver/repos/couchbase/server/memcached/daemon/memcached.cc:3940 (memcached+0x0000000e7b40)
    #4 setq_executor(conn*, void*) /home/daver/repos/couchbase/server/memcached/daemon/memcached.cc:4079 (memcached+0x0000000e0496)
    #5 process_bin_packet(conn*) /home/daver/repos/couchbase/server/memcached/daemon/memcached.cc:5051 (memcached+0x0000000d2100)
    #6 run_event_loop /home/daver/repos/couchbase/server/memcached/daemon/connections.cc:174 (memcached+0x0000000c70c3)
    #7 event_handler /home/daver/repos/couchbase/server/memcached/daemon/memcached.cc:6908 (memcached+0x0000000d7b24)
    #8 event_persist_closure /home/couchbase/jenkins/workspace/cbdeps-build/label/ubuntu14.04/release/sherlock/deps/packages/build/libevent/libevent-prefix/src/libevent/event.c:1319 (libevent_core-2.0.so.5+0x00000000b6c7)
    #9 platform_thread_wrap /home/daver/repos/couchbase/server/platform/src/cb_pthreads.c:23 (libplatform.so.0.1.0+0x000000003730)

Change-Id: Ia558b1947f2c913d6fa7943de0e85ba975a0c2b1
Reviewed-on: http://review.couchbase.org/52486
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
0ca2c64
@daverigby daverigby added a commit to daverigby/memcached that referenced this pull request Apr 13, 2016
@daverigby daverigby WIP: Initial index_engine work
To build:

    cd build/memcached
    make -j8

To run:

    ./memcached_testapp --gtest_filter=*HelloWorld* -e

Currently crashes when trying to perform a allocate:

(lldb) bt
* thread #4: tid = 0x8eca30, 0x00007fff977b3002 libsystem_kernel.dylib`__pthread_kill + 10, name = 'mc:worker_0, stop reason = signal SIGABRT
  * frame #0: 0x00007fff977b3002 libsystem_kernel.dylib` __pthread_kill  + 10
    frame #1: 0x00007fff969f05c5 libsystem_pthread.dylib` pthread_kill  + 90
    frame #2: 0x00007fff9a9136e7 libsystem_c.dylib` abort  + 129
    frame #3: 0x00000001008ecb58 index_engine.dylib` Index_Engine::allocate(handle=0x000000010316c200, cookie=0x0000000101afd4b0, item=0x0000700000182018, key=0x0000000101ba8820, nkey=1, nbytes=1, flags=0, exptime=0, datatype='\0')  + 72 at index_engine.cc:86
    frame #4: 0x00000001001ecb7d memcached_testapp` add_set_replace_executor(c=0x0000000101afd180, packet=0x0000000101ba8800, store_op=OPERATION_SET)  + 1085 at mcbp_executors.cc:2765
    frame #5: 0x00000001001e195f memcached_testapp` set_executor(c=0x0000000101afd180, packet=0x0000000101ba8800)  + 47 at mcbp_executors.cc:2903
    frame #6: 0x00000001001eb01b memcached_testapp` process_bin_packet(c=0x0000000101afd180)  + 1035 at mcbp_executors.cc:4619
    frame #7: 0x00000001001eabff memcached_testapp` mcbp_complete_nread(c=0x0000000101afd180)  + 223 at mcbp_executors.cc:4718
    frame #8: 0x0000000100235c6e memcached_testapp` conn_nread(c=0x0000000101afd180)  + 62 at statemachine_mcbp.cc:310
    frame #9: 0x00000001001ba62d memcached_testapp` McbpStateMachine::execute(this=0x0000000101a23b10, connection=0x0000000101afd180)  + 29 at statemachine_mcbp.h:43
    frame #10: 0x00000001001b749c memcached_testapp` McbpConnection::runStateMachinery(this=0x0000000101afd180)  + 636 at connection_mcbp.cc:1007
    frame #11: 0x00000001001b8e40 memcached_testapp` McbpConnection::runEventLoop(this=0x0000000101afd180, which=2)  + 64 at connection_mcbp.cc:1272
    frame #12: 0x00000001001be9f4 memcached_testapp` ::run_event_loop(c=0x0000000101afd180, which=2)  + 36 at connections.cc:153
    frame #13: 0x0000000100209ab7 memcached_testapp` event_handler(fd=43, which=2, arg=0x0000000101afd180)  + 1911 at memcached.cc:874
    frame #14: 0x0000000100771e01 libevent_core-2.0.5.dylib` event_base_loop  + 2385
    frame #15: 0x000000010024158c memcached_testapp` worker_libevent(arg=0x0000000101b6ec00)  + 92 at thread.cc:231
    frame #16: 0x00000001008846e9 libplatform.0.1.0.dylib` CouchbaseThread::run(this=0x0000000101a1ccd0)  + 521 at cb_pthreads.cc:54
    frame #17: 0x0000000100882fec libplatform.0.1.0.dylib` platform_thread_wrap(arg=0x0000000101a1ccd0)  + 220 at cb_pthreads.cc:66
    frame #18: 0x00007fff969eec13 libsystem_pthread.dylib` _pthread_body  + 131
    frame #19: 0x00007fff969eeb90 libsystem_pthread.dylib` _pthread_start  + 168
    frame #20: 0x00007fff969ec375 libsystem_pthread.dylib` thread_start  + 13

Change-Id: I9161e578d8a007808c176d99f92c49b8e50a2402
12cb35a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment