Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added Memory Hooks #9

Closed
wants to merge 3 commits into from
Closed

Added Memory Hooks #9

wants to merge 3 commits into from

Conversation

mikewied
Copy link
Contributor

No description provided.

Mike Wiederhold added 3 commits February 21, 2012 17:54
Added tcmalloc/tcmalloc-minimal to build script
37fd61b 
Change-Id: I1be0b10a5760813d5458175c54a0ea513a96fe34
Added allocation hooks to engine hooks api
843b53e 
Change-Id: I7593d8e45ce063b45207b9c2ee0e0b7259eca2a5
Allocator hooks no longer cares who the allocator is
f34f19c 
Change-Id: I4ee9ee6f5ff333edc9c32db6389eaaa1e1b09549
@mikewied
Copy link
Contributor Author

Please take a look at these when you guys have a chance. It might be a good idea to merge the second and third commits together, but I think it could go either way.

@ghost ghost assigned chiyoung and dustin Feb 28, 2012
@mikewied mikewied closed this Feb 28, 2012
ns-codereview pushed a commit that referenced this pull request Jun 30, 2015
@daverigby @trondn
Fix race in memcached_uptime access
8a92da8 
Identified by ThreadSanitizer:

WARNING: ThreadSanitizer: data race (pid=34137)
  Write of size 4 at 0x7fa0ab35b288 by main thread:
    #0 mc_time_clock_tick() /home/daver/repos/couchbase/server/memcached/daemon/mc_time.cc:171 (memcached+0x0000000f9df3)
    #1 mc_time_clock_event_handler(int, short, void*) /home/daver/repos/couchbase/server/memcached/daemon/mc_time.cc:158 (memcached+0x0000000fa08f)
    #2 event_process_active_single_queue /home/couchbase/jenkins/workspace/cbdeps-build/label/ubuntu14.04/release/sherlock/deps/packages/build/libevent/libevent-prefix/src/libevent/event.c:1368 (libevent_core-2.0.so.5+0x00000000b7cb)
    #3 __libc_start_main /build/buildd/eglibc-2.19/csu/libc-start.c:287 (libc.so.6+0x000000021ec4)

  Previous read of size 4 at 0x7fa0ab35b288 by thread T12 (mutexes: write M45839, write M45891):
    #0 mc_time_get_current_time /home/daver/repos/couchbase/server/memcached/daemon/mc_time.cc:94 (memcached+0x0000000fa0b6)
    #1 do_item_update /home/daver/repos/couchbase/server/memcached/engines/default_engine/items.c:358 (default_engine.so+0x000000008109)
    #2 item_get /home/daver/repos/couchbase/server/memcached/engines/default_engine/items.c:821 (default_engine.so+0x000000007deb)
    #3 default_get /home/daver/repos/couchbase/server/memcached/engines/default_engine/default_engine.c:335 (default_engine.so+0x000000003ef5)
    #4 EWB_Engine::get(engine_interface*, void const*, void**, void const*, int, unsigned short) /home/daver/repos/couchbase/server/memcached/engines/ewouldblock_engine/ewouldblock_engine.cc:247 (ewouldblock_engine.so+0x000000004fc1)
    #5 bucket_get /home/daver/repos/couchbase/server/memcached/daemon/memcached.cc:123 (memcached+0x0000000e096a)
    #6 process_bin_packet(conn*) /home/daver/repos/couchbase/server/memcached/daemon/memcached.cc:5051 (memcached+0x0000000d2100)
    #7 run_event_loop /home/daver/repos/couchbase/server/memcached/daemon/connections.cc:174 (memcached+0x0000000c70c3)
    #8 thread_libevent_process(int, short, void*) /home/daver/repos/couchbase/server/memcached/daemon/thread.cc:374 (memcached+0x0000000f8fc0)
    #9 event_persist_closure /home/couchbase/jenkins/workspace/cbdeps-build/label/ubuntu14.04/release/sherlock/deps/packages/build/libevent/libevent-prefix/src/libevent/event.c:1319 (libevent_core-2.0.so.5+0x00000000b6c7)
    #10 platform_thread_wrap /home/daver/repos/couchbase/server/platform/src/cb_pthreads.c:23 (libplatform.so.0.1.0+0x000000003730)

Change-Id: I9d73a6f5553d88beb1bcc2526763d892fe1b990f
Reviewed-on: http://review.couchbase.org/52484
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
ns-codereview pushed a commit that referenced this pull request Jun 30, 2015
@daverigby @trondn
ewouldblock_engine: Fix data race on cookie_map
0ca2c64 
As identified by ThreadSanitizer:

WARNING: ThreadSanitizer: data race (pid=950)
  Write of size 8 at 0x7d100002db60 by thread T17 (mutexes: write M45854):
    #0 operator delete(void*) <null>:0 (memcached+0x00000005a6bb)
    #1 __gnu_cxx::new_allocator<...>::deallocate(...) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/ext/new_allocator.h:110 (ewouldblock_engine.so+0x000000007677)
    #2 std::_Rb_tree<>::erase(...) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/bits/stl_tree.h:890 (ewouldblock_engine.so+0x000000005890)
    #3 bucket_unknown_command /home/daver/repos/couchbase/server/memcached/daemon/memcached.cc:130 (memcached+0x0000000e8cd0)
    #4 process_bin_unknown_packet(conn*) /home/daver/repos/couchbase/server/memcached/daemon/memcached.cc:1877 (memcached+0x0000000d2223)
    #5 run_event_loop /home/daver/repos/couchbase/server/memcached/daemon/connections.cc:174 (memcached+0x0000000c70c3)
    #6 event_handler /home/daver/repos/couchbase/server/memcached/daemon/memcached.cc:6908 (memcached+0x0000000d7b24)
    #7 event_persist_closure /home/couchbase/jenkins/workspace/cbdeps-build/label/ubuntu14.04/release/sherlock/deps/packages/build/libevent/libevent-prefix/src/libevent/event.c:1319 (libevent_core-2.0.so.5+0x00000000b6c7)
    #8 platform_thread_wrap /home/daver/repos/couchbase/server/platform/src/cb_pthreads.c:23 (libplatform.so.0.1.0+0x000000003730)

  Previous read of size 8 at 0x7d100002db60 by thread T16 (mutexes: write M45851, write M45962):
    #0 std::less<void const*>::operator()(void const* const&, void const* const&) const /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/bits/stl_function.h:371 (ewouldblock_engine.so+0x000000006ac8)
    #1 EWB_Engine::should_inject_error(...) /home/daver/repos/couchbase/server/memcached/engines/ewouldblock_engine/ewouldblock_engine.cc:110 (ewouldblock_engine.so+0x000000006c31)
    #2 EWB_Engine::allocate(...) /home/daver/repos/couchbase/server/memcached/engines/ewouldblock_engine/ewouldblock_engine.cc:211 (ewouldblock_engine.so+0x000000004d33)
    #3 add_set_replace_executor(conn*, void*, ENGINE_STORE_OPERATION) /home/daver/repos/couchbase/server/memcached/daemon/memcached.cc:3940 (memcached+0x0000000e7b40)
    #4 setq_executor(conn*, void*) /home/daver/repos/couchbase/server/memcached/daemon/memcached.cc:4079 (memcached+0x0000000e0496)
    #5 process_bin_packet(conn*) /home/daver/repos/couchbase/server/memcached/daemon/memcached.cc:5051 (memcached+0x0000000d2100)
    #6 run_event_loop /home/daver/repos/couchbase/server/memcached/daemon/connections.cc:174 (memcached+0x0000000c70c3)
    #7 event_handler /home/daver/repos/couchbase/server/memcached/daemon/memcached.cc:6908 (memcached+0x0000000d7b24)
    #8 event_persist_closure /home/couchbase/jenkins/workspace/cbdeps-build/label/ubuntu14.04/release/sherlock/deps/packages/build/libevent/libevent-prefix/src/libevent/event.c:1319 (libevent_core-2.0.so.5+0x00000000b6c7)
    #9 platform_thread_wrap /home/daver/repos/couchbase/server/platform/src/cb_pthreads.c:23 (libplatform.so.0.1.0+0x000000003730)

Change-Id: Ia558b1947f2c913d6fa7943de0e85ba975a0c2b1
Reviewed-on: http://review.couchbase.org/52486
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
daverigby added a commit to daverigby/memcached that referenced this pull request Apr 13, 2016
@daverigby
WIP: Initial index_engine work
12cb35a 
To build:

    cd build/memcached
    make -j8

To run:

    ./memcached_testapp --gtest_filter=*HelloWorld* -e

Currently crashes when trying to perform a allocate:

(lldb) bt
* thread couchbase#4: tid = 0x8eca30, 0x00007fff977b3002 libsystem_kernel.dylib`__pthread_kill + 10, name = 'mc:worker_0, stop reason = signal SIGABRT
  * frame #0: 0x00007fff977b3002 libsystem_kernel.dylib` __pthread_kill  + 10
    frame #1: 0x00007fff969f05c5 libsystem_pthread.dylib` pthread_kill  + 90
    frame #2: 0x00007fff9a9136e7 libsystem_c.dylib` abort  + 129
    frame #3: 0x00000001008ecb58 index_engine.dylib` Index_Engine::allocate(handle=0x000000010316c200, cookie=0x0000000101afd4b0, item=0x0000700000182018, key=0x0000000101ba8820, nkey=1, nbytes=1, flags=0, exptime=0, datatype='\0')  + 72 at index_engine.cc:86
    frame couchbase#4: 0x00000001001ecb7d memcached_testapp` add_set_replace_executor(c=0x0000000101afd180, packet=0x0000000101ba8800, store_op=OPERATION_SET)  + 1085 at mcbp_executors.cc:2765
    frame couchbase#5: 0x00000001001e195f memcached_testapp` set_executor(c=0x0000000101afd180, packet=0x0000000101ba8800)  + 47 at mcbp_executors.cc:2903
    frame couchbase#6: 0x00000001001eb01b memcached_testapp` process_bin_packet(c=0x0000000101afd180)  + 1035 at mcbp_executors.cc:4619
    frame couchbase#7: 0x00000001001eabff memcached_testapp` mcbp_complete_nread(c=0x0000000101afd180)  + 223 at mcbp_executors.cc:4718
    frame couchbase#8: 0x0000000100235c6e memcached_testapp` conn_nread(c=0x0000000101afd180)  + 62 at statemachine_mcbp.cc:310
    frame couchbase#9: 0x00000001001ba62d memcached_testapp` McbpStateMachine::execute(this=0x0000000101a23b10, connection=0x0000000101afd180)  + 29 at statemachine_mcbp.h:43
    frame couchbase#10: 0x00000001001b749c memcached_testapp` McbpConnection::runStateMachinery(this=0x0000000101afd180)  + 636 at connection_mcbp.cc:1007
    frame couchbase#11: 0x00000001001b8e40 memcached_testapp` McbpConnection::runEventLoop(this=0x0000000101afd180, which=2)  + 64 at connection_mcbp.cc:1272
    frame #12: 0x00000001001be9f4 memcached_testapp` ::run_event_loop(c=0x0000000101afd180, which=2)  + 36 at connections.cc:153
    frame #13: 0x0000000100209ab7 memcached_testapp` event_handler(fd=43, which=2, arg=0x0000000101afd180)  + 1911 at memcached.cc:874
    frame #14: 0x0000000100771e01 libevent_core-2.0.5.dylib` event_base_loop  + 2385
    frame #15: 0x000000010024158c memcached_testapp` worker_libevent(arg=0x0000000101b6ec00)  + 92 at thread.cc:231
    frame #16: 0x00000001008846e9 libplatform.0.1.0.dylib` CouchbaseThread::run(this=0x0000000101a1ccd0)  + 521 at cb_pthreads.cc:54
    frame #17: 0x0000000100882fec libplatform.0.1.0.dylib` platform_thread_wrap(arg=0x0000000101a1ccd0)  + 220 at cb_pthreads.cc:66
    frame #18: 0x00007fff969eec13 libsystem_pthread.dylib` _pthread_body  + 131
    frame #19: 0x00007fff969eeb90 libsystem_pthread.dylib` _pthread_start  + 168
    frame #20: 0x00007fff969ec375 libsystem_pthread.dylib` thread_start  + 13

Change-Id: I9161e578d8a007808c176d99f92c49b8e50a2402
ns-codereview pushed a commit that referenced this pull request May 19, 2017
@daverigby @trondn
MB-24460: Subdoc: release item if doc_flag:Add and EEXISTS
04a70b9 
When performing a multi-mutation with the doc_flag 'Add' set, if the
item does exist ensure it is released before returning EEXISTS to the
client.

This addresses the following memory leak:

    ==33026==ERROR: LeakSanitizer: detected memory leaks

    Direct leak of 104 byte(s) in 1 object(s) allocated from:
        #0 0x7f43bc927532 in operator new(unsigned long) (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x99532)
        #1 0x7f43b4c0191f in __unique_if<Item>::__unique_single std::make_unique<>() platform/include/platform/make_unique.h:43
        #2 0x7f43b4c0191f in StoredValue::toItem(bool, unsigned short) const ep-engine/src/stored-value.cc:166
        #3 0x7f43b4c2a3e9 in VBucket::getInternal(DocKey const&, void const*, EventuallyPersistentEngine&, int, get_options_t, bool) ep-engine/src/vbucket.cc:1576
        #4 0x7f43b4be1898 in KVBucket::getInternal(DocKey const&, unsigned short, void const*, vbucket_state_t, get_options_t) ep-engine/src/kv_bucket.cc:1434
        #5 0x7f43b4b9a5e1 in KVBucket::get(DocKey const&, unsigned short, void const*, get_options_t) ep-engine/src/kv_bucket.h:152
        #6 0x7f43b4b9a5e1 in EventuallyPersistentEngine::get(void const*, void**, DocKey const&, unsigned short, get_options_t) ep-engine/src/ep_engine.h:176
        #7 0x7f43b4b9a5e1 in EvpGet ep-engine/src/ep_engine.cc:268
        #8 0x7f43b1244d84  (<unknown module>)
        #9 0x6b38b0 in bucket_get(McbpConnection*, void**, DocKey const&, unsigned short, DocStateFilter) memcached/daemon/protocol/mcbp/engine_wrapper.cc:148
        #10 0x7723a3 in subdoc_fetch memcached/daemon/subdocument.cc:482
        #11 0x7723a3 in subdoc_executor memcached/daemon/subdocument.cc:394
        #12 0x7775fc in subdoc_multi_mutation_executor(McbpConnection*, void*) memcached/daemon/subdocument.cc:1568

Change-Id: Ib1958fa7f697baf60f8367dce977be2b611e2dad
Reviewed-on: http://review.couchbase.org/78347
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@mikewied @dustin @chiyoung