Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

libmusl compatibility #11

Closed
wants to merge 1 commit into from
Closed

libmusl compatibility #11

wants to merge 1 commit into from

Conversation

jlopez
Copy link

@jlopez jlopez commented Jun 8, 2016 •
edited

I'm working on compiling couchbase on alpine linux. There were just two changes needed to get forestdb to successfully run all tests:

spin_t

The existing code doesn't call spin_init() for some (most?) spin_t variables. The code works fine, however, since SPIN_INITIALIZER seems to be properly 'initializing' them. This is not the case when running on libmusl though, and calling spin_lock() on such 'initialized' spin_t's leads to a deadlock.

While the proper solution would entail properly initializing all spin_t variables by calling spin_init(), I'm reluctant to do so, as there's no clear place where this can be done. Instead, setting the initializer to the correct value for musl (spin_t)(0) instead of (spin_t)(1) does the trick.

pthread stack

Threads are crashing since the default musl pthread stack size is only 80k as opposed to the 2Mb when running on glibc. So I'm explicitly setting the stack size to 2Mb.

Change-Id: I02f7f4123a8d46d0ce1cef2285ac8fcefe02dc0d

@jlopez
libmusl compatibility
2478349 
Change-Id: I02f7f4123a8d46d0ce1cef2285ac8fcefe02dc0d
@chiyoung
Copy link
Contributor

chiyoung commented Jun 9, 2016

Thanks for reporting these issues.

We made a fix to the spin lock init issue:

http://review.couchbase.org/#/c/64846/

Regarding the pthread stack size limit, we will address it with a slightly different way soon and also make sure that we don't allocate large chunk memory in the local stack.

@jlopez
Copy link
Author

jlopez commented Jun 11, 2016

Great! I'll close this pull request then.

@jlopez jlopez closed this Jun 11, 2016
abhinavdangeti added a commit to abhinavdangeti/forestdb that referenced this pull request Jul 19, 2016
@abhinavdangeti
Address heap-use-after-free issue seen with compaction
fb67696 
16:45:42 ==68667==ERROR: AddressSanitizer: heap-use-after-free on address 0x61f0000088c0 at pc 0x0000005452bb bp 0x7fffdf556050 sp 0x7fffdf556048
16:45:42 READ of size 8 at 0x61f0000088c0 thread T0
16:45:42     #0 0x5452ba in FileMgr::getNewFile() /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/src/filemgr.h:636:16
16:45:42     couchbase#1 0x5452ba in FileMgr::updateFilePointers() /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/src/filemgr.cc:1528
16:45:42     couchbase#2 0x5452ba in FileMgr::close(FileMgr*, bool, char const*, ErrLogCallback*) /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/src/filemgr.cc:1677
16:45:42     couchbase#3 0x569bc3 in _fdb_close /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/src/forestdb.cc:7434:10
16:45:42     couchbase#4 0x5b3961 in _fdb_kvs_close(FdbKvsHandle*) /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/src/kv_instance.cc:1470:10
16:45:42     couchbase#5 0x5b3961 in fdb_kvs_close /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/src/kv_instance.cc:1537
16:45:42     couchbase#6 0x4f9323 in FileHandlePool::~FileHandlePool() /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/tests/usecase/usecase_test.cc:129:26
16:45:42     couchbase#7 0x4f94ed in FileHandlePool::~FileHandlePool() /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/tests/usecase/usecase_test.cc:124:31
16:45:42     couchbase#8 0x4f56de in test_writes_on_kv_stores_with_compaction(unsigned short, int) /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/tests/usecase/usecase_test.cc:815:5
16:45:42     couchbase#9 0x4f5f9a in main /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/tests/usecase/usecase_test.cc:880:5
16:45:42     couchbase#10 0x2b069543776c in __libc_start_main /build/buildd/eglibc-2.15/csu/libc-start.c:226
16:45:42     couchbase#11 0x447558 in _start (/home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/build/forestdb/tests/usecase/usecase_test+0x447558)
16:45:42
16:45:42 0x61f0000088c0 is located 2624 bytes inside of 3064-byte region [0x61f000007e80,0x61f000008a78)
16:45:42 freed by thread T47 here:
16:45:42     #0 0x4edf82 in operator delete(void*) (/home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/build/forestdb/tests/usecase/usecase_test+0x4edf82)
16:45:42     couchbase#1 0x541095 in FileMgr::freeFunc(FileMgr*) /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/src/filemgr.cc:1758:5
16:45:42     couchbase#2 0x525c7c in CompactorThread::run() /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/src/compactor.cc:398:17
16:45:42     couchbase#3 0x524de8 in launch_compactor_thread(void*) /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/src/compactor.cc:261:9
16:45:42     couchbase#4 0x2b06941a4e99 in start_thread /build/buildd/eglibc-2.15/nptl/pthread_create.c:308

Change-Id: Ifb5eb2255953b569891ebb7e59f36d902dad1152
abhinavdangeti added a commit to abhinavdangeti/forestdb that referenced this pull request Jul 19, 2016
@abhinavdangeti
Address heap-use-after-free issue seen with compaction
7119401 
16:45:42 ==68667==ERROR: AddressSanitizer: heap-use-after-free on address 0x61f0000088c0 at pc 0x0000005452bb bp 0x7fffdf556050 sp 0x7fffdf556048
16:45:42 READ of size 8 at 0x61f0000088c0 thread T0
16:45:42     #0 0x5452ba in FileMgr::getNewFile() /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/src/filemgr.h:636:16
16:45:42     couchbase#1 0x5452ba in FileMgr::updateFilePointers() /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/src/filemgr.cc:1528
16:45:42     couchbase#2 0x5452ba in FileMgr::close(FileMgr*, bool, char const*, ErrLogCallback*) /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/src/filemgr.cc:1677
16:45:42     couchbase#3 0x569bc3 in _fdb_close /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/src/forestdb.cc:7434:10
16:45:42     couchbase#4 0x5b3961 in _fdb_kvs_close(FdbKvsHandle*) /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/src/kv_instance.cc:1470:10
16:45:42     couchbase#5 0x5b3961 in fdb_kvs_close /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/src/kv_instance.cc:1537
16:45:42     couchbase#6 0x4f9323 in FileHandlePool::~FileHandlePool() /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/tests/usecase/usecase_test.cc:129:26
16:45:42     couchbase#7 0x4f94ed in FileHandlePool::~FileHandlePool() /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/tests/usecase/usecase_test.cc:124:31
16:45:42     couchbase#8 0x4f56de in test_writes_on_kv_stores_with_compaction(unsigned short, int) /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/tests/usecase/usecase_test.cc:815:5
16:45:42     couchbase#9 0x4f5f9a in main /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/tests/usecase/usecase_test.cc:880:5
16:45:42     couchbase#10 0x2b069543776c in __libc_start_main /build/buildd/eglibc-2.15/csu/libc-start.c:226
16:45:42     couchbase#11 0x447558 in _start (/home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/build/forestdb/tests/usecase/usecase_test+0x447558)
16:45:42
16:45:42 0x61f0000088c0 is located 2624 bytes inside of 3064-byte region [0x61f000007e80,0x61f000008a78)
16:45:42 freed by thread T47 here:
16:45:42     #0 0x4edf82 in operator delete(void*) (/home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/build/forestdb/tests/usecase/usecase_test+0x4edf82)
16:45:42     couchbase#1 0x541095 in FileMgr::freeFunc(FileMgr*) /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/src/filemgr.cc:1758:5
16:45:42     couchbase#2 0x525c7c in CompactorThread::run() /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/src/compactor.cc:398:17
16:45:42     couchbase#3 0x524de8 in launch_compactor_thread(void*) /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/src/compactor.cc:261:9
16:45:42     couchbase#4 0x2b06941a4e99 in start_thread /build/buildd/eglibc-2.15/nptl/pthread_create.c:308

Change-Id: Ifb5eb2255953b569891ebb7e59f36d902dad1152
abhinavdangeti added a commit to abhinavdangeti/forestdb that referenced this pull request Jul 26, 2016
@abhinavdangeti
Address heap-use-after-free issue seen with compaction
3bc1f73 
While updating the prevFile pointer of the next file in
the chain, update the pointer iff the next file points
to the current file.

16:45:42 ==68667==ERROR: AddressSanitizer: heap-use-after-free on address 0x61f0000088c0 at pc 0x0000005452bb bp 0x7fffdf556050 sp 0x7fffdf556048
16:45:42 READ of size 8 at 0x61f0000088c0 thread T0
16:45:42     #0 0x5452ba in FileMgr::getNewFile() /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/src/filemgr.h:636:16
16:45:42     couchbase#1 0x5452ba in FileMgr::updateFilePointers() /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/src/filemgr.cc:1528
16:45:42     couchbase#2 0x5452ba in FileMgr::close(FileMgr*, bool, char const*, ErrLogCallback*) /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/src/filemgr.cc:1677
16:45:42     couchbase#3 0x569bc3 in _fdb_close /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/src/forestdb.cc:7434:10
16:45:42     couchbase#4 0x5b3961 in _fdb_kvs_close(FdbKvsHandle*) /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/src/kv_instance.cc:1470:10
16:45:42     couchbase#5 0x5b3961 in fdb_kvs_close /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/src/kv_instance.cc:1537
16:45:42     couchbase#6 0x4f9323 in FileHandlePool::~FileHandlePool() /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/tests/usecase/usecase_test.cc:129:26
16:45:42     couchbase#7 0x4f94ed in FileHandlePool::~FileHandlePool() /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/tests/usecase/usecase_test.cc:124:31
16:45:42     couchbase#8 0x4f56de in test_writes_on_kv_stores_with_compaction(unsigned short, int) /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/tests/usecase/usecase_test.cc:815:5
16:45:42     couchbase#9 0x4f5f9a in main /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/tests/usecase/usecase_test.cc:880:5
16:45:42     couchbase#10 0x2b069543776c in __libc_start_main /build/buildd/eglibc-2.15/csu/libc-start.c:226
16:45:42     couchbase#11 0x447558 in _start (/home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/build/forestdb/tests/usecase/usecase_test+0x447558)
16:45:42
16:45:42 0x61f0000088c0 is located 2624 bytes inside of 3064-byte region [0x61f000007e80,0x61f000008a78)
16:45:42 freed by thread T47 here:
16:45:42     #0 0x4edf82 in operator delete(void*) (/home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/build/forestdb/tests/usecase/usecase_test+0x4edf82)
16:45:42     couchbase#1 0x541095 in FileMgr::freeFunc(FileMgr*) /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/src/filemgr.cc:1758:5
16:45:42     couchbase#2 0x525c7c in CompactorThread::run() /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/src/compactor.cc:398:17
16:45:42     couchbase#3 0x524de8 in launch_compactor_thread(void*) /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/src/compactor.cc:261:9
16:45:42     couchbase#4 0x2b06941a4e99 in start_thread /build/buildd/eglibc-2.15/nptl/pthread_create.c:308

Change-Id: Ifb5eb2255953b569891ebb7e59f36d902dad1152
ns-codereview pushed a commit that referenced this pull request Aug 19, 2021
@daverigby @trondn
Fix leaks in compact_functional_test
db3a673 
As reported by ASan:

    ==25822==ERROR: LeakSanitizer: detected memory leaks

    Direct leak of 721424 byte(s) in 11 object(s) allocated from:
	#0 0xffff9f6d18bc in malloc (/opt/gcc-10.2.0/lib64/libasan.so.6+0x9d8bc)
	#1 0x4b0284 in hash_init ../forestdb/src/hash.cc:32
	#2 0x40e254 in _fname_create ../forestdb/src/blockcache.cc:806
	#3 0x410698 in bcache_write ../forestdb/src/blockcache.cc:1063
	#4 0x457dc8 in filemgr_read ../forestdb/src/filemgr.cc:2147
	#5 0x4474d4 in _docio_read_through_buffer(docio_handle*, unsigned long, err_log_callback*, bool) ../forestdb/src/docio.cc:711
	#6 0x4474d4 in _docio_read_length ../forestdb/src/docio.cc:803
	#7 0x44c8c0 in docio_read_doc ../forestdb/src/docio.cc:1242
	#8 0x4d6a3c in fdb_kvs_header_read ../forestdb/src/kv_instance.cc:1142
	#9 0x485b94 in _fdb_open ../forestdb/src/forestdb.cc:2076
	#10 0x4adfbc in _fdb_recover_compaction(_fdb_kvs_handle*, char const*) ../forestdb/src/forestdb.cc:585
	#11 0x4869ac in _fdb_open ../forestdb/src/forestdb.cc:2237
	#12 0x486f74 in fdb_open ../forestdb/src/forestdb.cc:848
	#13 0x525da0 in compact_rename_to_original_test() ../forestdb/tests/functional/compact_functional_test.cc:525
	#14 0x546878 in main ../forestdb/tests/functional/compact_functional_test.cc:3904
	#15 0xffff9e4e2ce0 in __libc_start_main (/lib64/libc.so.6+0x1fce0)
	#16 0x404c4c  (/home/couchbase/server/build/forestdb/tests/functional/compact_functional_test+0x404c4c)

    Direct leak of 721424 byte(s) in 11 object(s) allocated from:
	#0 0xffff9f6d18bc in malloc (/opt/gcc-10.2.0/lib64/libasan.so.6+0x9d8bc)
	#1 0x4b0284 in hash_init ../forestdb/src/hash.cc:32
	#2 0x40e254 in _fname_create ../forestdb/src/blockcache.cc:806
	#3 0x410698 in bcache_write ../forestdb/src/blockcache.cc:1063
	#4 0x457dc8 in filemgr_read ../forestdb/src/filemgr.cc:2147
	#5 0x4474d4 in _docio_read_through_buffer(docio_handle*, unsigned long, err_log_callback*, bool) ../forestdb/src/docio.cc:711
	#6 0x4474d4 in _docio_read_length ../forestdb/src/docio.cc:803
	#7 0x44c8c0 in docio_read_doc ../forestdb/src/docio.cc:1242
	#8 0x4d6a3c in fdb_kvs_header_read ../forestdb/src/kv_instance.cc:1142
	#9 0x485b94 in _fdb_open ../forestdb/src/forestdb.cc:2076
	#10 0x486f74 in fdb_open ../forestdb/src/forestdb.cc:848
	#11 0x524460 in compaction_delete_old_test() ../forestdb/tests/functional/compact_functional_test.cc:345
	#12 0x546874 in main ../forestdb/tests/functional/compact_functional_test.cc:3903
	#13 0xffff9e4e2ce0 in __libc_start_main (/lib64/libc.so.6+0x1fce0)
	#14 0x404c4c  (/home/couchbase/server/build/forestdb/tests/functional/compact_functional_test+0x404c4c)

Change-Id: I6179654af9da764d4ceed04fb2702ca84f43c874
Reviewed-on: http://review.couchbase.org/c/forestdb/+/159308
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Trond Norbye <trond.norbye@couchbase.com>
greensky00 added a commit to greensky00/forestdb that referenced this pull request Sep 5, 2023
@greensky00
Bypass WAL in bottom-up build mode (couchbase#11)
d94628b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@jlopez @chiyoung