Windows cumulative patch #3552
Windows cumulative patch #3552
Conversation
|
@yuslepukhin has updated the pull request. View: changes |
|
@yuslepukhin has updated the pull request. View: changes |
|
@yuslepukhin has updated the pull request. View: changes |
yuslepukhin
force-pushed
the
windows_cumulative
branch
from
4bf43b9
to
d45cdc2
Compare
|
@yuslepukhin has updated the pull request. |
|
@yiwu-arbug @siying @maysamyabandeh @anand1976 Pls, take a look. This is fairly straight forward. |
There was a problem hiding this comment.
@siying has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
|
Looks good to me. I'm going to merge it after tests pass. |
| void OnStallConditionsChanged(const WriteStallInfo& info) override { | ||
| MutexLock l(&mutex_); | ||
| condition_ = info.condition.cur; | ||
| if (expected_set_ && | ||
| condition_ == expected_) { | ||
| cond_.Signal(); |
There was a problem hiding this comment.
@yuslepukhin ASAN complains this:
Failed to find test summary. This test binary may have crashed mid-run. Full output follows:
Note: Google Test filter = DBTest.SoftLimit
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from DBTest
[ RUN ] DBTest.SoftLimit
internal_repo_rocksdb/repo/db/db_test.cc:5391:9: runtime error: load of value 190, which is not a valid value for type 'bool'
#0 0x681976 in rocksdb::WriteStallListener::OnStallConditionsChanged(rocksdb::WriteStallInfo const&) internal_repo_rocksdb/repo/db/db_test.cc:5391
#1 0x7f7cf9d91494 in rocksdb::SuperVersionContext::Clean() internal_repo_rocksdb/repo/db/job_context.h:63
#2 0x7f7cf9d826e2 in rocksdb::JobContext::Clean() internal_repo_rocksdb/repo/db/job_context.h:162
#3 0x7f7cf9f33d83 in rocksdb::DBImpl::BackgroundCallFlush() internal_repo_rocksdb/repo/db/db_impl_compaction_flush.cc:1404
#4 0x7f7cf9f30fe5 in rocksdb::DBImpl::BGWorkFlush(void*) internal_repo_rocksdb/repo/db/db_impl_compaction_flush.cc:1250
#5 0x7f7cfa6af0f1 in rocksdb::ThreadPoolImpl::Schedule(void (*)(void*), void*, void*, void (*)(void*))::$_1::operator()() const internal_repo_rocksdb/repo/util/threadpool_imp.cc:441
#6 0x7f7cfa6aeefc in std::_Function_handler<void (), rocksdb::ThreadPoolImpl::Schedule(void (*)(void*), void*, void*, void (*)(void*))::$_1>::_M_invoke(std::_Any_data const&) third-party-buck/gcc-5-glibc-2.23/build/libgcc/include/c++/trunk/functional:1871
#7 0x7f7cfa18325e in std::function<void ()>::operator()() const third-party-buck/gcc-5-glibc-2.23/build/libgcc/include/c++/trunk/functional:2267
#8 0x7f7cfa6ab557 in rocksdb::ThreadPoolImpl::Impl::BGThread(unsigned long) internal_repo_rocksdb/repo/util/threadpool_imp.cc:240
#9 0x7f7cfa6ab839 in rocksdb::ThreadPoolImpl::Impl::BGThreadWrapper(void*) internal_repo_rocksdb/repo/util/threadpool_imp.cc:278
#10 0x7f7cfa6b560e in void* std::_Bind_simple<void* (*(rocksdb::BGThreadMetadata*))(void*)>::_M_invoke<0ul>(std::_Index_tuple<0ul>) third-party-buck/gcc-5-glibc-2.23/build/libgcc/include/c++/trunk/functional:1530
#11 0x7f7cfa6b5574 in std::_Bind_simple<void* (*(rocksdb::BGThreadMetadata*))(void*)>::operator()() third-party-buck/gcc-5-glibc-2.23/build/libgcc/include/c++/trunk/functional:1520
#12 0x7f7cfa6b52a8 in std::thread::_Impl<std::_Bind_simple<void* (*(rocksdb::BGThreadMetadata*))(void*)> >::_M_run() third-party-buck/gcc-5-glibc-2.23/build/libgcc/include/c++/trunk/thread:115
#13 0x7f7cf6e56170 in execute_native_thread_routine /home/engshare/third-party2/libgcc/5.x/src/gcc-5/x86_64-facebook-linux/libstdc++-v3/src/c++11/../../../.././libstdc++-v3/src/c++11/thread.cc:84
#14 0x7f7cf4cab7a8 in start_thread /home/engshare/third-party2/glibc/2.23/src/glibc-2.23/nptl/pthread_create.c:333
#15 0x7f7cf44d7a7c in __clone /home/engshare/third-party2/glibc/2.23/src/glibc-2.23/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:109
I assume it's because expected_set_ has never been initialized?
Checkpoint test pass. Add asserts to win_logger along with the Buffer Flush on close.
number w/o limitations of ReFS
Align unbuffered read offsets to sector size.
the notification would arrive in time but it does not always happen. Rework with a condition wait/notify when that takes place.
where the test attempted to open a directory using NewRandomAccesFile and would not check the return status.
yuslepukhin
force-pushed
the
windows_cumulative
branch
from
d45cdc2
to
4f6e086
Compare
|
@yuslepukhin has updated the pull request. |
yuslepukhin
force-pushed
the
windows_cumulative
branch
from
4f6e086
to
7469117
Compare
|
@yuslepukhin has updated the pull request. |
There was a problem hiding this comment.
@siying has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
|
@siying Travis failed due to time limit although there are some more badly written tests that now fail after rebase. This check in also ensures that AppVeyour fails when test fails. |
|
@siying Found one more issue |
|
@yuslepukhin has updated the pull request. View: changes, changes since last import |
|
@siying Would it be possible to make the build fail on std::thread reference instead of port::Thread? |
|
@yuslepukhin sure. I don't know how to do it though. |
There was a problem hiding this comment.
@siying is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
This patch addressed several issues.
Portability including db_test std::thread -> port::Thread Cc: @
and %z to ROCKSDB portable macro. Cc: @maysamyabandeh
Implement Env::AreFilesSame
Make the implementation of file unique number more robust
Get rid of C-runtime and go directly to Windows API when dealing
with file primitives.
Implement GetSectorSize() and aling unbuffered read on the value if
available.
Adjust Windows Logger for the new interface, implement CloseImpl() Cc: @anand1976
Fix test running script issue where $status var was of incorrect scope
so the failures were swallowed and not reported.
DestroyDB() creates a logger and opens a LOG file in the directory
being cleaned up. This holds a lock on the folder and the cleanup is
prevented. This fails one of the checkpoin tests. We observe the same in production.
We close the log file in this change.
Fix DBTest2.ReadAmpBitmapLiveInCacheAfterDBClose failure where the test
attempts to open a directory with NewRandomAccessFile which does not
work on Windows.
Fix DBTest.SoftLimit as it is dependent on thread timing. CC: @yiwu-arbug