Concurrent map #19
Merged
Concurrent map #19
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## schema_change #19 +/- ##
=================================================
+ Coverage 90.1% 90.18% +0.08%
=================================================
Files 82 83 +1
Lines 3496 3536 +40
=================================================
+ Hits 3150 3189 +39
- Misses 346 347 +1
Continue to review full report at Codecov.
|
This was referenced Apr 6, 2019
yashNaN
approved these changes
Apr 7, 2019
saikiriti93
approved these changes
Apr 7, 2019
yashNaN
pushed a commit
that referenced
this pull request
Sep 4, 2019
* Initial rewrite of SerializeRecord() * Add SqlTable pointer back to log record * Save work before merging in Tianyu's PR * Implement new ReadNextRecord for log_test * Tweak log_test for new logging format * Increase number of rows in log_test table for robustness * Remove DataTable nullptrs for logging benchmark * Add logging for inlined varlens * Add functions for calculating size of log and for serialization * Removing unused functions * BUGGY * Bug fix * Bug fix * Refactor * Formatting Changes * Add comments and Refactor * Put buffers vector fillup and queue fillup in different loops * Refactored access of first element in vector * Fixed Doxygen: Missing comments * Bug Fix in persisting buffers * Removed unnecessary nullptr check * Added comments * Created a new class for log writer thread * Created a new class for log writer thread * Added destructor for LogManager * Wrapper for Producer Consumer Queue * Robust start and shutdown * Fixed memory leak and formatting * Replaced infinite loop while persisting with CV * Condition Variables for Non-Blocking Enqueue * Bug fix * Deleted the container for spdlog * Added comments * Fix typo in comments * Fixed deadlock * Removed unused macro * Logging: Address review comments (#19) * Address review comments * Address Utkarsh's review comments * Delete log file before test case begins * Add deleting log file during set up to benchmark * Remove duplicate code and general clarity fixes * Change thread dispatcher API * Change LogManager to use thread registry for log consumer task * Fix clang tidy * Fix compilation * Fix comments * Remove unecessary destructor and more comments * Remove catalog, and add log manager settings to SettingsManager * Forgot benchmark folder * Fix up merge changes * Missed some fixes, as well as doxygen * clang tidy shut the fuck up for the love of all that is holy * Fix compilation error in benchmark * Rename LogConsumerTask to DiskLogWriterTask * Whoops, fixed memory leak * Whoops, fixed memory leak * Add settings test to help test coverage * Add latch for thread safety of ThreadRegistry tables * Missing include * Missing NOLINT * Modify ThreadRegistry API * Fix log manager constructors * Properly call commit callbacks when persist happens, as well as decouple serializing and processing in LogManager * Change registry API to construct and own tasks, plus some general log manager fixes * Change registry API to construct and own tasks, plus some general log manager fixes (fucked up last commit whoops) * Add LogSerializerTask and LogFlusherTask to handle log processing and flushing in background * Update Log test to use log manager tasks * Fix serialization bug involving varlens * Update DBMain and TPCCTest to use new log manager * Rename Shutdown to PersistAndStop, as well as update benchmarks to use new LogManager API * Fix clang-shitty * Change registry to use spin latch, plus formatting fixes * Changed DedicatedThreadOwner API. Added hella documentation * Rename DiskLogWriterTask to DiskLogConsumerTask * Fix connection dispatcher destructor bug * Doxygen fixes * Documentation fixes * Im dumb, forgot to reset commit callbacks, it was executing an absurd amount of callbacks * Fix race condition with task iniialization and termination * Formatting * Lint fix * Increase number of log manager buffers * Tweak logging intervals * Change API to allow use of ForceFlush, allowing clients to persist without shutting the whole thing down * Change file path to be stored as string * Formatting * Clang tidy fix * Possible fixes to tpcc and benchmarks * More fixes * Possible race condition fix * Resolve Justin's TODO'S * Move log flusher to separate file * Move log serializer task to separate file * Formatting and Lint fixes * Clang tidy fix * Update comments * Remove atomic bool flag and add more comments * Test moving flush buffers to a temp queue * Fix unused result warning * I forgot formatting fml * Decouple log manager from serializer task * Decouple log manager from disk writer task * Clarify names for DiskLogConsumerTask methods * Move log file persisting logic to DiskLogConsumerTask * Change certain sleeps to yield * Move buffer size constant to constants file * Add persist threshold to DiskLogConsumerTask * Doxygen and formatting * Remove unused variable * Fix race
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Updates the internal representation for versioned DataTable objects to use the wrapper for TBB's concurrent unordered hash map. The switch highlighted invalid assumptions in the SQL table test (scan does not guarantee any ordering on results) as well as a logic error in populating old versions into new versions (need to set null on missing columns).