libroach: make DBSstFileWriter output work with time bound iterator #27966
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This change is |
benesch
approved these changes
Jul 26, 2018
There was a problem hiding this comment.
I think it would be cleaner to move the properties collector into its own header file (tableprops.h?), but totally fine as is!
Reviewed 4 of 4 files at r1.
Reviewable status:complete! 0 of 0 LGTMs obtained (and 1 stale)
c-deps/libroach/db.cc, line 712 at r1 (raw file):
std::shared_ptr<rocksdb::TablePropertiesCollectorFactory> time_bound_prop_collector( DBMakeTimeBoundCollector()); options->table_properties_collector_factories.push_back(time_bound_prop_collector);
You should be able to either do
options->table_properties_collector_factories.emplace_back(DBMakeTimeBoundCollector());
or
options->table_properties_collector_factories.emplace_back(
std::make_shared(DBMakeTimeBoundCollector()));
to avoid the temporary.
danhhz
force-pushed
the
import_timebound
branch
2 times, most recently
from
4f34705
to
158d456
Compare
danhhz
commented
Jul 27, 2018
There was a problem hiding this comment.
Moved into timebound.{h,cc}. RFAL!
Reviewable status:
c-deps/libroach/db.cc, line 712 at r1 (raw file):
Previously, benesch (Nikhil Benesch) wrote…
You should be able to either do
options->table_properties_collector_factories.emplace_back(DBMakeTimeBoundCollector());or
options->table_properties_collector_factories.emplace_back( std::make_shared(DBMakeTimeBoundCollector()));to avoid the temporary.
Done
benesch
approved these changes
Jul 27, 2018
There was a problem hiding this comment.
Reviewed 8 of 8 files at r2.
Reviewable status:
The time bounded iterator optimization looks for sstable metadata that indicates the min and max mvcc timestamp included. The iterator (which is configured with a min and max bounds) can use this to skip tables that don't have relevant data. When the bounds aren't set, it has to fall back to checking the table. The TimeBoundTblPropCollectorFactory already sets the metadata on sstable creation, but it wasn't hooked up to DBSstFileWriter. This meant that IMPORT and RESTORE (both of which use DBSstFileWriter via the Import command) would create and ingest sstables that hit the no bounds fall back. This commit adds TimeBoundTblPropCollectorFactory to DBSstFileWriter which means data ingested via RESTORE and IMPORT now work with time bounded iterators just like any other sstables. Closes cockroachdb#26870 Release note (performance improvement): data ingested with RESTORE and IMPORT is now eligible for a performance optimization used in incremental BACKUP and CHANGEFEEDs
|
Thanks for the reviews! bors r=benesch |
craig bot
pushed a commit
that referenced
this pull request
Jul 27, 2018
27966: libroach: make DBSstFileWriter output work with time bound iterator r=benesch a=danhhz The time bounded iterator optimization looks for sstable metadata that indicates the min and max mvcc timestamp included. The iterator (which is configured with a min and max bounds) can use this to skip tables that don't have relevant data. When the bounds aren't set, it has to fall back to checking the table. The TimeBoundTblPropCollectorFactory already sets the metadata on sstable creation, but it wasn't hooked up to DBSstFileWriter. This meant that IMPORT and RESTORE (both of which use DBSstFileWriter via the Import command) would create and ingest sstables that hit the no bounds fall back. This commit adds TimeBoundTblPropCollectorFactory to DBSstFileWriter which means data ingested via RESTORE and IMPORT now work with time bounded iterators just like any other sstables. Closes #26870 Release note (performance improvement): data ingested with RESTORE and IMPORT is now eligible for a performance optimization used in incremental BACKUP and CHANGEFEEDs Co-authored-by: Daniel Harrison <daniel.harrison@gmail.com>
Build succeeded |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The time bounded iterator optimization looks for sstable metadata that
indicates the min and max mvcc timestamp included. The iterator (which
is configured with a min and max bounds) can use this to skip tables
that don't have relevant data. When the bounds aren't set, it has to
fall back to checking the table.
The TimeBoundTblPropCollectorFactory already sets the metadata on
sstable creation, but it wasn't hooked up to DBSstFileWriter. This meant
that IMPORT and RESTORE (both of which use DBSstFileWriter via the
Import command) would create and ingest sstables that hit the no bounds
fall back.
This commit adds TimeBoundTblPropCollectorFactory to DBSstFileWriter
which means data ingested via RESTORE and IMPORT now work with time
bounded iterators just like any other sstables.
Closes #26870
Release note (performance improvement): data ingested with RESTORE and
IMPORT is now eligible for a performance optimization used in
incremental BACKUP and CHANGEFEEDs