Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Introduce SegmentId class #6370

Merged
merged 48 commits into from
Jan 21, 2019
Merged

Introduce SegmentId class #6370

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
222f732
Introduce SegmentId class
leventov Sep 21, 2018
bba860f
tmp
leventov Sep 24, 2018
9549da1
Merge remote-tracking branch 'upstream/master' into segmentId
leventov Sep 28, 2018
3be105c
Fix SelectQueryRunnerTest
leventov Sep 28, 2018
1d82f8c
Fix indentation
leventov Sep 28, 2018
17a3651
Fixes
leventov Sep 30, 2018
0468ea1
Merge remote-tracking branch 'upstream/master' into segmentId
leventov Sep 30, 2018
e0c8ada
Remove Comparators.inverse() tests
leventov Sep 30, 2018
9586226
Refinements
leventov Sep 30, 2018
47e9007
Fix tests
leventov Oct 1, 2018
6da4f3a
Fix more tests
leventov Oct 1, 2018
2684edb
Remove duplicate DataSegmentTest, fixes #6064
leventov Oct 1, 2018
0a10fc3
SegmentDescriptor doc
leventov Oct 1, 2018
6213f7e
Merge remote-tracking branch 'upstream/master' into segmentId
leventov Oct 1, 2018
33f5ae0
Fix SQLMetadataStorageUpdaterJobHandler
leventov Oct 1, 2018
74b947f
Fix DataSegment deserialization for ignoring id
leventov Oct 2, 2018
7941d02
Merge remote-tracking branch 'upstream/master' into segmentId
leventov Oct 2, 2018
57a8be4
Merge remote-tracking branch 'upstream/master' into segmentId
leventov Oct 8, 2018
24dd181
Add comments
leventov Oct 10, 2018
78f6074
More comments
leventov Oct 10, 2018
62afb7d
Address more comments
leventov Oct 11, 2018
134fe00
Merge remote-tracking branch 'upstream/master' into segmentId
leventov Oct 11, 2018
6297261
Fix compilation
leventov Oct 11, 2018
c1c296e
Restore segment2 in SystemSchemaTest according to a comment
leventov Oct 12, 2018
218c8cd
Merge remote-tracking branch 'upstream/master' into segmentId
leventov Oct 12, 2018
c49b544
Fix style
leventov Oct 12, 2018
b8f5c22
Merge remote-tracking branch 'upstream/master' into segmentId
leventov Oct 15, 2018
f90df21
fix testServerSegmentsTable
leventov Oct 16, 2018
fbe9eff
Merge remote-tracking branch 'upstream/master' into segmentId
leventov Oct 23, 2018
995b447
Fix compilation
leventov Oct 23, 2018
ed045e5
Merge branch 'master' into segmentId
leventov Oct 23, 2018
3395b96
Merge remote-tracking branch 'upstream/master' into segmentId
leventov Oct 27, 2018
23a2e76
Merge remote-tracking branch 'upstream/master' into segmentId
leventov Oct 29, 2018
b3f5edb
Merge remote-tracking branch 'upstream/master' into segmentId
leventov Nov 8, 2018
bde967f
Merge remote-tracking branch 'upstream/master' into segmentId
leventov Nov 16, 2018
61f83ef
Merge remote-tracking branch 'upstream/master' into segmentId
leventov Nov 19, 2018
85319d0
Add comments about why SegmentId and SegmentIdWithShardSpec are separ…
leventov Nov 19, 2018
83acbb9
Merge remote-tracking branch 'upstream/master' into segmentId
leventov Nov 28, 2018
73216df
Merge remote-tracking branch 'upstream/master' into segmentId
leventov Dec 20, 2018
5619dbf
Merge remote-tracking branch 'upstream/master' into segmentId
leventov Jan 7, 2019
d7676b9
Fix SystemSchemaTest
leventov Jan 9, 2019
1591067
Merge remote-tracking branch 'upstream/master' into segmentId
leventov Jan 9, 2019
d8db03f
Fix style
leventov Jan 9, 2019
22573af
Compare SegmentDescriptor with SegmentId in Javadoc and comments rath…
leventov Jan 9, 2019
15624f5
Remove a link, see https://youtrack.jetbrains.com/issue/IDEA-205164
leventov Jan 10, 2019
d86799f
Merge remote-tracking branch 'upstream/master' into segmentId
leventov Jan 15, 2019
d889962
Fix compilation
leventov Jan 16, 2019
bb43e57
Merge remote-tracking branch 'upstream/master' into segmentId
leventov Jan 16, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
7 changes: 7 additions & 0 deletions .idea/inspectionProfiles/Druid.xml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 5 additions & 4 deletions benchmarks/src/main/java/org/apache/druid/benchmark/FilteredAggregatorBenchmark.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@
import org.apache.druid.segment.incremental.IncrementalIndex;
import org.apache.druid.segment.serde.ComplexMetrics;
import org.apache.druid.segment.writeout.OffHeapMemorySegmentWriteOutMediumFactory;
import org.apache.druid.timeline.SegmentId;
import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Fork;
Expand Down Expand Up @@ -266,8 +267,8 @@ public void querySingleIncrementalIndex(Blackhole blackhole)
{
QueryRunner<Result<TimeseriesResultValue>> runner = QueryBenchmarkUtil.makeQueryRunner(
factory,
"incIndex",
new IncrementalIndexSegment(incIndex, "incIndex")
SegmentId.dummy("incIndex"),
new IncrementalIndexSegment(incIndex, SegmentId.dummy("incIndex"))
);

List<Result<TimeseriesResultValue>> results = FilteredAggregatorBenchmark.runQuery(factory, runner, query);
Expand All @@ -283,8 +284,8 @@ public void querySingleQueryableIndex(Blackhole blackhole)
{
final QueryRunner<Result<TimeseriesResultValue>> runner = QueryBenchmarkUtil.makeQueryRunner(
factory,
"qIndex",
new QueryableIndexSegment("qIndex", qIndex)
SegmentId.dummy("qIndex"),
new QueryableIndexSegment(qIndex, SegmentId.dummy("qIndex"))
);

List<Result<TimeseriesResultValue>> results = FilteredAggregatorBenchmark.runQuery(factory, runner, query);
Expand Down
99 changes: 51 additions & 48 deletions benchmarks/src/main/java/org/apache/druid/benchmark/GroupByTypeInterfaceBenchmark.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@
import org.apache.druid.segment.incremental.IncrementalIndex;
import org.apache.druid.segment.serde.ComplexMetrics;
import org.apache.druid.segment.writeout.OffHeapMemorySegmentWriteOutMediumFactory;
import org.apache.druid.timeline.SegmentId;
import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Fork;
Expand Down Expand Up @@ -108,6 +109,8 @@
@Measurement(iterations = 30)
public class GroupByTypeInterfaceBenchmark
{
private static final SegmentId Q_INDEX_SEGMENT_ID = SegmentId.dummy("qIndex");

@Param({"4"})
private int numSegments;

Expand Down Expand Up @@ -462,8 +465,8 @@ public void querySingleQueryableIndexStringOnly(Blackhole blackhole)
{
QueryRunner<Row> runner = QueryBenchmarkUtil.makeQueryRunner(
factory,
"qIndex",
new QueryableIndexSegment("qIndex", queryableIndexes.get(0))
Q_INDEX_SEGMENT_ID,
new QueryableIndexSegment(queryableIndexes.get(0), Q_INDEX_SEGMENT_ID)
);

List<Row> results = GroupByTypeInterfaceBenchmark.runQuery(factory, runner, stringQuery);
Expand All @@ -480,8 +483,8 @@ public void querySingleQueryableIndexLongOnly(Blackhole blackhole)
{
QueryRunner<Row> runner = QueryBenchmarkUtil.makeQueryRunner(
factory,
"qIndex",
new QueryableIndexSegment("qIndex", queryableIndexes.get(0))
Q_INDEX_SEGMENT_ID,
new QueryableIndexSegment(queryableIndexes.get(0), Q_INDEX_SEGMENT_ID)
);

List<Row> results = GroupByTypeInterfaceBenchmark.runQuery(factory, runner, longQuery);
Expand All @@ -498,8 +501,8 @@ public void querySingleQueryableIndexFloatOnly(Blackhole blackhole)
{
QueryRunner<Row> runner = QueryBenchmarkUtil.makeQueryRunner(
factory,
"qIndex",
new QueryableIndexSegment("qIndex", queryableIndexes.get(0))
Q_INDEX_SEGMENT_ID,
new QueryableIndexSegment(queryableIndexes.get(0), Q_INDEX_SEGMENT_ID)
);

List<Row> results = GroupByTypeInterfaceBenchmark.runQuery(factory, runner, floatQuery);
Expand All @@ -516,8 +519,8 @@ public void querySingleQueryableIndexNumericOnly(Blackhole blackhole)
{
QueryRunner<Row> runner = QueryBenchmarkUtil.makeQueryRunner(
factory,
"qIndex",
new QueryableIndexSegment("qIndex", queryableIndexes.get(0))
Q_INDEX_SEGMENT_ID,
new QueryableIndexSegment(queryableIndexes.get(0), Q_INDEX_SEGMENT_ID)
);

List<Row> results = GroupByTypeInterfaceBenchmark.runQuery(factory, runner, longFloatQuery);
Expand All @@ -534,8 +537,8 @@ public void querySingleQueryableIndexNumericThenString(Blackhole blackhole)
{
QueryRunner<Row> runner = QueryBenchmarkUtil.makeQueryRunner(
factory,
"qIndex",
new QueryableIndexSegment("qIndex", queryableIndexes.get(0))
Q_INDEX_SEGMENT_ID,
new QueryableIndexSegment(queryableIndexes.get(0), Q_INDEX_SEGMENT_ID)
);

List<Row> results = GroupByTypeInterfaceBenchmark.runQuery(factory, runner, longFloatQuery);
Expand All @@ -546,8 +549,8 @@ public void querySingleQueryableIndexNumericThenString(Blackhole blackhole)

runner = QueryBenchmarkUtil.makeQueryRunner(
factory,
"qIndex",
new QueryableIndexSegment("qIndex", queryableIndexes.get(0))
Q_INDEX_SEGMENT_ID,
new QueryableIndexSegment(queryableIndexes.get(0), Q_INDEX_SEGMENT_ID)
);

results = GroupByTypeInterfaceBenchmark.runQuery(factory, runner, stringQuery);
Expand All @@ -565,8 +568,8 @@ public void querySingleQueryableIndexLongThenString(Blackhole blackhole)
{
QueryRunner<Row> runner = QueryBenchmarkUtil.makeQueryRunner(
factory,
"qIndex",
new QueryableIndexSegment("qIndex", queryableIndexes.get(0))
Q_INDEX_SEGMENT_ID,
new QueryableIndexSegment(queryableIndexes.get(0), Q_INDEX_SEGMENT_ID)
);

List<Row> results = GroupByTypeInterfaceBenchmark.runQuery(factory, runner, longQuery);
Expand All @@ -577,8 +580,8 @@ public void querySingleQueryableIndexLongThenString(Blackhole blackhole)

runner = QueryBenchmarkUtil.makeQueryRunner(
factory,
"qIndex",
new QueryableIndexSegment("qIndex", queryableIndexes.get(0))
Q_INDEX_SEGMENT_ID,
new QueryableIndexSegment(queryableIndexes.get(0), Q_INDEX_SEGMENT_ID)
);

results = GroupByTypeInterfaceBenchmark.runQuery(factory, runner, stringQuery);
Expand All @@ -595,8 +598,8 @@ public void querySingleQueryableIndexLongThenFloat(Blackhole blackhole)
{
QueryRunner<Row> runner = QueryBenchmarkUtil.makeQueryRunner(
factory,
"qIndex",
new QueryableIndexSegment("qIndex", queryableIndexes.get(0))
Q_INDEX_SEGMENT_ID,
new QueryableIndexSegment(queryableIndexes.get(0), Q_INDEX_SEGMENT_ID)
);

List<Row> results = GroupByTypeInterfaceBenchmark.runQuery(factory, runner, longQuery);
Expand All @@ -607,8 +610,8 @@ public void querySingleQueryableIndexLongThenFloat(Blackhole blackhole)

runner = QueryBenchmarkUtil.makeQueryRunner(
factory,
"qIndex",
new QueryableIndexSegment("qIndex", queryableIndexes.get(0))
Q_INDEX_SEGMENT_ID,
new QueryableIndexSegment(queryableIndexes.get(0), Q_INDEX_SEGMENT_ID)
);

results = GroupByTypeInterfaceBenchmark.runQuery(factory, runner, floatQuery);
Expand All @@ -625,8 +628,8 @@ public void querySingleQueryableIndexStringThenNumeric(Blackhole blackhole)
{
QueryRunner<Row> runner = QueryBenchmarkUtil.makeQueryRunner(
factory,
"qIndex",
new QueryableIndexSegment("qIndex", queryableIndexes.get(0))
Q_INDEX_SEGMENT_ID,
new QueryableIndexSegment(queryableIndexes.get(0), Q_INDEX_SEGMENT_ID)
);

List<Row> results = GroupByTypeInterfaceBenchmark.runQuery(factory, runner, stringQuery);
Expand All @@ -637,8 +640,8 @@ public void querySingleQueryableIndexStringThenNumeric(Blackhole blackhole)

runner = QueryBenchmarkUtil.makeQueryRunner(
factory,
"qIndex",
new QueryableIndexSegment("qIndex", queryableIndexes.get(0))
Q_INDEX_SEGMENT_ID,
new QueryableIndexSegment(queryableIndexes.get(0), Q_INDEX_SEGMENT_ID)
);

results = GroupByTypeInterfaceBenchmark.runQuery(factory, runner, longFloatQuery);
Expand All @@ -655,8 +658,8 @@ public void querySingleQueryableIndexStringThenLong(Blackhole blackhole)
{
QueryRunner<Row> runner = QueryBenchmarkUtil.makeQueryRunner(
factory,
"qIndex",
new QueryableIndexSegment("qIndex", queryableIndexes.get(0))
Q_INDEX_SEGMENT_ID,
new QueryableIndexSegment(queryableIndexes.get(0), Q_INDEX_SEGMENT_ID)
);

List<Row> results = GroupByTypeInterfaceBenchmark.runQuery(factory, runner, stringQuery);
Expand All @@ -667,8 +670,8 @@ public void querySingleQueryableIndexStringThenLong(Blackhole blackhole)

runner = QueryBenchmarkUtil.makeQueryRunner(
factory,
"qIndex",
new QueryableIndexSegment("qIndex", queryableIndexes.get(0))
Q_INDEX_SEGMENT_ID,
new QueryableIndexSegment(queryableIndexes.get(0), Q_INDEX_SEGMENT_ID)
);

results = GroupByTypeInterfaceBenchmark.runQuery(factory, runner, longQuery);
Expand All @@ -685,8 +688,8 @@ public void querySingleQueryableIndexStringTwice(Blackhole blackhole)
{
QueryRunner<Row> runner = QueryBenchmarkUtil.makeQueryRunner(
factory,
"qIndex",
new QueryableIndexSegment("qIndex", queryableIndexes.get(0))
Q_INDEX_SEGMENT_ID,
new QueryableIndexSegment(queryableIndexes.get(0), Q_INDEX_SEGMENT_ID)
);

List<Row> results = GroupByTypeInterfaceBenchmark.runQuery(factory, runner, stringQuery);
Expand All @@ -697,8 +700,8 @@ public void querySingleQueryableIndexStringTwice(Blackhole blackhole)

runner = QueryBenchmarkUtil.makeQueryRunner(
factory,
"qIndex",
new QueryableIndexSegment("qIndex", queryableIndexes.get(0))
Q_INDEX_SEGMENT_ID,
new QueryableIndexSegment(queryableIndexes.get(0), Q_INDEX_SEGMENT_ID)
);

results = GroupByTypeInterfaceBenchmark.runQuery(factory, runner, stringQuery);
Expand All @@ -715,8 +718,8 @@ public void querySingleQueryableIndexLongTwice(Blackhole blackhole)
{
QueryRunner<Row> runner = QueryBenchmarkUtil.makeQueryRunner(
factory,
"qIndex",
new QueryableIndexSegment("qIndex", queryableIndexes.get(0))
Q_INDEX_SEGMENT_ID,
new QueryableIndexSegment(queryableIndexes.get(0), Q_INDEX_SEGMENT_ID)
);

List<Row> results = GroupByTypeInterfaceBenchmark.runQuery(factory, runner, longQuery);
Expand All @@ -727,8 +730,8 @@ public void querySingleQueryableIndexLongTwice(Blackhole blackhole)

runner = QueryBenchmarkUtil.makeQueryRunner(
factory,
"qIndex",
new QueryableIndexSegment("qIndex", queryableIndexes.get(0))
Q_INDEX_SEGMENT_ID,
new QueryableIndexSegment(queryableIndexes.get(0), Q_INDEX_SEGMENT_ID)
);

results = GroupByTypeInterfaceBenchmark.runQuery(factory, runner, longQuery);
Expand All @@ -746,8 +749,8 @@ public void querySingleQueryableIndexFloatTwice(Blackhole blackhole)
{
QueryRunner<Row> runner = QueryBenchmarkUtil.makeQueryRunner(
factory,
"qIndex",
new QueryableIndexSegment("qIndex", queryableIndexes.get(0))
Q_INDEX_SEGMENT_ID,
new QueryableIndexSegment(queryableIndexes.get(0), Q_INDEX_SEGMENT_ID)
);

List<Row> results = GroupByTypeInterfaceBenchmark.runQuery(factory, runner, floatQuery);
Expand All @@ -758,8 +761,8 @@ public void querySingleQueryableIndexFloatTwice(Blackhole blackhole)

runner = QueryBenchmarkUtil.makeQueryRunner(
factory,
"qIndex",
new QueryableIndexSegment("qIndex", queryableIndexes.get(0))
Q_INDEX_SEGMENT_ID,
new QueryableIndexSegment(queryableIndexes.get(0), Q_INDEX_SEGMENT_ID)
);

results = GroupByTypeInterfaceBenchmark.runQuery(factory, runner, floatQuery);
Expand All @@ -776,8 +779,8 @@ public void querySingleQueryableIndexFloatThenLong(Blackhole blackhole)
{
QueryRunner<Row> runner = QueryBenchmarkUtil.makeQueryRunner(
factory,
"qIndex",
new QueryableIndexSegment("qIndex", queryableIndexes.get(0))
Q_INDEX_SEGMENT_ID,
new QueryableIndexSegment(queryableIndexes.get(0), Q_INDEX_SEGMENT_ID)
);

List<Row> results = GroupByTypeInterfaceBenchmark.runQuery(factory, runner, floatQuery);
Expand All @@ -788,8 +791,8 @@ public void querySingleQueryableIndexFloatThenLong(Blackhole blackhole)

runner = QueryBenchmarkUtil.makeQueryRunner(
factory,
"qIndex",
new QueryableIndexSegment("qIndex", queryableIndexes.get(0))
Q_INDEX_SEGMENT_ID,
new QueryableIndexSegment(queryableIndexes.get(0), Q_INDEX_SEGMENT_ID)
);

results = GroupByTypeInterfaceBenchmark.runQuery(factory, runner, longQuery);
Expand All @@ -806,8 +809,8 @@ public void querySingleQueryableIndexFloatThenString(Blackhole blackhole)
{
QueryRunner<Row> runner = QueryBenchmarkUtil.makeQueryRunner(
factory,
"qIndex",
new QueryableIndexSegment("qIndex", queryableIndexes.get(0))
Q_INDEX_SEGMENT_ID,
new QueryableIndexSegment(queryableIndexes.get(0), Q_INDEX_SEGMENT_ID)
);

List<Row> results = GroupByTypeInterfaceBenchmark.runQuery(factory, runner, floatQuery);
Expand All @@ -818,8 +821,8 @@ public void querySingleQueryableIndexFloatThenString(Blackhole blackhole)

runner = QueryBenchmarkUtil.makeQueryRunner(
factory,
"qIndex",
new QueryableIndexSegment("qIndex", queryableIndexes.get(0))
Q_INDEX_SEGMENT_ID,
new QueryableIndexSegment(queryableIndexes.get(0), Q_INDEX_SEGMENT_ID)
);

results = GroupByTypeInterfaceBenchmark.runQuery(factory, runner, stringQuery);
Expand Down
Loading