Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data Skipping Indices fix #4286

Merged
merged 181 commits into from Feb 9, 2019
Merged
Changes from all commits
Commits
Show all changes
181 commits
Select commit Hold shift + click to select a range
6f98649
made index parser
nikvas0 Dec 25, 2018
fcd49af
added index parsing
nikvas0 Dec 25, 2018
36083e1
some fixes
nikvas0 Dec 25, 2018
04a8ea8
added index interface and factory
nikvas0 Dec 26, 2018
b62197b
fixed compilation
nikvas0 Dec 26, 2018
c89df91
ptrs
nikvas0 Dec 26, 2018
9818d27
Merge pull request #1 from nikvas0/nikvas0/index_creation
nikvas0 Dec 26, 2018
9bf5b6b
added indexParts
nikvas0 Dec 28, 2018
06d8416
indextypes
nikvas0 Dec 28, 2018
68c3879
index condition
nikvas0 Dec 29, 2018
5079330
IndexCondition
nikvas0 Dec 29, 2018
f90cdca
added indexes in selectexecutor
nikvas0 Dec 29, 2018
33cf4c9
fix
nikvas0 Dec 29, 2018
ad2a453
changed comment
nikvas0 Dec 29, 2018
1b7c0ae
fix
nikvas0 Jan 2, 2019
f704a32
added granularity
nikvas0 Jan 2, 2019
b2da3a3
comments
nikvas0 Jan 2, 2019
69052b3
fix
nikvas0 Jan 3, 2019
35dbb94
fix
nikvas0 Jan 3, 2019
92a850c
added writing indexes
nikvas0 Jan 4, 2019
f927502
removed indexpart class
nikvas0 Jan 4, 2019
1c80628
fix
nikvas0 Jan 5, 2019
82cc39d
added setSkipIndexes
nikvas0 Jan 5, 2019
61b9c77
add rw for MergeTreeIndexes
nikvas0 Jan 5, 2019
c3f1784
fixes
nikvas0 Jan 5, 2019
83368a4
upd error
nikvas0 Jan 6, 2019
7e0e301
fix
nikvas0 Jan 6, 2019
f345941
Merge pull request #2 from nikvas0/nikvas0/index_stream
nikvas0 Jan 6, 2019
e95376e
Merge branch 'master' into nikvas0/index
nikvas0 Jan 6, 2019
17f6618
fix
nikvas0 Jan 6, 2019
11d37ef
reading
nikvas0 Jan 7, 2019
6871665
test index
nikvas0 Jan 7, 2019
69c6e77
fixed nullptr error
nikvas0 Jan 7, 2019
76c25c2
fixed
nikvas0 Jan 7, 2019
3cad26d
Merge pull request #3 from nikvas0/nikvas0/index_read
nikvas0 Jan 7, 2019
541c641
fix
nikvas0 Jan 7, 2019
48e136f
unique names
nikvas0 Jan 8, 2019
d1e2ab8
asts -> exprlist
nikvas0 Jan 8, 2019
15ee9e1
Merge pull request #4 from nikvas0/nikvas0/fix-double-index
nikvas0 Jan 8, 2019
49148ac
minmax index
nikvas0 Jan 8, 2019
ad4df16
fix
nikvas0 Jan 8, 2019
6eeed48
fixed select
nikvas0 Jan 8, 2019
e580180
fixed merging
nikvas0 Jan 9, 2019
91fb17f
fixed mutation
nikvas0 Jan 9, 2019
1e8fa5d
working minmax
nikvas0 Jan 9, 2019
d8f8b63
removed test index
nikvas0 Jan 9, 2019
0536711
Merge pull request #5 from nikvas0/nikvas0/minmax
nikvas0 Jan 9, 2019
10355ae
Merge remote-tracking branch 'upstream/master' into nikvas0/index
nikvas0 Jan 9, 2019
0ba6f14
fixed style
nikvas0 Jan 9, 2019
d9b7f30
added indexes to checkDataPart
nikvas0 Jan 9, 2019
6490c23
added tests for minmax index
nikvas0 Jan 10, 2019
3f8c2ad
fixed constructor
nikvas0 Jan 10, 2019
634d5eb
fix style
nikvas0 Jan 10, 2019
701627e
fixed includes
nikvas0 Jan 10, 2019
0c18e73
Merge remote-tracking branch 'upstream/master' into nikvas0/index
nikvas0 Jan 10, 2019
d310d1a
fixed setSkipIndexes
nikvas0 Jan 10, 2019
89b831e
added indexes meta to zookeeper
nikvas0 Jan 10, 2019
4d4af73
Merge pull request #6 from nikvas0/nikvas0/index_replication
nikvas0 Jan 10, 2019
e288022
Merge remote-tracking branch 'upstream/master' into nikvas0/index
nikvas0 Jan 13, 2019
183ddea
added parsing
nikvas0 Jan 13, 2019
4f48ebd
removed throw
nikvas0 Jan 13, 2019
d0db87b
alter cmds parse
nikvas0 Jan 13, 2019
e197697
fix
nikvas0 Jan 13, 2019
026a300
added alter
nikvas0 Jan 15, 2019
2f07e31
fix
nikvas0 Jan 15, 2019
b110738
alters fix
nikvas0 Jan 15, 2019
8c35ccd
fix alters
nikvas0 Jan 16, 2019
65539bf
fix "after"
nikvas0 Jan 16, 2019
f937048
Merge pull request #7 from nikvas0/nikvas0/index_alter
nikvas0 Jan 16, 2019
2de2b6f
fixed alter
nikvas0 Jan 16, 2019
b4ce03e
alter fix + test
nikvas0 Jan 16, 2019
08d23c1
fixes
nikvas0 Jan 16, 2019
579832f
upd setSkipIndexes
nikvas0 Jan 16, 2019
62dde8c
fixed alter bug with drop all indices
nikvas0 Jan 16, 2019
f8fa36c
fix metadata editing
nikvas0 Jan 16, 2019
062819f
new test and repl fix
nikvas0 Jan 17, 2019
59cb1cb
rm test files
nikvas0 Jan 17, 2019
d514d37
fixed repl alter
nikvas0 Jan 17, 2019
2fbb706
Merge pull request #8 from nikvas0/nikvas0/fix_repl_alter
nikvas0 Jan 17, 2019
6a09340
Merge remote-tracking branch 'upstream/master' into nikvas0/index
nikvas0 Jan 17, 2019
8d47872
fix
nikvas0 Jan 17, 2019
6b8c7da
fix
nikvas0 Jan 17, 2019
c4a725a
indices
nikvas0 Jan 17, 2019
8c2a23a
MTReadStream
nikvas0 Jan 17, 2019
955b002
upd test for bug
nikvas0 Jan 17, 2019
0e6f5b4
fix
nikvas0 Jan 17, 2019
9975286
Merge pull request #9 from nikvas0/nikvas0/fix_duplicate_idx
nikvas0 Jan 17, 2019
4a19523
Merge remote-tracking branch 'upstream/master' into nikvas0/index
nikvas0 Jan 20, 2019
4bf0ac8
added useful parsers and ast classes
nikvas0 Jan 20, 2019
8152bc4
fix
nikvas0 Jan 20, 2019
bc0e4fa
fix comments
nikvas0 Jan 20, 2019
13a4fb2
replaced columns
nikvas0 Jan 20, 2019
5e626be
fix
nikvas0 Jan 20, 2019
cf377b9
fixed parsing
nikvas0 Jan 20, 2019
9120beb
fixed printing
nikvas0 Jan 20, 2019
d654517
fix err
nikvas0 Jan 20, 2019
093b94a
basic IndicesDescription
nikvas0 Jan 20, 2019
3f5447f
go to IndicesDescr
nikvas0 Jan 20, 2019
977b1e2
moved indices
nikvas0 Jan 20, 2019
c32d198
go to indicesDescr
nikvas0 Jan 21, 2019
9f1a093
fix test minmax_index*
nikvas0 Jan 21, 2019
506f2c4
fixed MT alter
nikvas0 Jan 21, 2019
f0a0dbb
fixed bug with replMT indices storing in zk
nikvas0 Jan 21, 2019
03cf70e
Merge remote-tracking branch 'upstream/master' into nikvas0/index
nikvas0 Jan 21, 2019
2a76e86
Merge branch 'nikvas0/index' into nikvas0/parse_index
nikvas0 Jan 21, 2019
08adef7
rename
nikvas0 Jan 21, 2019
5183022
refactoring
nikvas0 Jan 21, 2019
b220682
Merge pull request #10 from nikvas0/nikvas0/parse_index
nikvas0 Jan 21, 2019
0c601ec
docs ru
nikvas0 Jan 21, 2019
2d06486
docs ru
nikvas0 Jan 22, 2019
5ed3269
docs en
nikvas0 Jan 22, 2019
9ee50da
refactor
nikvas0 Jan 22, 2019
2c16614
rename tests
nikvas0 Jan 22, 2019
d47cd48
Merge remote-tracking branch 'upstream/master' into nikvas0/index
nikvas0 Jan 22, 2019
71e61d8
fix docs
nikvas0 Jan 22, 2019
d8f3bf7
refactoring
nikvas0 Jan 22, 2019
7996c6a
Merge pull request #11 from nikvas0/nikvas0/index_docs
nikvas0 Jan 22, 2019
296ad91
fix
nikvas0 Jan 23, 2019
276dc00
fix
nikvas0 Jan 24, 2019
84c7307
fix
nikvas0 Jan 24, 2019
0ea0b17
fixed style
nikvas0 Jan 26, 2019
b2915bc
Merge remote-tracking branch 'upstream/master' into nikvas0/index
nikvas0 Jan 26, 2019
6b6b0aa
unique idx
nikvas0 Jan 26, 2019
387ed1b
unique
nikvas0 Jan 26, 2019
077212f
fix
nikvas0 Jan 26, 2019
b94915b
better minmax calculation
nikvas0 Jan 26, 2019
0ea031d
Merge branch 'nikvas0/index' into nikvas0/unique_index
nikvas0 Jan 26, 2019
537b23e
upd
nikvas0 Jan 27, 2019
c701cb7
added getBlock
nikvas0 Jan 27, 2019
902c9a9
unique_condition
nikvas0 Jan 27, 2019
aa37d95
added termForAST
nikvas0 Jan 27, 2019
00ec4cc
unique
nikvas0 Jan 28, 2019
a74ed80
fixed not
nikvas0 Jan 28, 2019
b4d1bf8
uniqueCondition::mayBeTrueOnGranule
nikvas0 Jan 28, 2019
7c036e9
fix
nikvas0 Jan 28, 2019
149b0d8
fixed bug with double column
nikvas0 Jan 29, 2019
67af986
is always true
nikvas0 Jan 29, 2019
68ea3f0
fix
nikvas0 Jan 29, 2019
4ec0923
key set
nikvas0 Jan 29, 2019
9311c01
spaces
nikvas0 Jan 29, 2019
514987e
test
nikvas0 Jan 29, 2019
d3b430d
tests
nikvas0 Jan 29, 2019
c4dad05
fix
nikvas0 Jan 29, 2019
4de473a
unique
nikvas0 Jan 29, 2019
371e165
fix
nikvas0 Jan 29, 2019
f6a6a44
Merge pull request #12 from nikvas0/nikvas0/unique_index
nikvas0 Jan 29, 2019
c12b03d
Merge remote-tracking branch 'upstream/master' into nikvas0/index
nikvas0 Jan 29, 2019
6d7ccc6
fix
nikvas0 Jan 29, 2019
c0d7a8b
Merge branch 'nikvas0/index' of github.com:nikvas0/ClickHouse into ni…
nikvas0 Jan 30, 2019
160c8c0
fixed bug with duplicate column
nikvas0 Jan 30, 2019
69daa33
removed unused data
nikvas0 Jan 30, 2019
bcd07a4
fix
nikvas0 Jan 30, 2019
0492ed7
fixes
nikvas0 Jan 30, 2019
476f33f
__bitSwapLastTwo
nikvas0 Jan 30, 2019
094ae0f
fix
nikvas0 Jan 30, 2019
fa07d1c
replace andBit
nikvas0 Feb 4, 2019
5a7ca35
unknown field
nikvas0 Feb 4, 2019
87a80a4
fixed create as
nikvas0 Feb 5, 2019
263b390
removed unnecessary actions
nikvas0 Feb 5, 2019
89913b9
fix
nikvas0 Feb 5, 2019
3bea035
fixed bug in unique_index
nikvas0 Feb 5, 2019
9921fc6
removed toString
nikvas0 Feb 5, 2019
9eaabcb
Merge remote-tracking branch 'upstream/master' into nikvas0/index_fix
nikvas0 Feb 5, 2019
ef154a4
r fix
nikvas0 Feb 6, 2019
b613d78
fix docs
nikvas0 Feb 6, 2019
258367c
ASTs
nikvas0 Feb 6, 2019
c3f7344
style
nikvas0 Feb 6, 2019
e3879c0
fix
nikvas0 Feb 6, 2019
fa1e46f
granularity
nikvas0 Feb 6, 2019
28870b4
fixed indices register
nikvas0 Feb 6, 2019
76a35b4
moved tests
nikvas0 Feb 6, 2019
abcfb53
fixed minmax tests
nikvas0 Feb 6, 2019
84261a2
fix test
nikvas0 Feb 6, 2019
a618c7f
fixed unique
nikvas0 Feb 6, 2019
6ef6536
Merge remote-tracking branch 'upstream/master' into nikvas0/index_fix
nikvas0 Feb 6, 2019
a675ba2
unique -> set
nikvas0 Feb 6, 2019
c6f0dd4
unique -> set : docs
nikvas0 Feb 6, 2019
a3e47ed
fix test
nikvas0 Feb 6, 2019
f0272b7
:fix set test
nikvas0 Feb 7, 2019
a904f37
upd minmax test
nikvas0 Feb 7, 2019
cecbd5d
Merge remote-tracking branch 'upstream/master' into nikvas0/index_fix
nikvas0 Feb 7, 2019
File filter...
Filter file types
Jump to…
Jump to file or symbol
Failed to load files and symbols.

Always

Just for now

@@ -460,14 +460,21 @@ ColumnsDescription InterpreterCreateQuery::setColumns(
ASTCreateQuery & create, const Block & as_select_sample, const StoragePtr & as_storage) const
{
ColumnsDescription res;
IndicesDescription indices;

if (create.columns_list && create.columns_list->columns)
if (create.columns_list)
{
res = getColumnsDescription(*create.columns_list->columns, context);
if (create.columns_list->columns)
res = getColumnsDescription(*create.columns_list->columns, context);
if (create.columns_list->indices)
for (const auto & index : create.columns_list->indices->children)
indices.indices.push_back(
std::dynamic_pointer_cast<ASTIndexDeclaration>(index->clone()));
}
else if (!create.as_table.empty())
{
res = as_storage->getColumns();
indices = as_storage->getIndicesDescription();
}
else if (create.select)
{
@@ -479,6 +486,8 @@ ColumnsDescription InterpreterCreateQuery::setColumns(

/// Even if query has list of columns, canonicalize it (unfold Nested columns).
ASTPtr new_columns = formatColumns(res);
ASTPtr new_indices = formatIndices(indices);

if (!create.columns_list)
{
auto new_columns_list = std::make_shared<ASTColumns>();
@@ -490,6 +499,11 @@ ColumnsDescription InterpreterCreateQuery::setColumns(
else
create.columns_list->set(create.columns_list->columns, new_columns);

if (new_indices && create.columns_list->indices)
create.columns_list->replace(create.columns_list->indices, new_indices);
else if (new_indices)
create.columns_list->set(create.columns_list->indices, new_indices);

/// Check for duplicates
std::set<String> all_columns;
auto check_column_already_exists = [&all_columns](const NameAndTypePair & column_name_and_type)
@@ -3,6 +3,7 @@
#include <Core/Field.h>
#include <Core/Types.h>
#include <Common/FieldVisitors.h>
#include <IO/WriteHelpers.h>
#include <Parsers/ASTExpressionList.h>
#include <Parsers/ASTFunction.h>
#include <Parsers/IAST.h>
@@ -21,7 +22,7 @@ class ASTIndexDeclaration : public IAST
String name;
IAST * expr;
ASTFunction * type;
Field granularity;
UInt64 granularity;

/** Get the text that identifies this element. */
String getID(char) const override { return "Index"; }
@@ -52,7 +53,7 @@ class ASTIndexDeclaration : public IAST
s.ostr << (s.hilite ? hilite_keyword : "") << " TYPE " << (s.hilite ? hilite_none : "");
type->formatImpl(s, state, frame);
s.ostr << (s.hilite ? hilite_keyword : "") << " GRANULARITY " << (s.hilite ? hilite_none : "");
s.ostr << applyVisitor(FieldVisitorToString(), granularity);
s.ostr << toString(granularity);
}
};

@@ -127,7 +127,7 @@ bool ParserIndexDeclaration::parseImpl(Pos & pos, ASTPtr & node, Expected & expe

auto index = std::make_shared<ASTIndexDeclaration>();
index->name = typeid_cast<const ASTIdentifier &>(*name).name;
index->granularity = typeid_cast<const ASTLiteral &>(*granularity).value;
index->granularity = typeid_cast<const ASTLiteral &>(*granularity).value.get<UInt64>();
index->set(index->expr, expr);
index->set(index->type, type);
node = index;
@@ -6,11 +6,11 @@
namespace DB
{

using IndicesAsts = std::vector<std::shared_ptr<ASTIndexDeclaration>>;
using IndicesASTs = std::vector<std::shared_ptr<ASTIndexDeclaration>>;

struct IndicesDescription
{
IndicesAsts indices;
IndicesASTs indices;

IndicesDescription() = default;

@@ -1148,7 +1148,7 @@ void MergeTreeData::checkAlter(const AlterCommands & commands)
}

void MergeTreeData::createConvertExpression(const DataPartPtr & part, const NamesAndTypesList & old_columns, const NamesAndTypesList & new_columns,
const IndicesAsts & old_indices, const IndicesAsts & new_indices, ExpressionActionsPtr & out_expression,
const IndicesASTs & old_indices, const IndicesASTs & new_indices, ExpressionActionsPtr & out_expression,
NameToNameMap & out_rename_map, bool & out_force_update_metadata) const
{
out_expression = nullptr;
@@ -1308,7 +1308,7 @@ void MergeTreeData::createConvertExpression(const DataPartPtr & part, const Name
MergeTreeData::AlterDataPartTransactionPtr MergeTreeData::alterDataPart(
const DataPartPtr & part,
const NamesAndTypesList & new_columns,
const IndicesAsts & new_indices,
const IndicesASTs & new_indices,
bool skip_sanity_checks)
{
ExpressionActionsPtr expression;
@@ -490,7 +490,7 @@ class MergeTreeData : public ITableDeclaration
AlterDataPartTransactionPtr alterDataPart(
const DataPartPtr & part,
const NamesAndTypesList & new_columns,
const IndicesAsts & new_indices,
const IndicesASTs & new_indices,
bool skip_sanity_checks);

/// Freezes all parts.
@@ -746,7 +746,7 @@ class MergeTreeData : public ITableDeclaration
/// Files to be deleted are mapped to an empty string in out_rename_map.
/// If part == nullptr, just checks that all type conversions are possible.
void createConvertExpression(const DataPartPtr & part, const NamesAndTypesList & old_columns, const NamesAndTypesList & new_columns,
const IndicesAsts & old_indices, const IndicesAsts & new_indices,
const IndicesASTs & old_indices, const IndicesASTs & new_indices,
ExpressionActionsPtr & out_expression, NameToNameMap & out_rename_map, bool & out_force_update_metadata) const;

/// Calculates column sizes in compressed form for the current state of data_parts. Call with data_parts mutex locked.
@@ -520,6 +520,14 @@ BlockInputStreams MergeTreeDataSelectExecutor::readFromParts(

RangesInDataParts parts_with_ranges;

std::vector<std::pair<MergeTreeIndexPtr, IndexConditionPtr>> useful_indices;
for (const auto & index : data.skip_indices)
{
auto condition = index->createIndexCondition(query_info, context);
if (!condition->alwaysUnknownOrTrue())
useful_indices.emplace_back(index, condition);
}

/// Let's find what range to read from each part.
size_t sum_marks = 0;
size_t sum_ranges = 0;
@@ -532,16 +540,9 @@ BlockInputStreams MergeTreeDataSelectExecutor::readFromParts(
else
ranges.ranges = MarkRanges{MarkRange{0, part->marks_count}};

/// It can be done in multiple threads (one thread for each part).
/// Maybe it should be moved to BlockInputStream, but it can cause some problems.
for (const auto & index : data.skip_indices)
{
auto condition = index->createIndexCondition(query_info, context);
if (!condition->alwaysUnknownOrTrue())
{
ranges.ranges = filterMarksUsingIndex(index, condition, part, ranges.ranges, settings);
}
}
for (const auto & index_and_condition : useful_indices)
ranges.ranges = filterMarksUsingIndex(
index_and_condition.first, index_and_condition.second, part, ranges.ranges, settings);

if (!ranges.ranges.empty())
{
@@ -26,7 +26,7 @@ void MergeTreeIndexFactory::registerIndex(const std::string &name, Creator creat
ErrorCodes::LOGICAL_ERROR);
}

std::unique_ptr<MergeTreeIndex> MergeTreeIndexFactory::get(
std::unique_ptr<IMergeTreeIndex> MergeTreeIndexFactory::get(
const NamesAndTypesList & columns,
std::shared_ptr<ASTIndexDeclaration> node,
const Context & context) const
@@ -54,4 +54,21 @@ std::unique_ptr<MergeTreeIndex> MergeTreeIndexFactory::get(
return it->second(columns, node, context);
}


std::unique_ptr<IMergeTreeIndex> minmaxIndexCreator(
const NamesAndTypesList & columns,
std::shared_ptr<ASTIndexDeclaration> node,
const Context & context);

std::unique_ptr<IMergeTreeIndex> setIndexCreator(
const NamesAndTypesList & columns,
std::shared_ptr<ASTIndexDeclaration> node,
const Context & context);

MergeTreeIndexFactory::MergeTreeIndexFactory()
{
registerIndex("minmax", minmaxIndexCreator);
registerIndex("set", setIndexCreator);
}

}
@@ -18,48 +18,51 @@ namespace DB
{

class MergeTreeData;
class MergeTreeIndex;
class IMergeTreeIndex;

using MergeTreeIndexPtr = std::shared_ptr<const MergeTreeIndex>;
using MutableMergeTreeIndexPtr = std::shared_ptr<MergeTreeIndex>;
using MergeTreeIndexPtr = std::shared_ptr<const IMergeTreeIndex>;
using MutableMergeTreeIndexPtr = std::shared_ptr<IMergeTreeIndex>;


struct MergeTreeIndexGranule
/// Stores some info about a single block of data.
struct IMergeTreeIndexGranule
{
virtual ~MergeTreeIndexGranule() = default;
virtual ~IMergeTreeIndexGranule() = default;

virtual void serializeBinary(WriteBuffer & ostr) const = 0;
virtual void deserializeBinary(ReadBuffer & istr) = 0;

virtual String toString() const = 0;
virtual bool empty() const = 0;

/// Updates the stored info using rows of the specified block.
/// Reads no more than `limit` rows.
/// After finishing updating `pos` will store the position of the first row which was not read.
virtual void update(const Block & block, size_t * pos, size_t limit) = 0;
};


using MergeTreeIndexGranulePtr = std::shared_ptr<MergeTreeIndexGranule>;
using MergeTreeIndexGranulePtr = std::shared_ptr<IMergeTreeIndexGranule>;
using MergeTreeIndexGranules = std::vector<MergeTreeIndexGranulePtr>;


/// Condition on the index.
class IndexCondition
class IIndexCondition
{
public:
virtual ~IndexCondition() = default;
virtual ~IIndexCondition() = default;
/// Checks if this index is useful for query.
virtual bool alwaysUnknownOrTrue() const = 0;

virtual bool mayBeTrueOnGranule(MergeTreeIndexGranulePtr granule) const = 0;
};

using IndexConditionPtr = std::shared_ptr<IndexCondition>;
using IndexConditionPtr = std::shared_ptr<IIndexCondition>;


/// Structure for storing basic index info like columns, expression, arguments, ...
class MergeTreeIndex
class IMergeTreeIndex
{
public:
MergeTreeIndex(
IMergeTreeIndex(
String name,
ExpressionActionsPtr expr,
const Names & columns,
@@ -73,7 +76,7 @@ class MergeTreeIndex
, header(header)
, granularity(granularity) {}

virtual ~MergeTreeIndex() = default;
virtual ~IMergeTreeIndex() = default;

/// gets filename without extension
String getFileName() const { return INDEX_FILE_PREFIX + name; }
@@ -91,7 +94,6 @@ class MergeTreeIndex
size_t granularity;
};


using MergeTreeIndices = std::vector<MutableMergeTreeIndexPtr>;


@@ -101,12 +103,12 @@ class MergeTreeIndexFactory : public ext::singleton<MergeTreeIndexFactory>

public:
using Creator = std::function<
std::unique_ptr<MergeTreeIndex>(
std::unique_ptr<IMergeTreeIndex>(
const NamesAndTypesList & columns,
std::shared_ptr<ASTIndexDeclaration> node,
const Context & context)>;

std::unique_ptr<MergeTreeIndex> get(
std::unique_ptr<IMergeTreeIndex> get(
const NamesAndTypesList & columns,
std::shared_ptr<ASTIndexDeclaration> node,
const Context & context) const;
@@ -116,7 +118,7 @@ class MergeTreeIndexFactory : public ext::singleton<MergeTreeIndexFactory>
const auto & getAllIndexes() const { return indexes; }

protected:
MergeTreeIndexFactory() = default;
MergeTreeIndexFactory();

private:
using Indexes = std::unordered_map<std::string, Creator>;
@@ -17,7 +17,7 @@ namespace ErrorCodes


MergeTreeMinMaxGranule::MergeTreeMinMaxGranule(const MergeTreeMinMaxIndex & index)
: MergeTreeIndexGranule(), index(index), parallelogram()
: IMergeTreeIndexGranule(), index(index), parallelogram()
{
}

@@ -52,22 +52,13 @@ void MergeTreeMinMaxGranule::deserializeBinary(ReadBuffer & istr)
}
}

String MergeTreeMinMaxGranule::toString() const
{
String res = "";

for (size_t i = 0; i < parallelogram.size(); ++i)
{
res += "["
+ applyVisitor(FieldVisitorToString(), parallelogram[i].left) + ", "
+ applyVisitor(FieldVisitorToString(), parallelogram[i].right) + "]";
}

return res;
}

void MergeTreeMinMaxGranule::update(const Block & block, size_t * pos, size_t limit)
{
if (*pos >= block.rows())
throw Exception(
"The provided position is not less than the number of block rows. Position: "
+ toString(*pos) + ", Block rows: " + toString(block.rows()) + ".", ErrorCodes::LOGICAL_ERROR);

size_t rows_read = std::min(limit, block.rows() - *pos);

for (size_t i = 0; i < index.columns.size(); ++i)
@@ -96,7 +87,7 @@ MinMaxCondition::MinMaxCondition(
const SelectQueryInfo &query,
const Context &context,
const MergeTreeMinMaxIndex &index)
: IndexCondition(), index(index), condition(query, context, index.columns, index.expr) {}
: IIndexCondition(), index(index), condition(query, context, index.columns, index.expr) {}

bool MinMaxCondition::alwaysUnknownOrTrue() const
{
@@ -109,7 +100,7 @@ bool MinMaxCondition::mayBeTrueOnGranule(MergeTreeIndexGranulePtr idx_granule) c
= std::dynamic_pointer_cast<MergeTreeMinMaxGranule>(idx_granule);
if (!granule)
throw Exception(
"Minmax index condition got wrong granule", ErrorCodes::LOGICAL_ERROR);
"Minmax index condition got a granule with the wrong type.", ErrorCodes::LOGICAL_ERROR);

return condition.mayBeTrueInParallelogram(granule->parallelogram, index.data_types);
}
@@ -123,11 +114,11 @@ MergeTreeIndexGranulePtr MergeTreeMinMaxIndex::createIndexGranule() const
IndexConditionPtr MergeTreeMinMaxIndex::createIndexCondition(
const SelectQueryInfo & query, const Context & context) const
{
return std::make_shared<MinMaxCondition>(query, context, *this);
return std::make_shared<MinMaxCondition>(query, context, *this);
};


std::unique_ptr<MergeTreeIndex> MergeTreeMinMaxIndexCreator(
std::unique_ptr<IMergeTreeIndex> minmaxIndexCreator(
const NamesAndTypesList & new_columns,
std::shared_ptr<ASTIndexDeclaration> node,
const Context & context)
@@ -158,7 +149,7 @@ std::unique_ptr<MergeTreeIndex> MergeTreeMinMaxIndexCreator(
}

return std::make_unique<MergeTreeMinMaxIndex>(
node->name, std::move(minmax_expr), columns, data_types, sample, node->granularity.get<size_t>());
node->name, std::move(minmax_expr), columns, data_types, sample, node->granularity);
}

}
ProTip! Use n and p to navigate between commits in a pull request.
You can’t perform that action at this time.