New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Move-away "uniqCombined" as a separate aggregated function #3406
Merged
alexey-milovidov
merged 20 commits into
ClickHouse:master
from
abyss7:3958/many_templates
Nov 22, 2018
Merged
Changes from 4 commits
Commits
Show all changes
20 commits
Select commit
Hold shift + click to select a range
6d783ef
Move-away "uniqCombined" as a separate aggregated function with HLL p…
abyss7 e16081d
Fix initialization of precision.
abyss7 d588120
Keep default value for HLL precision in tests - to not break them.
abyss7 45e6fd8
Fix compilation with gcc and code style
abyss7 28ea773
Use a single "place" for a default precision constant.
abyss7 19ed7b6
Minor optimizations
abyss7 2a53716
Move the precision template argument one level up.
abyss7 c8868cf
Update AggregateFunctionUniqCombined.cpp
alexey-milovidov f90783d
Update AggregateFunctionUniqCombined.cpp
alexey-milovidov bd148bb
Update AggregateFunctionUniqCombined.cpp
alexey-milovidov fd86a6b
Update AggregateFunctionUniqCombined.cpp
alexey-milovidov 9296ef5
Temporarily disable precalculated bias estimations for precisions oth…
abyss7 69ebaae
Merge branch '3958/many_templates' of github.com:abyss7/ClickHouse in…
abyss7 aa6a69b
Add TODO comment
abyss7 68138a7
Add tests for different HLL precisions
abyss7 0592081
Reduce maximum allowable size of |HashSet|.
abyss7 a78984b
Update reference.md
alexey-milovidov 9ce56c5
Update reference.md
alexey-milovidov 709b117
Update reference.md
alexey-milovidov 1002ed7
Merge branch 'master' into 3958/many_templates
alexey-milovidov File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
90 changes: 90 additions & 0 deletions
90
dbms/src/AggregateFunctions/AggregateFunctionUniqCombined.cpp
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,90 @@ | ||
#include <AggregateFunctions/AggregateFunctionUniqCombined.h> | ||
|
||
#include <AggregateFunctions/AggregateFunctionFactory.h> | ||
#include <AggregateFunctions/Helpers.h> | ||
|
||
#include <DataTypes/DataTypeDate.h> | ||
#include <DataTypes/DataTypeDateTime.h> | ||
|
||
namespace DB | ||
{ | ||
|
||
namespace ErrorCodes | ||
{ | ||
extern const int ILLEGAL_TYPE_OF_ARGUMENT; | ||
extern const int NUMBER_OF_ARGUMENTS_DOESNT_MATCH; | ||
extern const int ARGUMENT_OUT_OF_BOUND; | ||
} | ||
|
||
namespace | ||
{ | ||
|
||
AggregateFunctionPtr createAggregateFunctionUniqCombined( | ||
const std::string & name, const DataTypes & argument_types, const Array & params) | ||
{ | ||
UInt8 precision = 17; /// default value - must correlate with default ctor of |AggregateFunctionUniqCombinedData| | ||
|
||
if (!params.empty()) | ||
{ | ||
if (params.size() != 1) | ||
throw Exception( | ||
"Aggregate function " + name + " requires one parameter or less.", ErrorCodes::NUMBER_OF_ARGUMENTS_DOESNT_MATCH); | ||
|
||
UInt64 precision_param = applyVisitor(FieldVisitorConvertToNumber<UInt64>(), params[0]); | ||
|
||
// This range is hardcoded into |AggregateFunctionUniqCombinedData| | ||
if (precision_param > 20 || precision_param < 12) | ||
throw Exception( | ||
"Parameter for aggregate function " + name + "is out or range: [12, 20].", ErrorCodes::ARGUMENT_OUT_OF_BOUND); | ||
|
||
precision = precision_param; | ||
} | ||
|
||
if (argument_types.empty()) | ||
throw Exception("Incorrect number of arguments for aggregate function " + name, ErrorCodes::NUMBER_OF_ARGUMENTS_DOESNT_MATCH); | ||
|
||
/// We use exact hash function if the user wants it; | ||
/// or if the arguments are not contiguous in memory, because only exact hash function have support for this case. | ||
bool use_exact_hash_function = !isAllArgumentsContiguousInMemory(argument_types); | ||
|
||
if (argument_types.size() == 1) | ||
{ | ||
const IDataType & argument_type = *argument_types[0]; | ||
|
||
AggregateFunctionPtr res(createWithNumericType<AggregateFunctionUniqCombined>(*argument_types[0], precision)); | ||
|
||
WhichDataType which(argument_type); | ||
if (res) | ||
return res; | ||
else if (which.isDate()) | ||
return std::make_shared<AggregateFunctionUniqCombined<DataTypeDate::FieldType>>(precision); | ||
else if (which.isDateTime()) | ||
return std::make_shared<AggregateFunctionUniqCombined<DataTypeDateTime::FieldType>>(precision); | ||
else if (which.isStringOrFixedString()) | ||
return std::make_shared<AggregateFunctionUniqCombined<String>>(precision); | ||
else if (which.isUUID()) | ||
return std::make_shared<AggregateFunctionUniqCombined<DataTypeUUID::FieldType>>(precision); | ||
else if (which.isTuple()) | ||
{ | ||
if (use_exact_hash_function) | ||
return std::make_shared<AggregateFunctionUniqCombinedVariadic<true, true>>(argument_types, precision); | ||
else | ||
return std::make_shared<AggregateFunctionUniqCombinedVariadic<false, true>>(argument_types, precision); | ||
} | ||
} | ||
|
||
/// "Variadic" method also works as a fallback generic case for single argument. | ||
if (use_exact_hash_function) | ||
return std::make_shared<AggregateFunctionUniqCombinedVariadic<true, false>>(argument_types, precision); | ||
else | ||
return std::make_shared<AggregateFunctionUniqCombinedVariadic<false, false>>(argument_types, precision); | ||
} | ||
|
||
} // namespace | ||
|
||
void registerAggregateFunctionUniqCombined(AggregateFunctionFactory & factory) | ||
{ | ||
factory.registerFunction("uniqCombined", createAggregateFunctionUniqCombined); | ||
} | ||
|
||
} // namespace DB |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If it MUST be correlated it's better to set named constant and use it both here and in ctor.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It correlates with a hardcoded union member initialization, the constant won't help - so I left the comment. The code now looks like:
AggregateFunctionUniqCombinedDataWithKey() : set_17() {}
- Also I can use a macro-definition as constant, if it's better.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My point is there should be only one place there magic number 17 set.