-
Notifications
You must be signed in to change notification settings - Fork 418
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[GLUTEN-4350][CH] Improve NativeReader
's performance for small blocks
#4369
Conversation
Run Gluten Clickhouse CI |
0e4a5bb
to
f122f61
Compare
Run Gluten Clickhouse CI |
f122f61
to
2f494f3
Compare
Run Gluten Clickhouse CI |
Notice that when parse aggregate data, there are a lot of virtual function call to initialize an aggregate data place. Reduce these virtual function call should bring some improvement. Run a test on this. static void readFixedSizeAggregateData(DB::ReadBuffer &in, DB::ColumnPtr & column, size_t rows, NativeReader::ColumnParseUtil & column_parse_util)
{
ColumnAggregateFunction & real_column = typeid_cast<ColumnAggregateFunction &>(*column->assumeMutable());
auto & arena = real_column.createOrGetArena();
ColumnAggregateFunction::Container & vec = real_column.getData();
size_t initial_size = vec.size();
vec.reserve(initial_size + rows);
#if 0
for (size_t i = 0; i < rows; ++i)
{
AggregateDataPtr place = arena.alignedAlloc(column_parse_util.aggregate_state_size, column_parse_util.aggregate_state_align);
column_parse_util.aggregate_function->create(place);
auto n = in.read(place, column_parse_util.aggregate_state_size);
chassert(n == column_parse_util.aggregate_state_size);
vec.push_back(place);
}
#else
AggregateDataPtr places = arena.alignedAlloc(column_parse_util.aggregate_state_align_size * rows, column_parse_util.aggregate_state_align);
column_parse_util.aggregate_function->createPlaces(places, rows, column_parse_util.aggregate_state_align_size);
for (size_t i = 0; i < rows; ++i)
{
auto n = in.read(places, column_parse_util.aggregate_state_size);
chassert(n == column_parse_util.aggregate_state_size);
places += column_parse_util.aggregate_state_align_size;
vec.push_back(places);
}
#endif
} if (!column_parse_util.aggregate_state_align)
column_parse_util.aggregate_state_align_size = column_parse_util.aggregate_state_size;
else
{
auto n = column_parse_util.aggregate_state_size % column_parse_util.aggregate_state_align;
if (!n)
{
column_parse_util.aggregate_state_align_size = column_parse_util.aggregate_state_size + (column_parse_util.aggregate_state_align - n);
}
else
column_parse_util.aggregate_state_align_size = column_parse_util.aggregate_state_size;
} But need to modify |
2f494f3
to
260649a
Compare
Run Gluten Clickhouse CI |
260649a
to
78995ff
Compare
Run Gluten Clickhouse CI |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
What changes were proposed in this pull request?
(Please fill in changes proposed in this fix)
Fixes: #4350
After this modification.
How was this patch tested?
(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
unit tests
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)