Optimize unnecessary column copy for HashAgg

[guo-shaoge]

What problem does this PR solve?

Issue Number: close #8891

Problem Summary:
When there are group by key in select item(a.k.a. first_row), tiflash have extra agg func, which cause unnecessary copy from HashMap to result column.

What is changed and how it works?

Basic idea:

Optimization-1 (with collation):
What: For group by keys that with collation, there will be first_row/any agg func for it to keep original data. So no need to copy these keys from HashMap, instead just a pointer to reference its corresponding first_row/any result is enough.

How:

1. Detect if there is a first_row agg func in the select item.
2. If so, ignore any agg func. If not, add the 'any' agg func.
3. Also, set key_from_agg_func to indicate that this key is equivalent to first_row/any agg func, which can avoid copying this key from the HashMap in subsequent operations. (check DAGExpressionAnalyzer::buildAggGroupBy)
4. If all keys are included in first_row/any (which is rare, but still can happens), will skip copy keys(template argument skip_serialize_key is true)

Optimization-2(no collation)
What: When SQL query has agg func like: first_row(group_by_key_col), and that group_by_key_col has no collation. Then we can eliminate the first_row agg func, just use a pointer to reference the group by key is enough.

Results

1. 25% improvement
2. workload:
   1. 20M rows, very high NDV.
   2. 3 varchar columns, 3 decimal columns, 3 int columns (that means Aggregator will use HashMethodSerialized)

before:

after:

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No code

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

None

[guo-shaoge]

/run-all-tests

[guo-shaoge]

/test all

[guo-shaoge]

/test all

[guo-shaoge]

/test all

[guo-shaoge]

/test all

[windtalker]

LGTM

[SeaRise]

lgtm

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: SeaRise, windtalker

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [SeaRise,windtalker]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ti-chi-bot]

[LGTM Timeline notifier]

Timeline:

• 2024-05-28 03:00:13.985921633 +0000 UTC m=+2745367.743057204: ☑️ agreed by windtalker.
• 2024-05-29 01:58:46.211107092 +0000 UTC m=+2828079.968242665: ☑️ agreed by SeaRise.

[SeaRise]

/hold

[SeaRise]

comment /unhold to merge pr~
@guo-shaoge

[guo-shaoge]

/merge

[ti-chi-bot]

@guo-shaoge: Your PR was out of date, I have automatically updated it for you.

At the same time I will also trigger all tests for you:

/run-all-tests

trigger some heavy tests which will not run always when PR updated.

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

[guo-shaoge]

/test pull-integration-test