Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: optimize partition node, replace with sort node #22629

Merged
merged 3 commits into from Sep 14, 2023
Merged

Conversation

wangjiaming0909
Copy link
Contributor

No description provided.

@wangjiaming0909 wangjiaming0909 force-pushed the feat/TD-25620 branch 3 times, most recently from ee9b9cf to dc77d93 Compare August 30, 2023 05:45
Copy link
Contributor

@dapan1121 dapan1121 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. 生成group key列表的值效率不太高,目前是先拷贝到中间数组再拷贝到最终keyBuf中,可以一步完成或者直接比较各列的值不拷贝;
  2. 根据我们目前的使用方式,group key列表比如等于sort key列表或者是sort key列表的子集,因此计划中不需要传递group key的详细信息,只需要指示group key等于sort key或者是group key是sort key中的前几列即可,这样算子中也不需要重新建立group key的相关信息。

if (!pGroupIdCalc->lastKeyBuf) code = TSDB_CODE_OUT_OF_MEMORY;
}
if (TSDB_CODE_SUCCESS == code) {
pInfo->pGroupIdCalc = pGroupIdCalc;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pGroupIdCalc内存泄漏

source/libs/executor/src/sortoperator.c Outdated Show resolved Hide resolved
@wangjiaming0909 wangjiaming0909 force-pushed the feat/TD-25620 branch 2 times, most recently from 217c0ce to e3d78ab Compare September 11, 2023 08:58

int32_t blockDataGetSortedRows(SSDataBlock* pDataBlock, SArray* pOrderInfo) {
if (!pDataBlock || !pOrderInfo) return 0;
for (int32_t i = 0; i < taosArrayGetSize(pOrderInfo); ++i) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里是因为目前只有一列吗,要不这个逻辑就不对了吧

if (!pOrder) {
code = TSDB_CODE_OUT_OF_MEMORY;
} else {
pSort->excludePkCol = true;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

如果已经存在primary ts列,可以不加新列,excluedPkCol也不需要了

@@ -34,6 +34,7 @@
,,y,system-test,./pytest.sh python3 ./test.py -f 2-query/nestedQuery_time.py -Q 4
,,y,system-test,./pytest.sh python3 ./test.py -f 2-query/nestedQuery_26.py -Q 4
,,y,system-test,./pytest.sh python3 ./test.py -f 2-query/interval_limit_opt.py -Q 4
,,y,system-test,./pytest.sh python3 ./test.py -f 2-query/partition_by_col.py -Q 4
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

我们有几种查询policy,-Q 4只是其中一种,加一个新文件应该是类似这样加:
,,y,system-test,./pytest.sh python3 ./test.py -f 2-query/columnLenUpdated.py
,,y,system-test,./pytest.sh python3 ./test.py -f 2-query/columnLenUpdated.py -Q 2
,,y,system-test,./pytest.sh python3 ./test.py -f 2-query/columnLenUpdated.py -Q 3
,,y,system-test,./pytest.sh python3 ./test.py -f 2-query/columnLenUpdated.py -Q 4

@wangjiaming0909 wangjiaming0909 force-pushed the feat/TD-25620 branch 2 times, most recently from 9ceebeb to 12ffdbd Compare September 13, 2023 08:17
@dapan1121 dapan1121 merged commit 8ef0158 into 3.0 Sep 14, 2023
1 check passed
@gccgdb1234 gccgdb1234 deleted the feat/TD-25620 branch December 11, 2023 08:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants