Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: drop aggr tables in drop table #3908

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

vagetablechicken
Copy link
Collaborator

closes #3836

@github-actions github-actions bot added the storage-engine openmldb storage engine. nameserver & tablet label Apr 29, 2024
Copy link

codecov bot commented Apr 29, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 41.00%. Comparing base (d9bb344) to head (9570626).
Report is 1 commits behind head on main.

Additional details and impacted files
@@              Coverage Diff              @@
##               main    #3908       +/-   ##
=============================================
- Coverage     74.98%   41.00%   -33.99%     
  Complexity      658      658               
=============================================
  Files           751      195      -556     
  Lines        135067    11646   -123421     
  Branches       1501     2038      +537     
=============================================
- Hits         101276     4775    -96501     
+ Misses        33503     6583    -26920     
  Partials        288      288               

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

github-actions bot commented Apr 29, 2024

SDK Test Report

102 files  ±0  102 suites  ±0   2m 20s ⏱️ -4s
357 tests ±0  343 ✅ ±0  14 💤 ±0  0 ❌ ±0 
483 runs  ±0  469 ✅ ±0  14 💤 ±0  0 ❌ ±0 

Results for commit 9570626. ± Comparison against base commit d9bb344.

This pull request removes 30 and adds 9 tests. Note that renamed tests count towards both.
  PARTITION BY db1.t1.col2 ORDER BY db1.t1.col1
  PARTITION BY t1.col2 ORDER BY t1.col1
  ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW
 ) limit 10;](1)
 ) limit 10;](2)
 ) limit 10;](3)
 FROM db1.t1
 FROM t1
 WINDOW w1 AS (
 last join db2.t2 order by db2.t2.col1
…
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlLastJoinWithMultipleDB[,  SELECT sum(db1.t1.col1) over w1 as sum_t1_col1, db2.t2.str1 as t2_str1
 FROM db1.t1
 last join db2.t2 order by db2.t2.col1
 on db1.t1.col1 = db2.t2.col1 and db1.t1.col2 = db2.t2.col0
 WINDOW w1 AS (
  PARTITION BY db1.t1.col2 ORDER BY db1.t1.col1
  ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW
 ) limit 10;](2)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlLastJoinWithMultipleDB[db1,  SELECT sum(t1.col1) over w1 as sum_t1_col1, db2.t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0
 WINDOW w1 AS (
  PARTITION BY t1.col2 ORDER BY t1.col1
  ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW
 ) limit 10;](1)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlLastJoinWithMultipleDB[null,  SELECT sum(db1.t1.col1) over w1 as sum_t1_col1, db2.t2.str1 as t2_str1
 FROM db1.t1
 last join db2.t2 order by db2.t2.col1
 on db1.t1.col1 = db2.t2.col1 and db1.t1.col2 = db2.t2.col0
 WINDOW w1 AS (
  PARTITION BY db1.t1.col2 ORDER BY db1.t1.col1
  ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW
 ) limit 10;](3)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[, SELECT db2.t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0;
, SQL parse error: Fail to transform data provider op: table t1 not exists in database []](4)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[db1, SELECT db1.t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0;
, SQL parse error: Column Not found: db1.t2.str1](2)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[db1, SELECT db2.t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = t2.col1 and t1.col2 = db2.t2.col0;
, SQL parse error: Column Not found: .t2.col1](3)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[db1, SELECT t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0;
, SQL parse error: Column Not found: .t2.str1](1)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[null, SELECT db2.t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0;
, SQL parse error: Fail to transform data provider op: table t1 not exists in database []](5)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlWindowLastJoin[ SELECT sum(t1.col1) over w1 as sum_t1_col1, t2.str1 as t2_str1
 FROM t1
 last join t2 order by t2.col1
 on t1.col1 = t2.col1 and t1.col2 = t2.col0
 WINDOW w1 AS (
  PARTITION BY t1.col2 ORDER BY t1.col1
  ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW
 ) limit 10;](1)

♻️ This comment has been updated with latest results.

Copy link
Contributor

github-actions bot commented Apr 29, 2024

Linux Test Report

 55 files  ±0   62 suites  ±0   28m 47s ⏱️ -35s
679 tests ±0  658 ✅  - 13  7 💤 ±0  14 ❌ +13 
680 runs  ±0  659 ✅  - 13  7 💤 ±0  14 ❌ +13 

For more details on these failures, see this check.

Results for commit 9570626. ± Comparison against base commit d9bb344.

♻️ This comment has been updated with latest results.

@vagetablechicken
Copy link
Collaborator Author

truncate base table, will aggr table be cleaned?

@vagetablechicken
Copy link
Collaborator Author

truncate base table, will aggr table be cleaned?

auto aggrs = GetAggregators(tid, pid);
if (aggrs) {
for (const auto& aggr : *aggrs) {
auto agg_table = aggr->GetAggTable();
if (!agg_table) {
PDLOG(WARNING, "aggrate table does not exist. tid[%u] pid[%u] index pos[%u]",
tid, pid, aggr->GetIndexPos());
response->set_code(::openmldb::base::ReturnCode::kTableIsNotExist);
response->set_msg("aggrate table does not exist");
return;
}
uint32_t agg_tid = agg_table->GetId();
uint32_t agg_pid = agg_table->GetPid();
if (auto status = TruncateTableInternal(agg_tid, agg_pid); !status.OK()) {
PDLOG(WARNING, "truncate aggrate table failed. tid[%u] pid[%u] index pos[%u]",
agg_tid, agg_pid, aggr->GetIndexPos());
base::SetResponseStatus(status, response);
return;
}
PDLOG(INFO, "truncate aggrate table success. tid[%u] pid[%u] index pos[%u]",
agg_tid, agg_pid, aggr->GetIndexPos());
}
}

GetAggregators will get all aggr tables of base table, and truncate it.

Copy link
Collaborator

@tobegit3hub tobegit3hub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
storage-engine openmldb storage engine. nameserver & tablet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

long_window bugs
3 participants