feat: load, write supports iceberg

[vagetablechicken]

closes #3695

[codecov]

Codecov Report

Attention: 54 lines in your changes are missing coverage. Please review.

Comparison is base (7d755ff) 40.18% compared to head (7c0a4f2) 74.93%.

Files	Patch %	Lines
...paradigm/openmldb/batch/utils/DataSourceUtil.scala	50.53%	46 Missing ⚠️
..._4paradigm/openmldb/batch/utils/HybridseUtil.scala	20.00%	4 Missing ⚠️
...paradigm/openmldb/batch/nodes/SelectIntoPlan.scala	25.00%	3 Missing ⚠️
...aradigm/openmldb/batch/nodes/CreateTablePlan.scala	0.00%	1 Missing ⚠️

Additional details and impacted files

@@              Coverage Diff              @@
##               main    #3737       +/-   ##
=============================================
+ Coverage     40.18%   74.93%   +34.74%     
  Complexity      655      655               
=============================================
  Files           193      733      +540     
  Lines         11384   131618   +120234     
  Branches       1366     1369        +3     
=============================================
+ Hits           4575    98627    +94052     
- Misses         6510    32692    +26182     
  Partials        299      299               

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[github-actions]

SDK Test Report

101 files  ±0  101 suites  ±0   2m 2s ⏱️ -4s
344 tests ±0  331 ✅ ±0  13 💤 ±0  0 ❌ ±0 
469 runs  ±0  456 ✅ ±0  13 💤 ±0  0 ❌ ±0 

Results for commit 7c0a4f2. ± Comparison against base commit 7d755ff.

This pull request removes 34 and adds 13 tests. Note that renamed tests count towards both.

  PARTITION BY db1.t1.col2 ORDER BY db1.t1.col1
  PARTITION BY t1.col2 ORDER BY t1.col1
  ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW
 ) limit 10;](1)
 ) limit 10;](2)
 ) limit 10;](3)
 FROM db1.t1
 FROM t1
 WINDOW w1 AS (
 last join db2.t2 order by db2.t2.col1
…

com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlLastJoinWithMultipleDB[,  SELECT sum(db1.t1.col1) over w1 as sum_t1_col1, db2.t2.str1 as t2_str1
 FROM db1.t1
 last join db2.t2 order by db2.t2.col1
 on db1.t1.col1 = db2.t2.col1 and db1.t1.col2 = db2.t2.col0
 WINDOW w1 AS (
  PARTITION BY db1.t1.col2 ORDER BY db1.t1.col1
  ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW
 ) limit 10;](2)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlLastJoinWithMultipleDB[db1,  SELECT sum(t1.col1) over w1 as sum_t1_col1, db2.t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0
 WINDOW w1 AS (
  PARTITION BY t1.col2 ORDER BY t1.col1
  ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW
 ) limit 10;](1)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlLastJoinWithMultipleDB[null,  SELECT sum(db1.t1.col1) over w1 as sum_t1_col1, db2.t2.str1 as t2_str1
 FROM db1.t1
 last join db2.t2 order by db2.t2.col1
 on db1.t1.col1 = db2.t2.col1 and db1.t1.col2 = db2.t2.col0
 WINDOW w1 AS (
  PARTITION BY db1.t1.col2 ORDER BY db1.t1.col1
  ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW
 ) limit 10;](3)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[, SELECT db2.t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0;
, SQL parse error: Fail to transform data provider op: table t1 not exists in database []](4)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[db1, SELECT db1.t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0;
, SQL parse error: Column Not found: db1.t2.str1](2)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[db1, SELECT db2.t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = t2.col1 and t1.col2 = db2.t2.col0;
, SQL parse error: Column Not found: .t2.col1](3)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[db1, SELECT t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0;
, SQL parse error: Column Not found: .t2.str1](1)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[null, SELECT db2.t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0;
, SQL parse error: Fail to transform data provider op: table t1 not exists in database []](5)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlWindowLastJoin[ SELECT sum(t1.col1) over w1 as sum_t1_col1, t2.str1 as t2_str1
 FROM t1
 last join t2 order by t2.col1
 on t1.col1 = t2.col1 and t1.col2 = t2.col0
 WINDOW w1 AS (
  PARTITION BY t1.col2 ORDER BY t1.col1
  ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW
 ) limit 10;](1)
com._4paradigm.openmldb.batch.utils.DataSourceUtilTest ‑ Test AutoLoad Csv
…

This pull request removes 1 skipped test and adds 1 skipped test. Note that renamed tests count towards both.

com._4paradigm.openmldb.batch.utils.HybridseUtilTest ‑ Test read from hive

com._4paradigm.openmldb.batch.utils.DataSourceUtilTest ‑ Test read from hive

[github-actions]

Linux Test Report

    57 files  +     5     244 suites  +187   1h 18m 25s ⏱️ + 50m 29s
12 574 tests +12 026  12 567 ✅ +12 025  7 💤 +3  0 ❌  - 2 
17 831 runs  +17 282  17 824 ✅ +17 281  7 💤 +3  0 ❌  - 2 

Results for commit 7c0a4f2. ± Comparison against base commit 7d755ff.

♻️ This comment has been updated with latest results.