[improvement-15760][datasource-plugin] fix sql task split error #15760

[songwenyong]

fix #15760

• Fix the bug in SQL splitting by completing the task in two steps: 1. removeComment 2. split

• Add a unit test for Hive SQL splitting.

Purpose of the pull request

Fix the SQL splitting bug in the datasource-plugin module so that the framework can correctly recognize the -- comments at the end of SQL lines.

Brief change log

Modify the DataSourceProcessor.splitAndRemoveComment method to make two calls. First, call SQLParserUtils.removeComment, and then call SQLParserUtils.split

Verify this pull request

This pull request is code cleanup without any test coverage.

(or)

This pull request is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(or)

If your pull request contain incompatible change, you should also add it to docs/docs/en/guide/upgrede/incompatible.md

[fuchanghai]

hi @songwenyong cool,but i dont think this is bug ，i think this a feature, cc @caishunfeng @EricGao888 WDYT?

[SbloodyS]

hi @songwenyong cool,but i dont think this is bug ，i think this a feature, cc @caishunfeng @EricGao888 WDYT?

It's more like an improvement for me.

[SbloodyS]

Please link this pr to your issue. @songwenyong

[fuchanghai]

hi @songwenyong cool,but i dont think this is bug ，i think this a feature, cc @caishunfeng @EricGao888 WDYT?

It's more like an improvement for me.

yep, you are right

[SbloodyS]

Please change title to [Improvement-IssueId] and link this pr to issue. @songwenyong

[songwenyong]

Please change title to [Improvement-IssueId] and link this pr to issue. @songwenyong

This PR resolves the existing issue #15760 which created by stevqin. Do I need to create a new Improvement issue?

[rickchengx]

LGTM

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 22.22222% with 14 lines in your changes are missing coverage. Please review.

Project coverage is 39.11%. Comparing base (c0435e5) to head (a3f0b57).

❗ Current head a3f0b57 differs from pull request most recent head 8aa4421. Consider uploading reports for the commit 8aa4421 to get more accurate results

Files	Patch %	Lines
...ce/api/datasource/AbstractDataSourceProcessor.java	0.00%	2 Missing ⚠️
...lickhouse/param/ClickHouseDataSourceProcessor.java	0.00%	2 Missing ⚠️
...source/dameng/param/DamengDataSourceProcessor.java	0.00%	2 Missing ⚠️
...n/datasource/db2/param/Db2DataSourceProcessor.java	0.00%	2 Missing ⚠️
...ostgresql/param/PostgreSQLDataSourceProcessor.java	0.00%	2 Missing ⚠️
.../sqlserver/param/SQLServerDataSourceProcessor.java	0.00%	2 Missing ⚠️
...tasource/trino/param/TrinoDataSourceProcessor.java	0.00%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##                dev   #15794      +/-   ##
============================================
- Coverage     39.11%   39.11%   -0.01%     
  Complexity     4887     4887              
============================================
  Files          1326     1326              
  Lines         45206    45215       +9     
  Branches       4818     4818              
============================================
+ Hits          17683    17685       +2     
- Misses        25635    25641       +6     
- Partials       1888     1889       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[fuchanghai]

LGTM

[sonarcloud]

Quality Gate failed

Failed conditions
20.0% Coverage on New Code (required ≥ 60%)

See analysis details on SonarCloud

[SbloodyS]

+1