Skip to content

[Feat]Restore presto and spark sql datasource in datasource plugin module#7619

Merged
CalvinKirs merged 4 commits intoapache:devfrom
sparklezzz:datasource_plugin_restore_presto_spark
Mar 24, 2022
Merged

[Feat]Restore presto and spark sql datasource in datasource plugin module#7619
CalvinKirs merged 4 commits intoapache:devfrom
sparklezzz:datasource_plugin_restore_presto_spark

Conversation

@sparklezzz
Copy link
Contributor

Purpose of the pull request

Restore presto sql and spark sql datasource in datasource plugin module, which were removed during refactor in v2.0.1

Brief change log

Verify this pull request

This pull request is code cleanup without any test coverage.

(or)

This pull request is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

@codecov-commenter
Copy link

Codecov Report

Merging #7619 (a04c329) into dev (871db33) will decrease coverage by 0.31%.
The diff coverage is 7.60%.

Impacted file tree graph

@@             Coverage Diff              @@
##                dev    #7619      +/-   ##
============================================
- Coverage     41.34%   41.02%   -0.32%     
+ Complexity     3678     3670       -8     
============================================
  Files           641      648       +7     
  Lines         26473    26613     +140     
  Branches       2961     2967       +6     
============================================
- Hits          10944    10917      -27     
- Misses        14502    14669     +167     
  Partials       1027     1027              
Impacted Files Coverage Δ
...ugin/datasource/presto/PrestoDataSourceClient.java 0.00% <0.00%> (ø)
...nscheduler/plugin/datasource/utils/CommonUtil.java 0.00% <0.00%> (ø)
...plugin/datasource/spark/SparkDataSourceClient.java 1.51% <1.51%> (ø)
...gin/datasource/presto/PrestoDataSourceChannel.java 50.00% <50.00%> (ø)
...lugin/datasource/spark/SparkDataSourceChannel.java 50.00% <50.00%> (ø)
...asource/presto/PrestoDataSourceChannelFactory.java 66.66% <66.66%> (ø)
...atasource/spark/SparkDataSourceChannelFactory.java 66.66% <66.66%> (ø)
...tasource/clickhouse/ClickHouseConnectionParam.java 0.00% <0.00%> (-50.00%) ⬇️
...urce/clickhouse/ClickHouseDataSourceProcessor.java 2.00% <0.00%> (-50.00%) ⬇️
...ource/clickhouse/ClickHouseDataSourceParamDTO.java 0.00% <0.00%> (-33.34%) ⬇️
... and 17 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 871db33...a04c329. Read the comment docs.

@sonarqubecloud
Copy link

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 14 Code Smells

6.7% 6.7% Coverage
13.9% 13.9% Duplication

@lenboo
Copy link
Contributor

lenboo commented Dec 27, 2021

@sparklezzz
Is there an issue that can be associated with this PR?

@mgduoduo
Copy link
Contributor

mgduoduo commented Feb 8, 2022

Looking forward to an early merger. we can't config the presto connection now after upgrading to version 2.0.3,
error: "datasource plugin 'presto' is not found"
however it works fine in 2.0.0.

@Narcasserun
Copy link
Contributor

Maybe we can expose the jdbcUrl parameter without ds assembling it by ourselves, which can reduce a lot of repetitive code, and can reuse the same logic like hive, presto, spark, etc.

Copy link
Contributor

@Narcasserun Narcasserun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a lot of duplicate codes in hive, Presto and spark data sources. Maybe we should reuse logic

@EricJoy2048
Copy link
Member

Hi @sparklezzz I have the same problem. When can you come back?

@sonarqubecloud
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 1 Code Smell

43.8% 43.8% Coverage
0.0% 0.0% Duplication

Copy link
Contributor

@caishunfeng caishunfeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@CalvinKirs CalvinKirs merged commit 934d0ad into apache:dev Mar 24, 2022
@brave-lee brave-lee added this to the 2.0.7 milestone Sep 20, 2022
@brave-lee brave-lee modified the milestones: 2.0.7, 2.0.8 Oct 20, 2022
@brave-lee brave-lee removed this from the 2.0.8 milestone Feb 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

9 participants