Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Mongodb datasource module #3764

Merged
merged 8 commits into from Nov 16, 2022

Conversation

FinalTarget
Copy link
Contributor

@FinalTarget FinalTarget commented Nov 3, 2022

What is the purpose of the change

Access the mongodb data source, and realize the interface for obtaining databases, collections and fields from mongodb

Related issues/PRs

Related issues: #2718
Related pr:

Brief change log

  • Add linkis-metadata-query-service-mongodb module.

Checklist

  • I have read the Contributing Guidelines on pull requests.
  • I have explained the need for this PR and the problem it solves
  • I have explained the changes or the new features added to this PR
  • I have added tests corresponding to this change
  • I have updated the documentation to reflect this change
  • I have verified that this change is backward compatible (If not, please discuss on the Linkis mailing list first)
  • If this is a code change: I have written unit tests to fully verify the new behavior.

@codecov
Copy link

codecov bot commented Nov 3, 2022

Codecov Report

Merging #3764 (d19387b) into dev-1.3.2 (b30064d) will decrease coverage by 0.07%.
The diff coverage is n/a.

❗ Current head d19387b differs from pull request most recent head 8161268. Consider uploading reports for the commit 8161268 to get more accurate results

@@               Coverage Diff               @@
##             dev-1.3.2    #3764      +/-   ##
===============================================
- Coverage        14.20%   14.13%   -0.08%     
+ Complexity        1552     1512      -40     
===============================================
  Files             1077     1046      -31     
  Lines            39354    38647     -707     
  Branches          5510     5426      -84     
===============================================
- Hits              5592     5464     -128     
+ Misses           32995    32422     -573     
+ Partials           767      761       -6     
Impacted Files Coverage Δ
...a/org/apache/linkis/scheduler/queue/Consumer.scala 85.71% <0.00%> (-14.29%) ⬇️
...eplugin/spark/factory/SparkEngineConnFactory.scala 22.91% <0.00%> (-0.80%) ⬇️
...lugin/spark/executor/SparkEngineConnExecutor.scala 29.32% <0.00%> (-0.31%) ⬇️
...e/linkis/engineplugin/spark/common/SparkKind.scala 32.07% <0.00%> (-0.07%) ⬇️
...eplugin/spark/errorcode/SparkErrorCodeSummary.java 0.00% <0.00%> (ø)
...n/spark/factory/SparkDataCalcExecutorFactory.scala
...gin/spark/datacalc/model/DataCalcPluginConfig.java
...spark/datacalc/source/ManagedJdbcSourceConfig.java
...is/engineplugin/spark/datacalc/sink/FileSink.scala
...gin/spark/datacalc/sink/ManagedJdbcSinkConfig.java
... and 26 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@casionone casionone changed the base branch from release-1.3.0 to dev-1.3.2 November 3, 2022 13:10
@binbinCheng
Copy link
Contributor

need mvn spotless:apply

<version>3.12.7</version>
</dependency>
<dependency>
<groupId>org.apache.linkis</groupId>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

linkis-metadata-query-common duplicate
need to be removed

@@ -36,6 +36,7 @@
<module>linkis-metadata-query/service/mysql</module>
<module>linkis-metadata-query/service/hdfs</module>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  <module>linkis-metadata-query/server</module>
  <module>linkis-metadata-query/service/mongodb</module>

Need to adjust the order

 <module>linkis-metadata-query/service/mongodb</module>
 <module>linkis-metadata-query/server</module>

@casionone casionone changed the title 【Feature】Mongodb datasource module [Feature] Mongodb datasource module Nov 7, 2022
@casionone
Copy link
Contributor

mongo-java-driver-3.12.7.jar is not in file known-dependencies.txt
need to be added to known-dependencies.txt

The corresponding license file also needs to be added
can refer to:
a7cfa33#diff-aa42a02989789460780e3c807f268202cdfca6ad5b7763e293d362d7416ec6ad

FinalTarget and others added 4 commits November 15, 2022 14:16
2.Remove duplicate dependencies <artifactId>linkis-metadata-query-common</artifactId>.
3.Adjust compilation order.
4.Add the license information of mongo-java-driver-3.12.7.jar

apache#2718
@jacktao007
Copy link
Contributor

Can mongo use sql?(mongo 能用sql吗?)

Copy link
Contributor

@peacewong peacewong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@peacewong peacewong merged commit 6dc07c3 into apache:dev-1.3.2 Nov 16, 2022
@peacewong peacewong deleted the dev-1.3.2-mongodb branch November 18, 2022 09:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants