[Feature][Connector-V2] add sqlserver connector #2646

liugddx · 2022-09-05T04:29:52Z

Purpose of this pull request

support sqlserver connector

Check list

Code changed are covered with tests, or it does not need tests for reason:
If any new Jar binary package adding in your PR, please add License Notice according
New License Guide
If necessary, please update the documentation to describe the new feature. https://github.com/apache/incubator-seatunnel/tree/dev/docs

githublaohu · 2022-09-05T18:57:22Z

是否应该是jdbc-connent

CalvinKirs

hi, could you add the doc?

...tor-jdbc-flink-e2e/src/test/java/org/apache/seatunnel/e2e/flink/v2/jdbc/JdbcSqlserverIT.java

...k-connector-v2-e2e/src/test/java/org/apache/seatunnel/e2e/flink/v2/jdbc/JdbcSqlserverIT.java

ic4y

Sort out all duplicate files. The test needs to check whether the data in the source table and the sink table are consistent

pom.xml

seatunnel-connectors-v2/connector-jdbc/pom.xml

...tor-jdbc-flink-e2e/src/test/java/org/apache/seatunnel/e2e/flink/v2/jdbc/JdbcSqlserverIT.java

...k-connector-v2-e2e/src/test/java/org/apache/seatunnel/e2e/flink/v2/jdbc/JdbcSqlserverIT.java

...nel-e2e/seatunnel-flink-connector-v2-e2e/src/test/resources/container-license-acceptance.txt

.../seatunnel-flink-connector-v2-e2e/src/test/resources/jdbc/jdbc_sqlserver_source_to_sink.conf

...k-connector-v2-e2e/src/test/java/org/apache/seatunnel/e2e/spark/v2/jdbc/JdbcSqlserverIT.java

.../seatunnel-spark-connector-v2-e2e/src/test/resources/jdbc/jdbc_sqlserver_source_to_sink.conf

...tor-jdbc-spark-e2e/src/test/java/org/apache/seatunnel/e2e/spark/v2/jdbc/JdbcSqlserverIT.java

liugddx · 2022-09-18T11:33:29Z

PTAL @ic4y @EricJoy2048 @CalvinKirs thanks.

liugddx · 2022-09-18T11:34:31Z

PTAL @ic4y @EricJoy2048 @CalvinKirs thanks.

e2e test pass

...onnector-v2-e2e/connector-jdbc-flink-e2e/src/test/resources/container-license-acceptance.txt

...onnector-v2-e2e/connector-jdbc-spark-e2e/src/test/resources/container-license-acceptance.txt

ic4y · 2022-09-20T03:45:33Z

LGTM @Hisoka-X

docs/en/connector-v2/sink/Jdbc.md

Co-authored-by: Hisoka <fanjiaeminem@qq.com>

liugddx · 2022-09-20T10:22:32Z

PTAL @CalvinKirs @TyrantLucifer @EricJoy2048 .thanks

TyrantLucifer

It also has some problems of your pr. Please fix CI first.

liugddx · 2022-09-22T02:43:35Z

@ashulin @TyrantLucifer @CalvinKirs @Hisoka-X @EricJoy2048 PTAL thanks.

liugddx · 2022-09-24T13:04:41Z

PTAL @CalvinKirs @TyrantLucifer @Hisoka-X .thanks

hailin0 · 2022-09-29T03:33:04Z

LGTM

liugddx · 2022-09-29T05:41:36Z

...atunnel-e2e-common/src/test/java/org/apache/seatunnel/e2e/common/AbstractFlinkContainer.java

@@ -32,22 +31,19 @@
 * The before method will create a Flink cluster, and after method will close the Flink cluster.
 * You can use {@link AbstractFlinkContainer#executeJob} to submit a seatunnel config and run a seatunnel job.
 */
-@Slf4j


I found out that all the integration tests had failed,and the reason is log be inherited.So i fix it provisionally.

@hailin0 @TyrantLucifer @ashulin

I found out that all the integration tests had failed,and the reason is log be inherited.So i fix it provisionally.

@hailin0 @TyrantLucifer @ashulin

@hailin0 PTAL

I found out that all the integration tests had failed,and the reason is log be inherited.So i fix it provisionally.
@hailin0 @TyrantLucifer @ashulin

@hailin0 PTAL

#2946 has resolved this problem, these files should be reverted.

I found out that all the integration tests had failed,and the reason is log be inherited.So i fix it provisionally.
@hailin0 @TyrantLucifer @ashulin

@hailin0 PTAL

#2946 has resolved this problem, these files should be reverted.

yes. I've revert.

liugddx · 2022-09-29T05:59:33Z

please rerun CI thanks @TyrantLucifer

TyrantLucifer · 2022-09-29T06:02:59Z

please rerun CI thanks @TyrantLucifer

Wait until the running workflow is finished before restarting. Please be patient.

# Conflicts: # seatunnel-e2e/seatunnel-connector-v2-e2e/connector-jdbc-e2e/src/test/java/org/apache/seatunnel/connectors/seatunnel/jdbc/JdbcDmdbIT.java # seatunnel-e2e/seatunnel-spark-connector-v2-e2e/connector-jdbc-spark-e2e/src/test/java/org/apache/seatunnel/e2e/spark/v2/jdbc/JdbcDmdbIT.java

liugddx · 2022-09-29T14:41:18Z

@TyrantLucifer @Hisoka-X PTAL

TyrantLucifer · 2022-09-30T06:24:05Z

...atunnel-e2e-common/src/test/java/org/apache/seatunnel/e2e/common/AbstractFlinkContainer.java

@@ -32,22 +31,19 @@
 * The before method will create a Flink cluster, and after method will close the Flink cluster.
 * You can use {@link AbstractFlinkContainer#executeJob} to submit a seatunnel config and run a seatunnel job.
 */
-@Slf4j


Please revert this file because pr #2946 has resolved problems of e2e test.

TyrantLucifer · 2022-09-30T06:24:15Z

...atunnel-e2e-common/src/test/java/org/apache/seatunnel/e2e/common/AbstractSparkContainer.java

 import org.junit.jupiter.api.AfterAll;
 import org.junit.jupiter.api.BeforeAll;
 import org.junit.jupiter.api.TestInstance;
 import org.testcontainers.containers.Container;

 import java.io.IOException;

-@Slf4j


Please revert this file because pr #2946 has resolved problems of e2e test.

TyrantLucifer · 2022-09-30T06:26:17Z

seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-jdbc-flink-e2e/pom.xml

+        <dependency>
+            <groupId>org.testcontainers</groupId>
+            <artifactId>mssqlserver</artifactId>
+            <version>1.17.3</version>


Using ${testcontainer.version} instead of it.

TyrantLucifer · 2022-09-30T06:28:12Z

seatunnel-e2e/seatunnel-spark-connector-v2-e2e/connector-jdbc-spark-e2e/pom.xml

+        <dependency>
+            <groupId>org.testcontainers</groupId>
+            <artifactId>mssqlserver</artifactId>
+            <version>1.17.3</version>


Use ${testcontainer.version} instead of it.

TyrantLucifer

Let's wait CI. Thank you for your contribution.

liugddx · 2022-09-30T10:01:23Z

rerun CI? I don't think this mistake affects @TyrantLucifer

TyrantLucifer · 2022-09-30T10:09:54Z

rerun CI? I don't think this mistake affects @TyrantLucifer

CI had been optimized, so some steps will be enabled, before your pr is merged CI must run successfully. It's a community norm.

There are some problems with the dependency checking step, pls check again.

Hisoka-X · 2022-09-30T10:48:19Z

rerun CI? I don't think this mistake affects @TyrantLucifer

CI had been optimized, so some steps will be enabled, before your pr is merged CI must run successfully. It's a community norm.

There are some problems with the dependency checking step, pls check again.

At now the dependency have some problem because st-engine bring some dependency but not update LICENSE file, we will update it later.

Hisoka-X · 2022-09-30T10:48:21Z

rerun CI? I don't think this mistake affects @TyrantLucifer

CI had been optimized, so some steps will be enabled, before your pr is merged CI must run successfully. It's a community norm.

There are some problems with the dependency checking step, pls check again.

At now the dependency have some problem because st-engine bring some dependency but not update LICENSE file, we will update it later.

TyrantLucifer · 2022-09-30T11:08:53Z

rerun CI? I don't think this mistake affects @TyrantLucifer

CI had been optimized, so some steps will be enabled, before your pr is merged CI must run successfully. It's a community norm.

There are some problems with the dependency checking step, pls check again.

At now the dependency have some problem because st-engine bring some dependency but not update LICENSE file, we will update it later.

Good, let's waiting other steps of CI. I had restarted the failed steps

liugddx force-pushed the jdbc-sqlserver branch from 89ac070 to ade455b Compare September 5, 2022 12:02

liugddx force-pushed the jdbc-sqlserver branch from 9a9aaaa to 40209ae Compare September 6, 2022 06:23

CalvinKirs reviewed Sep 6, 2022

View reviewed changes

liugddx force-pushed the jdbc-sqlserver branch from edb858d to 769f2a7 Compare September 6, 2022 12:49

TyrantLucifer added the connectors-v2 label Sep 6, 2022

EricJoy2048 reviewed Sep 7, 2022

View reviewed changes

...tor-jdbc-flink-e2e/src/test/java/org/apache/seatunnel/e2e/flink/v2/jdbc/JdbcSqlserverIT.java Outdated Show resolved Hide resolved

...k-connector-v2-e2e/src/test/java/org/apache/seatunnel/e2e/flink/v2/jdbc/JdbcSqlserverIT.java Outdated Show resolved Hide resolved

ic4y reviewed Sep 7, 2022

View reviewed changes

liugddx marked this pull request as draft September 8, 2022 15:54

liugddx force-pushed the jdbc-sqlserver branch from f481ea7 to 8c3ed2d Compare September 9, 2022 15:03

liugddx marked this pull request as ready for review September 9, 2022 15:06

Hisoka-X requested a review from ic4y September 17, 2022 11:27

liugddx added 3 commits September 18, 2022 16:02

refactor sqlserver connector

811fb7e

merge dev and fix some conflict

6d2d720

add sqlserver doc

ff16d7c

liugddx force-pushed the jdbc-sqlserver branch from 8c3ed2d to ff16d7c Compare September 18, 2022 08:03

liugddx added 2 commits September 18, 2022 17:17

[Feature][Connector-V2] add sqlserver jdbc driver into container.

113a0bb

[Feature][Connector-V2] fix some bug

a7f26c5

liugddx mentioned this pull request Sep 18, 2022

[Feature][Connector-V2] sql-server connector #2601

Closed

3 tasks

ic4y reviewed Sep 20, 2022

View reviewed changes

...onnector-v2-e2e/connector-jdbc-flink-e2e/src/test/resources/container-license-acceptance.txt Show resolved Hide resolved

...onnector-v2-e2e/connector-jdbc-spark-e2e/src/test/resources/container-license-acceptance.txt Show resolved Hide resolved

ic4y previously approved these changes Sep 20, 2022

View reviewed changes

Hisoka-X reviewed Sep 20, 2022

View reviewed changes

docs/en/connector-v2/sink/Jdbc.md Outdated Show resolved Hide resolved

Update docs/en/connector-v2/sink/Jdbc.md

d3759a1

Co-authored-by: Hisoka <fanjiaeminem@qq.com>

liugddx dismissed ic4y’s stale review via d3759a1 September 20, 2022 06:27

Hisoka-X previously approved these changes Sep 20, 2022

View reviewed changes

TyrantLucifer requested changes Sep 20, 2022

View reviewed changes

Merge branch 'dev' into jdbc-sqlserver

e905dff

Merge remote-tracking branch 'upstream/dev' into jdbc-sqlserver

89dffe1

EricJoy2048 requested a review from TyrantLucifer September 28, 2022 08:33

liugddx added 4 commits September 29, 2022 11:45

Merge branch 'dev' into jdbc-sqlserver

27e7f95

fix some merge error

c72699d

fix some merge error

4e75da2

fix log has private access

9210881

liugddx commented Sep 29, 2022

View reviewed changes

TyrantLucifer reviewed Sep 30, 2022

View reviewed changes

liugddx added 2 commits September 30, 2022 15:07

revert

7ca1e2b

fix some review problem

7e2fed4

TyrantLucifer approved these changes Sep 30, 2022

View reviewed changes

Hisoka-X approved these changes Sep 30, 2022

View reviewed changes

ashulin approved these changes Sep 30, 2022

View reviewed changes

TyrantLucifer merged commit 05d105d into apache:dev Sep 30, 2022

liugddx deleted the jdbc-sqlserver branch October 4, 2022 08:08

EricJoy2048 mentioned this pull request Oct 8, 2022

[Umbrella][Connector-V2] SeaTunnel Connector V2 Tracking Table #3018

Open

3 tasks

zhilinli123 mentioned this pull request Apr 11, 2023

[Feature][Connector-V2][Docs] Unified format for connector-v2 docs #4544

Open

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feature][Connector-V2] add sqlserver connector #2646

[Feature][Connector-V2] add sqlserver connector #2646

liugddx commented Sep 5, 2022 •

edited

Loading

githublaohu commented Sep 5, 2022

CalvinKirs left a comment

ic4y left a comment

liugddx commented Sep 18, 2022

liugddx commented Sep 18, 2022

ic4y commented Sep 20, 2022

liugddx commented Sep 20, 2022

TyrantLucifer left a comment

liugddx commented Sep 22, 2022

liugddx commented Sep 24, 2022

hailin0 commented Sep 29, 2022

liugddx Sep 29, 2022

EricJoy2048 Sep 30, 2022

TyrantLucifer Sep 30, 2022

liugddx Sep 30, 2022

liugddx commented Sep 29, 2022

TyrantLucifer commented Sep 29, 2022

liugddx commented Sep 29, 2022

TyrantLucifer Sep 30, 2022

TyrantLucifer Sep 30, 2022

TyrantLucifer Sep 30, 2022

TyrantLucifer Sep 30, 2022

TyrantLucifer left a comment

liugddx commented Sep 30, 2022

TyrantLucifer commented Sep 30, 2022

Hisoka-X commented Sep 30, 2022

Hisoka-X commented Sep 30, 2022

TyrantLucifer commented Sep 30, 2022

[Feature][Connector-V2] add sqlserver connector #2646

[Feature][Connector-V2] add sqlserver connector #2646

Conversation

liugddx commented Sep 5, 2022 • edited Loading

Purpose of this pull request

Check list

githublaohu commented Sep 5, 2022

CalvinKirs left a comment

Choose a reason for hiding this comment

ic4y left a comment

Choose a reason for hiding this comment

liugddx commented Sep 18, 2022

liugddx commented Sep 18, 2022

ic4y commented Sep 20, 2022

liugddx commented Sep 20, 2022

TyrantLucifer left a comment

Choose a reason for hiding this comment

liugddx commented Sep 22, 2022

liugddx commented Sep 24, 2022

hailin0 commented Sep 29, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

liugddx commented Sep 29, 2022

TyrantLucifer commented Sep 29, 2022

liugddx commented Sep 29, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TyrantLucifer left a comment

Choose a reason for hiding this comment

liugddx commented Sep 30, 2022

TyrantLucifer commented Sep 30, 2022

Hisoka-X commented Sep 30, 2022

Hisoka-X commented Sep 30, 2022

TyrantLucifer commented Sep 30, 2022

liugddx commented Sep 5, 2022 •

edited

Loading