[Feature][Connector-V2][Jdbc] support gbase 8a #3026

liugddx · 2022-10-08T15:28:22Z

close #3016

Purpose of this pull request

gbase 8a source connector.

Check list

Code changed are covered with tests, or it does not need tests for reason:
If any new Jar binary package adding in your PR, please add License Notice according
New License Guide
If necessary, please update the documentation to describe the new feature. https://github.com/apache/incubator-seatunnel/tree/dev/docs

liugddx · 2022-10-11T09:19:59Z

please run CI @EricJoy2048

liugddx · 2022-10-11T09:32:50Z

gbase8a must use sql date java.sql.Date,java.sql.Time,java.sql.Timestamp,otherwise it will be abnormal.

Exception in thread "main" com.gbase.jdbc.GBaseDataTruncation: Data truncation: Incorrect date value: '******'
	at com.gbase.jdbc.GBaseIO.checkErrorPacket(GBaseIO.java:3631)
	at com.gbase.jdbc.GBaseIO.checkErrorPacket(GBaseIO.java:3565)
	at com.gbase.jdbc.GBaseIO.sendCommand(GBaseIO.java:1996)
	at com.gbase.jdbc.GBaseIO.sqlQueryDirect(GBaseIO.java:2150)
	at com.gbase.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2652)
	at com.gbase.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2133)
	at com.gbase.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2429)
	at com.gbase.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2347)
	at com.gbase.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2332)

hailin0 · 2022-10-12T06:33:49Z

please fix ci error

liugddx · 2022-10-12T06:40:17Z

Hold on,i'm fixing it.   Best Regards liugddx ***@***.***  

…

------------------ Original ------------------ From: ***@***.***>; Date: 2022年10月12日(星期三) 下午2:34 To: ***@***.***>; Cc: ***@***.***>; ***@***.***>; Subject: Re: [apache/incubator-seatunnel] [Feature][Connector-V2][Jdbc] support gbase 8a (PR #3026) please fix ci error — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

EricJoy2048 · 2022-10-13T13:53:41Z

...he/seatunnel/connectors/seatunnel/jdbc/internal/dialect/gbase8a/Gbase8aJdbcRowConverter.java

+
+    @Override
+    public SeaTunnelRow toInternal(ResultSet rs, ResultSetMetaData metaData, SeaTunnelRowType typeInfo) throws SQLException {
+        List<Object> fields = new ArrayList<>();


gave a init length is better.

EricJoy2048 · 2022-10-13T13:56:48Z

...he/seatunnel/connectors/seatunnel/jdbc/internal/dialect/gbase8a/Gbase8aJdbcRowConverter.java

+
+        for (int i = 1; i <= seaTunnelDataTypes.length; i++) {
+            Object seatunnelField;
+            SeaTunnelDataType<?> seaTunnelDataType = seaTunnelDataTypes[i - 1];


use seaTunnelDataType.getSqlType() is better.

EricJoy2048 · 2022-10-13T13:58:47Z

...org/apache/seatunnel/connectors/seatunnel/jdbc/internal/dialect/oracle/OracleTypeMapper.java

            case ORACLE_NUMBER:
-                if (precision < 38) {
-                    return new DecimalType(precision, scale);
-                }


can you add a comment about why delete this?

liugddx · 2022-10-14T12:49:06Z

@EricJoy2048 @Hisoka-X @hailin0 please code review,thanks

liugddx · 2022-10-15T14:18:36Z

rerun CI and help to review thanks. @CalvinKirs

...he/seatunnel/connectors/seatunnel/jdbc/internal/dialect/gbase8a/Gbase8aJdbcRowConverter.java

…e/seatunnel/connectors/seatunnel/jdbc/internal/dialect/gbase8a/Gbase8aJdbcRowConverter.java Co-authored-by: Eric <gaojun2048@gmail.com>

EricJoy2048

LGTM @CalvinKirs @ic4y PTAL

Hisoka-X · 2022-10-18T10:02:49Z

seatunnel-api/src/main/java/org/apache/seatunnel/api/table/type/SqlDateType.java

+import java.sql.Timestamp;
+import java.util.Objects;
+
+public class SqlDateType<T> implements SeaTunnelDataType<T> {


@ashulin PTAL about add new SqlDateType.

IMO, this is not necessary.
The same effect can be achieved using LocalTimeType.
Multiple time types can confuse developers/users

Hisoka-X · 2022-10-18T10:04:40Z

Why we need add SqlDateType? It's same with LocalTimeType.

liugddx · 2022-10-18T10:33:38Z

Why we need add SqlDateType? It's same with LocalTimeType.

gbase8a must use sql date java.sql.Date,java.sql.Time,java.sql.Timestamp,otherwise it will be abnormal.

Exception in thread "main" com.gbase.jdbc.GBaseDataTruncation: Data truncation: Incorrect date value: '******'
	at com.gbase.jdbc.GBaseIO.checkErrorPacket(GBaseIO.java:3631)
	at com.gbase.jdbc.GBaseIO.checkErrorPacket(GBaseIO.java:3565)
	at com.gbase.jdbc.GBaseIO.sendCommand(GBaseIO.java:1996)
	at com.gbase.jdbc.GBaseIO.sqlQueryDirect(GBaseIO.java:2150)
	at com.gbase.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2652)
	at com.gbase.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2133)
	at com.gbase.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2429)
	at com.gbase.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2347)
	at com.gbase.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2332)

@ashulin @Hisoka-X Gbase does not support localTimeType type and i don't know what better way to do it.

liugddx · 2022-10-18T12:32:06Z

Gbase8a test is ok,however, the sink does not support localTimeType type.Now I'm using the Assert test results . @EricJoy2048 @Hisoka-X @ashulin

Hisoka-X · 2022-10-19T02:42:42Z

The CI not passed, please check

liugddx · 2022-10-19T05:55:17Z

The CI not passed, please check

Done.

Hisoka-X

LGTM

EricJoy2048

+1

* gbase 8a connector * add gbase8a e2e test

gbase 8a connector

eb44fc4

liugddx marked this pull request as draft October 8, 2022 15:28

CalvinKirs added the connectors-v2 label Oct 9, 2022

liugddx added 10 commits October 9, 2022 14:13

Merge remote-tracking branch 'upstream/dev' into jdbc-gbase8a

e8a1ecc

add gbase8a e2e test

74928f9

add gbase8a e2e test

9971e65

fix some error

c1c86a7

fix some error

6582058

FIX time type error

f524864

FIX time type error

106db76

FIX time type error

186418b

FIX time type error

cc26aa8

add gbase8a doc

6f979b6

liugddx marked this pull request as ready for review October 11, 2022 09:18

liugddx added 2 commits October 12, 2022 13:07

Merge remote-tracking branch 'upstream/dev' into jdbc-gbase8a

1df87b6

reslove merge error

f7845c4

liugddx added 4 commits October 12, 2022 15:26

fix oracle connector error

f13c34f

fix oracle connector error

373e8f9

fix oracle float precision problem

d4fb795

Merge remote-tracking branch 'upstream/dev' into jdbc-gbase8a

e13d2d5

EricJoy2048 reviewed Oct 13, 2022

View reviewed changes

liugddx added 2 commits October 13, 2022 22:41

fix some error

271e968

fix some error

ec75d10

Merge remote-tracking branch 'upstream/dev' into jdbc-gbase8a

51f05c6

EricJoy2048 reviewed Oct 16, 2022

View reviewed changes

...he/seatunnel/connectors/seatunnel/jdbc/internal/dialect/gbase8a/Gbase8aJdbcRowConverter.java Outdated Show resolved Hide resolved

Update seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apach…

b14234c

…e/seatunnel/connectors/seatunnel/jdbc/internal/dialect/gbase8a/Gbase8aJdbcRowConverter.java Co-authored-by: Eric <gaojun2048@gmail.com>

liugddx requested a review from EricJoy2048 October 18, 2022 04:28

EricJoy2048 previously approved these changes Oct 18, 2022

View reviewed changes

Merge branch 'dev' into jdbc-gbase8a

d0430dd

Hisoka-X dismissed EricJoy2048’s stale review via d0430dd October 18, 2022 10:01

Hisoka-X reviewed Oct 18, 2022

View reviewed changes

liugddx added 3 commits October 18, 2022 19:27

fix jdbc e2e test error

52a0de4

revert gbase8a date type.

179aa2f

modify gbase8a e2e test

84eef7e

fix jdbc connector e2e bug

fe7f505

revert code

b3b1779

Hisoka-X approved these changes Oct 19, 2022

View reviewed changes

liugddx requested a review from EricJoy2048 October 19, 2022 08:31

EricJoy2048 approved these changes Oct 19, 2022

View reviewed changes

EricJoy2048 merged commit dc6e85d into apache:dev Oct 19, 2022

EricJoy2048 mentioned this pull request Oct 20, 2022

[Umbrella][Connector-V2] SeaTunnel Connector V2 Tracking Table #3018

Open

3 tasks

liugddx deleted the jdbc-gbase8a branch October 20, 2022 05:27

Carl-Zhou-CN pushed a commit to Carl-Zhou-CN/incubator-seatunnel that referenced this pull request Oct 27, 2022

[Feature][Connector-V2][Jdbc] support gbase 8a (apache#3026)

1c7bd0c

* gbase 8a connector * add gbase8a e2e test

Carl-Zhou-CN pushed a commit to Carl-Zhou-CN/incubator-seatunnel that referenced this pull request Oct 31, 2022

[Feature][Connector-V2][Jdbc] support gbase 8a (apache#3026)

7eb5349

* gbase 8a connector * add gbase8a e2e test

zhilinli123 mentioned this pull request Apr 11, 2023

[Feature][Connector-V2][Docs] Unified format for connector-v2 docs #4544

Open

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feature][Connector-V2][Jdbc] support gbase 8a #3026

[Feature][Connector-V2][Jdbc] support gbase 8a #3026

liugddx commented Oct 8, 2022 •

edited

liugddx commented Oct 11, 2022

liugddx commented Oct 11, 2022

hailin0 commented Oct 12, 2022

liugddx commented Oct 12, 2022 via email

EricJoy2048 Oct 13, 2022

EricJoy2048 Oct 13, 2022

EricJoy2048 Oct 13, 2022

liugddx commented Oct 14, 2022

liugddx commented Oct 15, 2022

EricJoy2048 left a comment •

edited

Hisoka-X Oct 18, 2022

ashulin Oct 18, 2022 •

edited

Hisoka-X commented Oct 18, 2022

liugddx commented Oct 18, 2022

liugddx commented Oct 18, 2022

Hisoka-X commented Oct 19, 2022

liugddx commented Oct 19, 2022

Hisoka-X left a comment

EricJoy2048 left a comment

[Feature][Connector-V2][Jdbc] support gbase 8a #3026

[Feature][Connector-V2][Jdbc] support gbase 8a #3026

Conversation

liugddx commented Oct 8, 2022 • edited

Purpose of this pull request

Check list

liugddx commented Oct 11, 2022

liugddx commented Oct 11, 2022

hailin0 commented Oct 12, 2022

liugddx commented Oct 12, 2022 via email

EricJoy2048 Oct 13, 2022

Choose a reason for hiding this comment

EricJoy2048 Oct 13, 2022

Choose a reason for hiding this comment

EricJoy2048 Oct 13, 2022

Choose a reason for hiding this comment

liugddx commented Oct 14, 2022

liugddx commented Oct 15, 2022

EricJoy2048 left a comment • edited

Choose a reason for hiding this comment

Hisoka-X Oct 18, 2022

Choose a reason for hiding this comment

ashulin Oct 18, 2022 • edited

Choose a reason for hiding this comment

Hisoka-X commented Oct 18, 2022

liugddx commented Oct 18, 2022

liugddx commented Oct 18, 2022

Hisoka-X commented Oct 19, 2022

liugddx commented Oct 19, 2022

Hisoka-X left a comment

Choose a reason for hiding this comment

EricJoy2048 left a comment

Choose a reason for hiding this comment

liugddx commented Oct 8, 2022 •

edited

EricJoy2048 left a comment •

edited

ashulin Oct 18, 2022 •

edited