-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Fix][Doris] Fix the abnormality of deleting data in CDC scenario. #7315
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Hisoka-X
approved these changes
Aug 6, 2024
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Thanks @liugddx !
hailin0
approved these changes
Aug 6, 2024
luzongzhu
pushed a commit
to luzongzhu/seatunnel
that referenced
this pull request
Aug 9, 2024
jovezhong
added a commit
to jovezhong/seatunnel
that referenced
this pull request
Aug 12, 2024
commit 9d161d2 Author: lizhenglei <127465317+jackyyyyyssss@users.noreply.github.com> Date: Mon Aug 12 14:03:21 2024 +0800 [Fix] Fix http e2e case (apache#7356) * 1 * fix * fix * fix * fix * fix --------- Co-authored-by: lizhenglei <673421862@qq.com> commit 2489f64 Author: hailin0 <wanghailin@apache.org> Date: Mon Aug 12 13:36:15 2024 +0800 [Improve][Connector] Add multi-table sink option check (apache#7360) * [Improve][Connector] Add multi-table sink option check * fix commit fa34ac9 Author: Jia Fan <fanjiaeminem@qq.com> Date: Mon Aug 12 13:35:43 2024 +0800 [Improve][API] Check catalog table fields name legal before send to downstream (apache#7358) * [Improve][API] Check catalog table fields name legal before send to downstream * update commit 068c5e3 Author: Tyrantlucifer <TyrantLucifer@gmail.com> Date: Mon Aug 12 12:55:41 2024 +0800 [Core][Flink] refactor flink proxy source/sink (apache#7355) commit e23e3ac Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Sun Aug 11 17:40:02 2024 +0800 Bump org.apache.activemq:activemq-client (apache#7323) Bumps org.apache.activemq:activemq-client from 5.14.5 to 5.15.16. commit d46cf16 Author: Jarvis <liunaijie1996@163.com> Date: Sat Aug 10 23:47:12 2024 +0800 [Feature] Split transform and move jar into connectors directory (apache#7218) commit 862e205 Author: Jia Fan <fanjiaeminem@qq.com> Date: Sat Aug 10 15:21:36 2024 +0800 [Improve][Document] Change deprecated connector name in setup.md (apache#7366) commit 2769ed5 Author: hailin0 <wanghailin@apache.org> Date: Sat Aug 10 09:58:03 2024 +0800 [Hotfix][Connector] Fix jdbc compile error (apache#7359) commit 3130ae0 Author: xxsc0529 <93303124+xxsc0529@users.noreply.github.com> Date: Fri Aug 9 20:02:06 2024 +0800 [Fix][Connector-V2][OceanBase] Remove OceanBase catalog's dependency on mysql driver (apache#7311) commit 9216627 Author: chaos <71205599+chaos-cn@users.noreply.github.com> Date: Fri Aug 9 13:53:28 2024 +0800 Update ConsoleSinkFactory.java (apache#7350) Add existing rules to the optionRule commit dc3c239 Author: dailai <dailai@chinatelecom.cn> Date: Fri Aug 9 13:11:30 2024 +0800 [Improve][Jdbc] Skip all index when auto create table to improve performance of write (apache#7288) commit 16950a6 Author: Jia Fan <fanjiaeminem@qq.com> Date: Thu Aug 8 17:27:47 2024 +0800 [Improve][Connector-V2] Improve doris error msg (apache#7343) commit a18fca8 Author: Alex Ting <alextinng@hotmail.com> Date: Thu Aug 8 11:22:41 2024 +0800 [Fix][Connector-tdengine] Fix sql exception and concurrentmodifyexception when connect to taos and read data commit 064fcad Author: lizhenglei <127465317+jackyyyyyssss@users.noreply.github.com> Date: Wed Aug 7 21:57:57 2024 +0800 [Improve][Transform] Improve DynamicCompile transform (apache#7319) commit 16eeb1c Author: He Wang <wanghechn@qq.com> Date: Wed Aug 7 14:54:46 2024 +0800 [Improve][Jdbc] Remove MysqlType references in JdbcDialect (apache#7333) commit 191d9e1 Author: ChunFuWu <319355703@qq.com> Date: Wed Aug 7 14:53:51 2024 +0800 [FIX][SFTP] Fix username parameter error in sftp sink document (apache#7334) commit 764d8b0 Author: Jia Fan <fanjiaeminem@qq.com> Date: Wed Aug 7 12:46:30 2024 +0800 [Improve][API] Make sure the table name in TablePath not be null (apache#7252) commit 8916523 Author: hailin0 <wanghailin@apache.org> Date: Wed Aug 7 12:46:12 2024 +0800 [Hotfix][Connector] Fix kafka consumer log next startup offset (apache#7312) commit 819c685 Author: hailin0 <wanghailin@apache.org> Date: Wed Aug 7 12:45:54 2024 +0800 [Improve][Jdbc] Merge user config primary key when create table (apache#7313) commit 7c3cd99 Author: hailin0 <wanghailin@apache.org> Date: Wed Aug 7 12:44:41 2024 +0800 [Improve] Remove unused code (apache#7324) commit 460e73e Author: hailin0 <wanghailin@apache.org> Date: Wed Aug 7 12:44:24 2024 +0800 [E2E] Enable fakesource e2e of spark/flink (apache#7325) commit f6a1e51 Author: hailin0 <wanghailin@apache.org> Date: Wed Aug 7 12:44:13 2024 +0800 [E2E] Enable JdbcPostgresIdentifierIT (apache#7326) commit a12786b Author: Carl-Zhou-CN <1058249259@qq.com> Date: Wed Aug 7 12:27:50 2024 +0800 [DOC][Oss] fix document configuration is rectified when the oss is selected as the checkpoint base (apache#7332) commit a4db64d Author: zhangdonghao <39961809+hawk9821@users.noreply.github.com> Date: Wed Aug 7 11:52:19 2024 +0800 [Improve][E2E] Support windows for the e2e of paimon (apache#7329) commit 855254e Author: Jia Fan <fanjiaeminem@qq.com> Date: Wed Aug 7 11:03:06 2024 +0800 [Feature][Transform] Add LLM transform (apache#7303) * [Feature][Transform] Add LLM transform * update * update * retrigger commit f0cefbe Author: Arin <136636751+asapekia@users.noreply.github.com> Date: Tue Aug 6 19:23:38 2024 +0530 [Feature] [Activemq] Added activemq sink (apache#7251) commit bb2c912 Author: Guangdong Liu <804167098@qq.com> Date: Tue Aug 6 21:46:12 2024 +0800 [Fix][Doris] Fix the abnormality of deleting data in CDC scenario. (apache#7315) commit c94ea32 Author: Jia Fan <fanjiaeminem@qq.com> Date: Tue Aug 6 11:23:52 2024 +0800 [Fix][Doc] Fix miss sink-options-placeholders.md in sidebars (apache#7310) commit 73632ba Author: hailin0 <wanghailin@apache.org> Date: Tue Aug 6 10:08:45 2024 +0800 [Hotfix][Zeta] Fix task cannot be stopped when system is busy (apache#7292) commit 9d56cc3 Author: virvle <55478661+virvle@users.noreply.github.com> Date: Mon Aug 5 17:51:26 2024 +0800 Update Oracle-CDC.md (apache#7285) commit 4f120ff Author: Jia Fan <fanjiaeminem@qq.com> Date: Mon Aug 5 14:34:01 2024 +0800 [Improve] Update snapshot version to 2.3.7 (apache#7305) commit 2fd4eec Author: Jia Fan <fanjiaeminem@qq.com> Date: Mon Aug 5 11:35:02 2024 +0800 [Fix][Doc] Fix hybrid cluster deployment document display error (apache#7306) commit 82f5d8c Author: hailin0 <wanghailin@apache.org> Date: Sat Aug 3 12:02:19 2024 +0800 [Improve][SQL-Transform] Remove escape identifier from output fields (apache#7297) commit 34a6b8e Author: Jast <admin@hadoop.wiki> Date: Wed Jul 31 14:43:11 2024 +0800 [hotfix][connector-v2-hbase]fix and optimize hbase source problem (apache#7148) * [hotfix][improve][doc]optimize connector hbase source * [doc]add dependent document * [doc]update dependent document * [improve]improve static use * [hotfix]add test case * [hotfix]add test case --------- Co-authored-by: Jia Fan <fanjiaeminem@qq.com> commit 9df557c Author: lizhenglei <127465317+jackyyyyyssss@users.noreply.github.com> Date: Tue Jul 30 21:37:04 2024 +0800 [Improve][Transform] Improve DynamicCompile transform (apache#7264) commit b9acb57 Author: hailin0 <wanghailin@apache.org> Date: Tue Jul 30 16:12:57 2024 +0800 [Feature][Transforms] Support cast to bytes function of sql (apache#7284) commit ec1c319 Author: Nothing. <oswinwuo.o@gmail.com> Date: Mon Jul 29 21:53:01 2024 +0800 [Docs]translate event-listener doc into chinese (apache#7274) commit f012b2a Author: dailai <dailai@chinatelecom.cn> Date: Mon Jul 29 21:51:59 2024 +0800 [Improve][Connector-v2] Optimize the way of databases and tables are checked for existence (apache#7261) commit 4897491 Author: 卢宗柱 <luzongzhu@sina.com> Date: Mon Jul 29 17:29:23 2024 +0800 fix [Bug] Unable to create a source for identifier 'Iceberg'. apache#7182 (apache#7279) Co-authored-by: luzongzhu <luzongzhu@semi-tech.com> commit 21c4f52 Author: corgy-w <73771213+corgy-w@users.noreply.github.com> Date: Mon Jul 29 17:28:10 2024 +0800 [Improve][Core] Improve base on plugin name of lookup strategy (apache#7278) * [bug][plugin-discovery] fix multi plugin discovery * [bug][plugin-discovery] optimize code --------- Co-authored-by: wangchao <wangchao001@zhongan.com> commit 3ccc6a8 Author: hailin0 <wanghailin@apache.org> Date: Mon Jul 29 10:41:09 2024 +0800 [Hotfix][Zeta] Fix task cannot be stopped when system is busy (apache#7280) commit d886495 Author: Guangdong Liu <804167098@qq.com> Date: Fri Jul 26 10:45:32 2024 +0800 [Fix][Connector-V2] Fix doris TRANSFER_ENCODING header error (apache#7267) commit 31e59cd Author: Xuzz <90698333+Xuzhengz@users.noreply.github.com> Date: Fri Jul 26 10:44:50 2024 +0800 [Feature][Jdbc] Support hive compatibleMode add inceptor dialect (apache#7262) commit d003bd8 Author: zhangdonghao <39961809+hawk9821@users.noreply.github.com> Date: Fri Jul 26 10:44:09 2024 +0800 [Feature][Zeta] Added the metrics information of table statistics in multi-table mode (apache#7212) commit 876d2f0 Author: Jia Fan <fanjiaeminem@qq.com> Date: Thu Jul 25 10:24:55 2024 +0800 [Fix][Dist] Remove aws jar dependency in release package (apache#7257) * [Fix][Dist] Remove aws jar dependency in release package * [Fix][Dist] Remove aws jar dependency in release package commit 0d08b20 Author: dailai <dailai@chinatelecom.cn> Date: Thu Jul 25 10:00:13 2024 +0800 [Improve][Connector-v2] Optimize the count table rows for jdbc-oracle and oracle-cdc (apache#7248) commit cc59499 Author: Jia Fan <fanjiaeminem@qq.com> Date: Tue Jul 23 14:27:07 2024 +0800 [Improve][Core] Move MultiTableSink to seatunnel-api module (apache#7243) * [Improve][Core] Move MultiTableSink to seatunnel-api module * [Improve][Core] Move MultiTableSink to seatunnel-api module commit fe0c477 Author: Eric <gaojun2048@gmail.com> Date: Tue Jul 23 08:36:58 2024 +0800 [Hotfix][Zeta] Fix taskgroup failed log lost (apache#7241) commit b0fe432 Author: hawk9821 <39961809+hawk9821@users.noreply.github.com> Date: Mon Jul 22 16:12:12 2024 +0800 The isNullable attribute is true when the primary key field in the Paimon table converts the Column object. apache#7231 (apache#7242) commit 4b3af9b Author: Guangdong Liu <804167098@qq.com> Date: Mon Jul 22 13:04:19 2024 +0800 [Improve][Doris Connector] Unified serialization method,Use RowToJsonConverter and TextSerializationSchema (apache#7229) * 1 * 1 * 1 * Update seatunnel-connectors-v2/connector-doris/src/main/java/org/apache/seatunnel/connectors/doris/source/serialization/RowBatch.java Co-authored-by: Jia Fan <fanjiaeminem@qq.com> --------- Co-authored-by: gdliu3 <gdliu3@iflytek.com> Co-authored-by: Jia Fan <fanjiaeminem@qq.com> commit 4ec25f3 Author: Jia Fan <fanjiaeminem@qq.com> Date: Mon Jul 22 12:01:36 2024 +0800 [Fix][Zeta] Fix release slot resource twice (apache#7236) commit 7dc3fa8 Author: litiliu <38579068+litiliu@users.noreply.github.com> Date: Mon Jul 22 11:41:33 2024 +0800 [Fix][Connector kafka]Fix Kafka consumer stop fetching after TM node restarted (apache#7233) commit 063f83c Author: Guangdong Liu <804167098@qq.com> Date: Mon Jul 22 10:40:58 2024 +0800 [Improve][Restapi] Add ip and port to monitoring information (apache#7203) commit c4ca741 Author: hailin0 <wanghailin@apache.org> Date: Sat Jul 20 21:00:06 2024 +0800 [Feature][Core] Support using upstream table placeholders in sink options and auto replacement (apache#7131) 、 commit 821dfc8 Author: lizhenglei <127465317+jackyyyyyssss@users.noreply.github.com> Date: Sat Jul 20 20:25:50 2024 +0800 [Feature][Transform] DynamicCompile add transform (apache#7170) commit 1e7c78d Author: hilo <1964736605@qq.com> Date: Fri Jul 19 19:44:18 2024 +0800 [Improve][Doc] Add IMap OSS config jars into document (apache#7029) commit d39ad93 Author: Jia Fan <fanjiaeminem@qq.com> Date: Fri Jul 19 19:42:03 2024 +0800 [Fix][Zeta] Fix hybrid deployment can not get worker when init (apache#7235) commit 2058f5d Author: ChunFuWu <319355703@qq.com> Date: Fri Jul 19 14:21:23 2024 +0800 [Chore][NOTICE] Correct year in NOTICE file (apache#7232) commit 6e33a97 Author: Jia Fan <fanjiaeminem@qq.com> Date: Thu Jul 18 23:03:27 2024 +0800 [Fix] Fix Hana type converter decimal scale is 0 convert to int error (apache#7167) commit d1b4a7f Author: Jia Fan <fanjiaeminem@qq.com> Date: Thu Jul 18 15:31:30 2024 +0800 [Improve][Zeta] Add check for submit duplicate job id (apache#7021) commit e44e8b9 Author: hailin0 <wanghailin@apache.org> Date: Thu Jul 18 15:30:49 2024 +0800 [Improve][Jdbc] Support write unicode text into sqlserver (apache#7159) commit f40f11a Author: tcodehuber <tcodehuber@gmail.com> Date: Thu Jul 18 00:03:33 2024 +0800 [Docs] Update engine related docs info (apache#7228)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose of this pull request
Does this PR introduce any user-facing change?
How was this patch tested?
Check list
New License Guide
release-note
.