Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Catalog refactor #45

Merged
merged 84 commits into from
Jun 29, 2022
Merged

Catalog refactor #45

merged 84 commits into from
Jun 29, 2022

Conversation

dmetasoul01
Copy link
Contributor

Close #23

sunmaosen and others added 30 commits June 29, 2022 17:23
@dmetasoul-opensource
Flink catalog management
219b07d 
Flink sink writer
New meta
@dmetasoul-opensource
Flink catalog management
94d9ae0 
Flink sink writer
New meta
@dmetasoul-opensource
new postgres sql (#24, #27)
a4563bb
@dmetasoul-opensource
delete
6606c21
@dmetasoul-opensource
Metadata for pg (#24)
fdef281
@dmetasoul-opensource
Metadata for pg (#24)
e978ccb
@dmetasoul-opensource
Metadata for pg (#24)
6762492
@dmetasoul-opensource
rebuild (#28)
80964d1 
 snapshot
 snapshotmanagement
 sparkutil
 createtablecommand
@dmetasoul-opensource
add MetaVersion (#28)
ea56331
@dmetasoul-opensource
DataFileOp add new property:size and modificationTime (#40)
90dfe58
@dmetasoul-opensource
adjust metadata add datacommitinfo to metainfo (#40)
cddd9d8 
delete configuration from doMetaCommit
@dmetasoul-opensource
add new MetaCommit and DataOperate
f7b5987
@dmetasoul-opensource
modify data_commit_info and data_file_op:add column timestamp and cha…
4321e5f 
…nge file_exist_cols
@dmetasoul-opensource
adjust metadata for DataFileInfo (#40)
d58409a
@dmetasoul-opensource
modify data_commit_info and data_file_op:add column timestamp and cha…
22ffad2 
…nge file_exist_cols (#27)
@dmetasoul-opensource
change TransactionalWrite and CommitType (#28)
c1711bd
@dmetasoul-opensource
re-write TransactionalCommit commit method (#28)
6f17506
@dmetasoul-opensource
adjust metadata for DataFileInfo (#28)
ade9ea2
@dmetasoul-opensource
adjust metadata for DataFileInfo (#28)
ef67254
@dmetasoul-opensource
fix data_commit_info property bug (#28)
397df81
@dmetasoul-opensource
overall code merge
f7028ec
@dmetasoul-opensource
debug modify
7451946
@dmetasoul-opensource
debug modify:DelayedCommitProtocol.commitTask
ec6f6ba
@dmetasoul-opensource
debug modify: meta data format change
5f3b288
@dmetasoul-opensource
debug modify: data type change
2a708a8
@dmetasoul-opensource
remove materialized view related code (#32)
e15103f
@dmetasoul-opensource
modify the type of map data in configuration transfer to jsonObject
0a88dbf
@dmetasoul-opensource
DataCommitInfoDao:add detection when query criteria are empty (#24, #28)
b9304d4
@dmetasoul-opensource
adjust some bugs for mor
448d42a
@dmetasoul-opensource
adjust scan for MOR
9fc01b4
sunmaosen and others added 19 commits June 29, 2022 17:45
@dmetasoul-opensource
fix streaming bug and parquet scan with merge duplicate hash-key rows…
584f367 
… in one file
@dmetasoul-opensource
fix cdc test cases
2c84cbb
@dmetasoul-opensource
fix casesensitivity case
1890560
@dmetasoul-opensource
fix bug when scan file is empty
4b12d28
@dmetasoul-opensource
adjust test case
0a1a50c
@dmetasoul-opensource
add partition snapshot read (#41)
9b627d0 
add partition rollback (#42)
add compact for different path
fix some code styles
@dmetasoul-opensource
add function: data auto mount hive table
9da56bd
@dmetasoul-opensource
fix cache bug
7419049
@dmetasoul-opensource
add setting default mergeOp function (#44)
67f13f3
@dmetasoul-opensource
add dev docker file
140648d
@dmetasoul-opensource
use self packaged spark for dev docker
9b1c8de
@dmetasoul-opensource
custom database config
ffae095
@dmetasoul-opensource
fix bug：path resolution error when mount hive data (#43)
b3d8260
@dmetasoul-opensource
bugfix: add tini apt install in docker/dev.dockerfile
8427577
@dmetasoul-opensource
adjust error operation info
151e0e5
@dmetasoul-opensource
remove todo and useless code info
6a9c0da
@dmetasoul-opensource
remove test case useless code
c6f5517
@dmetasoul-opensource
delete meta test class
c755db0
@dmetasoul-opensource
fix bug: commitData result judge condition error
7ad7dd6
@dmetasoul01 dmetasoul01 added enhancement New feature or request spark spark support into lakesoul epic labels Jun 29, 2022
@dmetasoul01 dmetasoul01 self-assigned this Jun 29, 2022
@dmetasoul01 dmetasoul01 added this to In progress in [Catalog] Catalog Refactor via automation Jun 29, 2022
@moresun moresun merged commit 1ddec6a into main Jun 29, 2022
[Catalog] Catalog Refactor automation moved this from In progress to Done Jun 29, 2022
@dmetasoul01 dmetasoul01 deleted the develop/catalog_refactor branch July 24, 2022 05:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@moresun moresun Awaiting requested review from moresun

@F-PHantam F-PHantam Awaiting requested review from F-PHantam

Assignees

@dmetasoul01 dmetasoul01

Labels
enhancement New feature or request epic spark spark support into lakesoul
Projects
No open projects
[Catalog] Catalog Refactor
Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[EPIC] Catalog refactor
3 participants
@dmetasoul01 @moresun @dmetasoul-opensource