Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: support rollback info compress #3172

Merged
merged 41 commits into from
Dec 15, 2020

Conversation

caohdgege
Copy link
Contributor

@caohdgege caohdgege commented Oct 9, 2020

Ⅰ. Describe what this PR did

support rollback info compress
支持对undo_log中的rollback_info进行压缩存储

顺便把 OracleUndoLogManager 和 PostgresqlUndoLogManager 中的hard code用常量替换了

Ⅱ. Does this pull request fix one issue?

fixes #3159

Ⅲ. Why don't you add test cases (unit test/integration test)?

already add

Ⅳ. Describe how to verify it

Ⅴ. Special notes for reviews

@caohdgege
Copy link
Contributor Author

zip(default)和gzip两种压缩方式的压缩比例都挺高的,然后总的压缩+插入时间也比较短
lz4不适合,虽然压缩的快,但是压缩比例不高,这样压缩时间+插入undo_log的时间会比其他压缩方式长,不值得
bzip2和7z也不太合适,压缩用时太长,比别人压缩+插入的时间都长了。。。

@caohdgege
Copy link
Contributor Author

image
image

@funky-eyes
Copy link
Contributor

good job

@caohdgege
Copy link
Contributor Author

ps: 以上测试时间数据都是在MySQL下测试得来,pg和Oracle只是进行了功能测试,没有进行性能测试。

@caohdgege
Copy link
Contributor Author

补一下测试的时候的日志和统计数据。
测试日志.zip

@funky-eyes
Copy link
Contributor

config.txt application.properties application.properties supplement the related configuration

@codecov-io
Copy link

codecov-io commented Oct 10, 2020

Codecov Report

Merging #3172 (c3e480a) into develop (23a407e) will decrease coverage by 0.14%.
The diff coverage is 42.30%.

Impacted file tree graph

@@              Coverage Diff              @@
##             develop    #3172      +/-   ##
=============================================
- Coverage      51.11%   50.96%   -0.15%     
+ Complexity      3297     3295       -2     
=============================================
  Files            614      616       +2     
  Lines          20125    20180      +55     
  Branches        2514     2524      +10     
=============================================
- Hits           10287    10285       -2     
- Misses          8806     8857      +51     
- Partials        1032     1038       +6     
Impacted Files Coverage Δ Complexity Δ
...n/src/main/java/io/seata/common/DefaultValues.java 0.00% <ø> (ø) 0.00 <0.00> (ø)
...ava/io/seata/core/constants/ConfigurationKeys.java 0.00% <ø> (ø) 0.00 <0.00> (ø)
...ata/core/rpc/processor/server/BatchLogHandler.java 0.00% <ø> (ø) 0.00 <0.00> (ø)
...io/seata/rm/datasource/sql/serial/SerialArray.java 0.00% <ø> (ø) 0.00 <0.00> (ø)
.../io/seata/rm/datasource/undo/UndoLogConstants.java 100.00% <ø> (ø) 1.00 <0.00> (ø)
...ing/boot/autoconfigure/SeataAutoConfiguration.java 0.00% <0.00%> (ø) 0.00 <0.00> (ø)
...ta/spring/boot/autoconfigure/StarterConstants.java 100.00% <ø> (ø) 1.00 <0.00> (ø)
...gure/properties/client/UndoCompressProperties.java 30.76% <30.76%> (ø) 1.00 <1.00> (?)
...ata/rm/datasource/undo/AbstractUndoLogManager.java 51.42% <37.50%> (+4.22%) 19.00 <2.00> (+2.00)
...m/datasource/undo/oracle/OracleUndoLogManager.java 9.09% <50.00%> (+0.98%) 1.00 <0.00> (ø)
... and 14 more

@funky-eyes funky-eyes added the module/rm-datasource rm-datasource module label Oct 10, 2020
@funky-eyes funky-eyes added the type: feature Category issues or prs related to feature request. label Oct 21, 2020
@wangliang181230 wangliang181230 added this to the 1.5.0 milestone Oct 29, 2020
caohdgege and others added 9 commits October 31, 2020 16:02
# Conflicts:
#	seata-spring-boot-starter/src/main/java/io/seata/spring/boot/autoconfigure/StarterConstants.java
# Conflicts:
#	script/client/spring/application.properties
#	script/client/spring/application.yml
#	script/config-center/config.txt
#	seata-spring-boot-starter/src/main/java/io/seata/spring/boot/autoconfigure/StarterConstants.java
Copy link
Contributor

@funky-eyes funky-eyes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@jsbxyyx jsbxyyx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

# Conflicts:
#	compressor/seata-compressor-all/pom.xml
@funky-eyes funky-eyes merged commit aeeda42 into apache:develop Dec 15, 2020
@caohdgege caohdgege deleted the feature-add-zip-for-rollback-info branch February 23, 2021 09:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
module/rm-datasource rm-datasource module type: feature Category issues or prs related to feature request. waitting-to-merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

add rollback_info field unzip for undolog
6 participants