feature: add LZ4 compressor #2510

diguage · 2020-04-04T14:51:58Z

LZ4 is an extremely fast Compression algorithm. Maybe it should be supported.

zjinlei · 2020-04-04T15:34:02Z

please fix CI error:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:3.0.0:check (validate) on project seata-compressor-lz4: Failed during checkstyle execution: There are 4 errors reported by Checkstyle 6.18 with /home/travis/build/seata/seata/style/seata_checkstyle.xml ruleset.

funky-eyes · 2020-04-04T16:38:17Z

please format the code according to the p3c specification

diguage · 2020-04-07T04:21:41Z

please fix CI error:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:3.0.0:check (validate) on project seata-compressor-lz4: Failed during checkstyle execution: There are 4 errors reported by Checkstyle 6.18 with /home/travis/build/seata/seata/style/seata_checkstyle.xml ruleset.

There is an error. And there is no report. So I cannot find it.

@a364176773 I use p3c to check the code. There is not any warning.

Please help me.

funky-eyes · 2020-04-07T04:26:55Z

please fix CI error:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:3.0.0:check (validate) on project seata-compressor-lz4: Failed during checkstyle execution: There are 4 errors reported by Checkstyle 6.18 with /home/travis/build/seata/seata/style/seata_checkstyle.xml ruleset.

There is an error. And there is no report. So I cannot find it.

@a364176773 I use p3c to check the code. There is not any warning.

Please help me.

please check the code format and local mvn clean install -DskipTests=true

diguage · 2020-04-07T04:39:56Z

please check the code format and local mvn clean install -DskipTests=true

I ran the command. The log had an error. But did not show the detail.

The log is as following:

...
[INFO] seata-spring-boot-starter 1.2.0-SNAPSHOT ........... SUCCESS [  1.714 s]
[INFO] seata-compressor-lz4 1.2.0-SNAPSHOT ................ FAILURE [  0.235 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:27 min
[INFO] Finished at: 2020-04-07T12:39:15+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:3.0.0:check (validate) on project seata-compressor-lz4: Failed during checkstyle execution: There is 1 error reported by Checkstyle 6.18 with /Users/diguage/develop/java/javaprojects/seata/style/seata_checkstyle.xml ruleset. -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :seata-compressor-lz4

funky-eyes · 2020-04-07T04:42:00Z

please check the code format and local mvn clean install -DskipTests=true

I ran the command. The log had an error. But did not show the detail.

The log is as following:

...
[INFO] seata-spring-boot-starter 1.2.0-SNAPSHOT ........... SUCCESS [  1.714 s]
[INFO] seata-compressor-lz4 1.2.0-SNAPSHOT ................ FAILURE [  0.235 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:27 min
[INFO] Finished at: 2020-04-07T12:39:15+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:3.0.0:check (validate) on project seata-compressor-lz4: Failed during checkstyle execution: There is 1 error reported by Checkstyle 6.18 with /Users/diguage/develop/java/javaprojects/seata/style/seata_checkstyle.xml ruleset. -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :seata-compressor-lz4

scroll up to find the corresponding module log, which should have a prompt message

diguage · 2020-04-07T04:43:09Z

scroll up to find the corresponding module log, which should have a prompt message

I saw the seata/compressor/seata-compressor-lz4/target/checkstyle-result.xml file. There is a tip.

<file name="/Users/diguage/develop/java/javaprojects/seata/compressor/seata-compressor-lz4/src/main/java/io/seata/compressor/lz4/Lz4Util.java">
<error line="18" severity="error" message="Using the &apos;.*&apos; form of import should be avoided - net.jpountz.lz4.*." source="com.puppycrawl.tools.checkstyle.checks.imports.AvoidStarImportCheck"/>
</file>

Thank you very much.

slievrly · 2020-04-07T04:48:40Z

@diguage mvn clean install -DskipTests=true -Dmaven.javadoc.skip=true -Dcheckstyle.skip=false

codecov-io · 2020-04-07T04:55:46Z

Codecov Report

Merging #2510 into develop will increase coverage by 0.03%.
The diff coverage is 76.66%.

@@              Coverage Diff              @@
##             develop    #2510      +/-   ##
=============================================
+ Coverage      51.18%   51.21%   +0.03%     
- Complexity      2806     2813       +7     
=============================================
  Files            554      556       +2     
  Lines          17772    17801      +29     
  Branches        2099     2100       +1     
=============================================
+ Hits            9096     9117      +21     
- Misses          7819     7825       +6     
- Partials         857      859       +2

Impacted Files	Coverage Δ	Complexity Δ
...in/java/io/seata/compressor/sevenz/SevenZUtil.java	`82.14% <ø> (ø)`	`6.00 <0.00> (ø)`
...main/java/io/seata/compressor/bzip2/BZip2Util.java	`76.19% <ø> (ø)`	`5.00 <0.00> (ø)`
.../java/io/seata/core/compressor/CompressorType.java	`0.00% <0.00%> (ø)`	`0.00 <0.00> (ø)`
...src/main/java/io/seata/compressor/lz4/Lz4Util.java	`80.00% <80.00%> (ø)`	`6.00 <6.00> (?)`
...in/java/io/seata/compressor/lz4/Lz4Compressor.java	`100.00% <100.00%> (ø)`	`3.00 <3.00> (?)`
...o/seata/server/coordinator/DefaultCoordinator.java	`54.63% <0.00%> (-0.52%)`	`28.00% <0.00%> (-1.00%)`
...in/java/io/seata/server/session/GlobalSession.java	`83.71% <0.00%> (-0.46%)`	`71.00% <0.00%> (-1.00%)`

diguage · 2020-04-07T04:56:14Z

@diguage mvn clean install -DskipTests=true -Dmaven.javadoc.skip=true -Dcheckstyle.skip=false

Thank you. I found the error, and fixed it. Thanks again.

ph3636

Add enumeration to CompressorType

compressor/seata-compressor-lz4/pom.xml

compressor/seata-compressor-lz4/src/main/java/io/seata/compressor/lz4/Lz4Util.java

ph3636 · 2020-04-08T06:44:06Z

compressor/seata-compressor-lz4/src/main/java/io/seata/compressor/lz4/Lz4Util.java

+            throw new NullPointerException("bytes is null");
+        }
+
+        ByteArrayOutputStream outputStream = new ByteArrayOutputStream(ARRAY_SIZE);


You don't need to initialize such a large value, just keep the default value

The default size is 32. Is it too small?
Maybe I can change the ARRAY_SIZE to 1024. Is it OK?

compressor/seata-compressor-lz4/src/main/java/io/seata/compressor/lz4/Lz4Util.java

ph3636 · 2020-04-12T15:21:16Z

seata-all file is missing some configuration

diguage · 2020-04-13T01:52:44Z

seata-all file is missing some configuration

OK

zjinlei · 2020-04-13T04:59:43Z

Please keep the commit log, we can only review the increment.

ph3636 · 2020-04-13T05:23:46Z

seata-all file is missing some configuration

OK

The 682 line of seata-al file is missing some configuration

diguage · 2020-04-13T08:05:29Z

The 682 line of seata-all file is missing some configuration

OK 👌

diguage · 2020-04-13T08:07:26Z

Please keep the commit log, we can only review the increment.
👌 Thanks.

ph3636

LGTM

zjinlei

LGTM
Agree with @ph3636

all/pom.xml

slievrly

LGTM

diguage force-pushed the lz4-compressor branch 3 times, most recently from a44a3fb to c41baea Compare April 4, 2020 14:56

zjinlei added the first-time contributor first-time contributor label Apr 4, 2020

zjinlei changed the title ~~add LZ4 compressor~~ feature: add LZ4 compressor Apr 4, 2020

diguage force-pushed the lz4-compressor branch 2 times, most recently from f78f4dc to cac665f Compare April 7, 2020 03:46

diguage force-pushed the lz4-compressor branch from cac665f to de9502a Compare April 7, 2020 04:46

diguage force-pushed the lz4-compressor branch from de9502a to ebc488d Compare April 7, 2020 05:12

zjinlei added this to the 1.2.0 milestone Apr 8, 2020

ph3636 reviewed Apr 8, 2020

View reviewed changes

diguage force-pushed the lz4-compressor branch 5 times, most recently from 7d855c5 to 481b9e0 Compare April 10, 2020 12:46

Zh1Cheung mentioned this pull request Apr 10, 2020

optimize: add methods to reduce redundant code #2494

Merged

slievrly modified the milestones: 1.2.0, 1.3.0 Apr 12, 2020

diguage force-pushed the lz4-compressor branch from 481b9e0 to 33c6aae Compare April 13, 2020 01:52

ph3636 approved these changes Apr 13, 2020

View reviewed changes

zjinlei approved these changes Apr 13, 2020

View reviewed changes

slievrly reviewed Apr 23, 2020

View reviewed changes

all/pom.xml Show resolved Hide resolved

diguage added 3 commits April 24, 2020 10:13

add LZ4 compressor

6076608

add missing config for seata-all

e1c18cc

add lz4-java dependency in all/pom.pom

6911527

diguage force-pushed the lz4-compressor branch from 11ea2d6 to 6911527 Compare April 24, 2020 02:20

zjinlei and others added 2 commits April 25, 2020 14:12

Merge branch 'develop' into lz4-compressor

a4cd9e4

Merge branch 'develop' into lz4-compressor

1ee32e6

slievrly approved these changes Apr 28, 2020

View reviewed changes

Merge branch 'develop' into lz4-compressor

40f5eb0

slievrly merged commit 58aed9e into apache:develop Apr 29, 2020

diguage deleted the lz4-compressor branch April 29, 2020 07:55

xiaowan pushed a commit to xiaowan/seata that referenced this pull request Apr 30, 2020

feature: add LZ4 compressor (apache#2510)

4cd1603

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feature: add LZ4 compressor #2510

feature: add LZ4 compressor #2510

diguage commented Apr 4, 2020 •

edited

zjinlei commented Apr 4, 2020 •

edited

funky-eyes commented Apr 4, 2020

diguage commented Apr 7, 2020

funky-eyes commented Apr 7, 2020

diguage commented Apr 7, 2020

funky-eyes commented Apr 7, 2020

diguage commented Apr 7, 2020 •

edited

slievrly commented Apr 7, 2020

codecov-io commented Apr 7, 2020 •

edited

diguage commented Apr 7, 2020

ph3636 left a comment

ph3636 Apr 8, 2020

diguage Apr 8, 2020 •

edited

ph3636 commented Apr 12, 2020

diguage commented Apr 13, 2020

zjinlei commented Apr 13, 2020

ph3636 commented Apr 13, 2020

diguage commented Apr 13, 2020

diguage commented Apr 13, 2020

ph3636 left a comment

zjinlei left a comment

slievrly left a comment

feature: add LZ4 compressor #2510

feature: add LZ4 compressor #2510

Conversation

diguage commented Apr 4, 2020 • edited

zjinlei commented Apr 4, 2020 • edited

funky-eyes commented Apr 4, 2020

diguage commented Apr 7, 2020

funky-eyes commented Apr 7, 2020

diguage commented Apr 7, 2020

funky-eyes commented Apr 7, 2020

diguage commented Apr 7, 2020 • edited

slievrly commented Apr 7, 2020

codecov-io commented Apr 7, 2020 • edited

Codecov Report

diguage commented Apr 7, 2020

ph3636 left a comment

Choose a reason for hiding this comment

ph3636 Apr 8, 2020

Choose a reason for hiding this comment

diguage Apr 8, 2020 • edited

Choose a reason for hiding this comment

ph3636 commented Apr 12, 2020

diguage commented Apr 13, 2020

zjinlei commented Apr 13, 2020

ph3636 commented Apr 13, 2020

diguage commented Apr 13, 2020

diguage commented Apr 13, 2020

ph3636 left a comment

Choose a reason for hiding this comment

zjinlei left a comment

Choose a reason for hiding this comment

slievrly left a comment

Choose a reason for hiding this comment

diguage commented Apr 4, 2020 •

edited

zjinlei commented Apr 4, 2020 •

edited

diguage commented Apr 7, 2020 •

edited

codecov-io commented Apr 7, 2020 •

edited

diguage Apr 8, 2020 •

edited