Repository navigation
Crayfish against other CRC libraries
The speed of crayfish-common-crc 2.1.0 against the other CRC libraries for Java: snksoft/java-crc, ggrandes/crc, Apache Commons Codec, Hadoop and Circe (Apache BookKeeper, Pulsar), and against the classes of the JDK (java.util.zip.CRC32, CRC32C). JMH on JDK 8, 21 and 25, all six CRCs of the catalogue used here (CRC-8/SMBUS, CRC-16/ARC, CRC-32/ISO-HDLC, CRC-32/ISCSI, CRC-64/ECMA-182, CRC-64/XZ), six lengths from 16 bytes to 1 MB. See How it was measured before quoting a number: every point is one round, not the median of several.
-
CRC-32 and CRC-32C on Java 9+ with the classes of the JDK (
CrcModel.of(name, true)): crayfish is on the level ofjava.util.zipitself, because that is what it calls. At 4 KB CRC-32C runs at 29.8 GB/s on JDK 25 and 22.2 GB/s on JDK 21, about 16 and 12 times as fast as the table libraries (Commons Codec, Hadoop: 1.2–1.9 GB/s) and 40–70 times as fast as snksoft/java-crc and ggrandes/crc (0.4–0.5 GB/s). -
On JDK 21 crayfish
jdk=truewas slower than the barejava.util.zipin this run: 15.7 against 20.9 GB/s for CRC-32 and 22.2 against 27.4 GB/s for CRC-32C at 4 KB. One round cannot tell whether this is noise or the alignment effect described on Speed of 1.2.0 -> 2.1.0; on JDK 25 and 8 the difference does not appear. -
Circe is as fast as the JDK for CRC-32 (it calls
CRC32) and for CRC-32C only when the JVM runs with--add-opens java.base/java.util.zip=ALL-UNNAMED; without the flag its CRC-32C is table-driven (0.5–0.6 GB/s at 4 KB). - CRC-8, CRC-16, CRC-64 are calculated with tables by every library. At 4 KB crayfish is 2.5–5.2 times as fast as the others; below 64 bytes the gap shrinks, and for CRC-64/ECMA-182 at 16 bytes ggrandes/crc is faster on JDK 21 and 8 (crayfish takes 1.05–1.3 times as long).
-
JDK 8 has no
CRC32C, so crayfish calculates CRC-32C with its tables even withjdk=true(1.57 GB/s at 4 KB), and Commons Codec and Hadoop are faster there (2.2 GB/s).
The bars are the throughput of one call on a byte[] of 4096 bytes (blue: crayfish, light blue: crayfish with its lookup tables, grey: the others). Models, tables and checksum objects are created once.



snksoft/java-crc needs Java 11+ and is not in the JDK 8 chart; CRC32C of the JDK does not exist there.
crayfish against the slowest and the fastest of the other libraries (all of them calculate with tables): the time of the other library over that of crayfish, above 1 means crayfish is faster.
| CRC | JDK 8 | JDK 21 | JDK 25 |
|---|---|---|---|
| CRC-8/SMBUS | ×2.86–3.57 | ×3.39–4.52 | ×3.17–5.17 |
| CRC-16/ARC | ×3.04–3.27 | ×3.02–3.44 | ×3.40–4.38 |
| CRC-64/ECMA-182 | ×2.56–2.71 | ×2.88–3.49 | ×3.11–3.97 |
| CRC-64/XZ | ×2.58–2.82 | ×2.67–3.09 | ×2.51–3.20 |
The same at 1 MB lies between ×2.4 and ×5.2 on every JDK.
- crayfish-common-crc 2.1.0, snksoft/java-crc 1.1.0, ggrandes/crc (
org.javastack:crc) 1.0.1, Apache Commons Codec 1.23.0 (PureJavaCrc32,PureJavaCrc32C), Hadoop 3.4.1 (PureJavaCrc32,PureJavaCrc32C), Circe 4.17.4, JMH 1.37. - One call calculates the CRC of the whole
byte[](new Random(42).nextBytes), the result goes to JMH.Mode.AverageTime, 3 warm-up iterations of 500 ms, 5 measurement iterations of 500 ms, one fork and one thread per benchmark and length. -
One round per JDK (
ROUNDS=1), without-XX:ObjectAlignmentInBytes=16. Differences of ten percent or so are inside the noise, andCRC32of the JDK, so crayfishjdk=trueand Circe, depends on the address of the array (see Speed of 1.2.0 -> 2.1.0). - JDK 25: Oracle 25.0.4; JDK 21: BellSoft Liberica 21.0.12; JDK 8: Oracle 1.8.0_503. The three are different vendors, so compare the libraries within a JDK rather than the absolute numbers between JDKs.
- All the values of every library were checked against a bit-by-bit reference before the run (33 benchmark methods on JDK 25 and 21, 26 on JDK 8, none failed).
- Circe was measured twice for CRC-32C: as a plain
java -cpuser gets it, and with--add-opens java.base/java.util.zip=ALL-UNNAMED. - The benchmark project (
crc-benchmarks) is a separate Maven project withrun.sh;JAVA=<jdk>/bin/java LABEL=jdk25 ROUNDS=5 ./run.shrepeats a run and writesREPORT.md,summary.csvand the raw JMH JSON.