Skip to content

Comments

feat(java): add fast util serializers#3154

Closed
Pigsy-Monk wants to merge 1 commit intoapache:mainfrom
Pigsy-Monk:zwsong
Closed

feat(java): add fast util serializers#3154
Pigsy-Monk wants to merge 1 commit intoapache:mainfrom
Pigsy-Monk:zwsong

Conversation

@Pigsy-Monk
Copy link
Contributor

Why?

FastUtilSerializer provides optimized serialization for fastutil collections (such as ShortOpenHashSet, ShortArrayList, etc.), which are commonly used in high-performance applications. However, the existing implementation lacked comprehensive unit tests and performance benchmarks to validate its correctness and demonstrate the optimization benefits.

This PR adds:

  1. Comprehensive unit tests for FastUtilSerializer to ensure correctness
  2. Performance tests comparing FastUtilSerializer with Fory's default serializer
  3. Documentation of the optimization benefits, particularly for ShortSet serialization

What does this PR do?

1. Added Test Dependencies

  • Added fastutil (version 8.5.6) as a test dependency in fory-core/pom.xml

2. Created Unit Tests (FastUtilSerializerTest.java)

Comprehensive test coverage including:

  • testShortOpenHashSet - Tests serialization/deserialization of ShortOpenHashSet
  • testShortArrayList - Tests ShortArrayList serialization
  • testIntArrayList, testLongArrayList, testByteArrayList - Tests for various primitive array lists
  • testObject2ObjectOpenHashMap - Tests map serialization
  • testRegisterAll - Tests the registerAll() method
  • Empty collection tests
  • Large dataset tests

3. Created Performance Tests (FastUtilShortSetPerformanceTest.java)

Performance comparison tests that demonstrate:

  • Size comparison: Compares serialized size between default serializer and FastUtilSerializer for different data sizes (10, 100, 1000, 10000 elements)
  • Detailed optimization analysis: Shows bytes saved and compression ratio
  • Sparse data testing: Tests with large gaps between values
  • Dense data testing: Tests with consecutive values
  • Performance benchmark: Measures serialization/deserialization time and speedup

4. Test Results

The performance tests validate that:

  • FastUtilSerializer provides equal or better serialization efficiency compared to the default serializer
  • Serialization/deserialization correctness is maintained
  • The optimization benefits are measurable and documented

Related issues

@Pigsy-Monk Pigsy-Monk force-pushed the zwsong branch 5 times, most recently from c2ae53d to ea4aa17 Compare January 17, 2026 04:34
@chaokunyang
Copy link
Collaborator

chaokunyang commented Jan 17, 2026

@Pigsy-Monk Thanks for contributing this feature. But we can't merge this. We're working hard to minimize our library size, we can't introduce a new dependency.

@Pigsy-Monk
Copy link
Contributor Author

Hi, the dependency to fast util is only for test. And so the scope of dependency is also for test only. And I found it's already in fory-testsuite module. I can move the two tests there and leave the code in fory-core . @chaokunyang

@chaokunyang
Copy link
Collaborator

We have no plans to introduce serializer for third party serializers in fory-core currently. You can publish a separate jar for those serializer by yourself. Or implement Externalizable for those types in fast util directly. Fory has excellent performance for Externalizable too.

@Pigsy-Monk
Copy link
Contributor Author

Ok I see thanks for replying.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants