[SPARK-51806][BUILD] Upgrade kryo-shaded to 4.0.3#50586
[SPARK-51806][BUILD] Upgrade kryo-shaded to 4.0.3#50586LuciferYang wants to merge 6 commits intoapache:masterfrom
kryo-shaded to 4.0.3#50586Conversation
| <artifactId>kryo-shaded</artifactId> | ||
| <version>${kryo.version}</version> | ||
| </dependency> | ||
| <dependency> |
There was a problem hiding this comment.
Kryo 4.0.x depends on version objenesis 2.5.1 , while Spark currently depends on objenesis 3.3. Here, it is uniformly defined as version 3.3.
There was a problem hiding this comment.
I don't think it's necessary to use exclusion. But just to be on the safe side, let me exclude it
|
cc @yaooqinn Let's test 4.0.3 first |
LuciferYang
changed the title
kryo-shaded to 4.0.3
|
will update bench result later |
| basicTypes: Double with unsafe:false 222 225 2 4.5 221.9 0.7X | ||
| Array: Int with unsafe:false 13 14 1 78.0 12.8 12.9X | ||
| Array: Long with unsafe:false 21 21 1 48.2 20.8 8.0X | ||
| Array: Float with unsafe:false 6 6 0 178.9 5.6 29.6X |
There was a problem hiding this comment.
Double in throughput 🤔
Are we are expecting this kind of improvement on a hotfix bump I wonder.
There was a problem hiding this comment.
@hiufung-kwok Judging from the historical update records, the current data seems relatively normal, whereas the previous update results may have had some unknown errors. It is also possible that there are certain unstable factors affecting the results of this test case, which could be worth investigating.
|
friendly ping @yaooqinn |
|
Merged to master. Can you backport @LuciferYang since it's a bugfix release of Kryo? |
### What changes were proposed in this pull request? This PR upgrades `kryo-shaded` from 4.0.2 to 4.0.3. ### Why are the changes needed? To bring in the latest patch release of `kryo-shaded`. - EsotericSoftware/kryo@kryo-parent-4.0.2...kryo-parent-4.0.3 - EsotericSoftware/kryo#651 - EsotericSoftware/kryo#850 - EsotericSoftware/kryo#978 In addition, Apache Spark 4.0+ has been using `4.0.3`. - apache/spark#50586 ### How was this patch tested? Pass the CIs. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (claude-opus-4-6) Closes #2587 from dongjoon-hyun/ORC-2137. Authored-by: Dongjoon Hyun <dongjoon@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
### What changes were proposed in this pull request? This PR upgrades `kryo-shaded` from 4.0.2 to 4.0.3. ### Why are the changes needed? To bring in the latest patch release of `kryo-shaded`. - EsotericSoftware/kryo@kryo-parent-4.0.2...kryo-parent-4.0.3 - EsotericSoftware/kryo#651 - EsotericSoftware/kryo#850 - EsotericSoftware/kryo#978 In addition, Apache Spark 4.0+ has been using `4.0.3`. - apache/spark#50586 ### How was this patch tested? Pass the CIs. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (claude-opus-4-6) Closes #2587 from dongjoon-hyun/ORC-2137. Authored-by: Dongjoon Hyun <dongjoon@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org> (cherry picked from commit 1d5387e) Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
3 participants
What changes were proposed in this pull request?
This pr aims to upgrade
kryo-shadedfrom 4.0.2 to 4.0.3.Why are the changes needed?
This version brings some bug fixes and performance improvements related to chunked encoding:
full changes as follows:
Does this PR introduce any user-facing change?
No
How was this patch tested?
Was this patch authored or co-authored using generative AI tooling?
No