chore: update releases-0.5.0 to latest main branch#1532
Merged
chaokunyang merged 68 commits intoapache:releases-0.5.0from Apr 17, 2024
Merged
chore: update releases-0.5.0 to latest main branch#1532chaokunyang merged 68 commits intoapache:releases-0.5.0from
chaokunyang merged 68 commits intoapache:releases-0.5.0from
Conversation
avoid actions that use node16 https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/ also - only publish if running on main fork - not on user forks
As a native English speaker, the existing text doesn't read properly for me. Blazing is being used as an adverb so should become Blazingly. https://en.wiktionary.org/wiki/blazingly For instance, if you wanted to use `incredible` - you would say `incredibly fast`, not `incredible fast`.
This PR add a `release.py` python script for bump version. In future, we will add other release related function here. Related issue: apache#1389
This PR adds DISCLAIMER and NOTICE for built jars to comply with ASF release policy
The bigdecimal serialization doesn't pass precison, this PR fixed it
This PR closes apache#1404 by: - Upgrade slf4j to 2.0.12 - Using print instead of slf4j for graalvm
…nt (apache#1434) N/A Signed-off-by: LiangliangSui <coolsui.coding@gmail.com>
…tion (apache#1413) ## What does this PR do? This PR standardizes fury cross-language serialization specification. It comes with following changes: - Remove type tag from the protocol since it introduce space and performance overhead to the implementation. The `type tag` version can be seen in https://github.com/apache/incubator-fury/blob/6ea2e0b83d5449d63ca62296ff0dfd67b96c5bc5/docs/protocols/xlang_object_graph_spec.md . - Fury preserves `0~63` for internal types, but let users register type by id from `0`(added by 64 automatically） to setup type mapping between languages. - Streamline the type systems, only `bool/byte/i16/i32/i64/half-float/float/double/string/enum/list/set/map/Duration/Timestamp/decimal/binary/array/tensor/sparse/tensor/arrow/record/batch/arrow/table` are allowed. - Formulized the binary format for above types. - Add type disambiguation: the deserialization are determined by data type in serialized binary and target type jointly. - Introduce meta string encoding algorithm for field name to reduce space cost by 3/8. - Introduce schema consist mode format for struct. - Introduce schema envolution mode for struct: - this mode can embeed meta in the data or share across multiple messages, - it can avoid the cost of type tag comparison in frameworks like protobuf This protocol also supports object inheriance for xlang serializaiton. This is a feature request that users has been discussed for a long time in protobuf/flatbuffer: - google/flatbuffers#4006 - protocolbuffers/protobuf#5645 Although there are some languages such as `rust/golang` doesn't support inheriance, there are many cases only langauges like `java/c#/python/javascript` are involved, and the support for inheriance is not complexed in the protocol level, so we added the inheriance support in the protocol. And in languages such as `rust/golang`, we can use some annotation to mark composition field as parent class for serialization layout, or we can disable inheriance foor such languages at the protocol level. The protocol support polymorphic natively by type id, so I don't include types such as `OneOf/Union`. With this protocol, you can even serialize multiple rust `dyn trait` object which implement same trait., and get exactly the same objects when deserialization. ## Related issue This PR Closes apache#1418 --------- Co-authored-by: Twice <twice@apache.org>
This PR add openj9 ci, since we have openj9 users, see apache#1433 . Class GC in linux distribution are not supported in this PR.
ideally all files should have license headers relates to apache#1441
Since our serialization spec has been formulized, we can publish it to our website now. This PR sync specification files to incubator-fury-website --------- Co-authored-by: LiangliangSui <116876207+LiangliangSui@users.noreply.github.com>
…he#1453) Signed-off-by: LiangliangSui <coolsui.coding@gmail.com>
Closes apache#1445 Closes apache#1444 Closes apache#1443 Closes apache#1440
Extract public Fury method to the `BaseFury` interface and implement them in `ThreadPoolFury` and `ThreadLocalFury`, so end users can avoid using `execute()` and other utility methods. Off topic: further steps might be changing `BaseFury` interface name to `Fury` while existing class renaming to something like `FuryImpl`. `FuryBuilder` should return the new `Fury` interface, so we will hide system methods like `xwriteRef` and others. WDYT?
apache#1469) This PR optimizes readVarInt/readInt performance by reduce code size of readVarInt for better jvm jit
…pache#1470) This PR reduced code size of read long for jvm jit inline: - Reduced readVarLong code size by separating little/big endian - Reduced readLong code size by separating little/big endian - Reduced unsafeGetLong code size by separating little/big endian and generate online
- optimize read char jvm jit inline by separate little/big endian methods - optimize read short jvm jit inline by separate little/big endian methods - generate unsafe get short code online - generate unsafe get char code online
- optimize read float jvm jit inline by separate little/big endian methods - optimize read double jvm jit inline by separate little/big endian methods - generate unsafe get float code online - generate unsafe get double code online
N/A --------- Signed-off-by: LiangliangSui <coolsui.coding@gmail.com>
When PR has an impact on Fury performance, it is recommended to use perf type and provide benchmark data in the PR description. Signed-off-by: LiangliangSui <coolsui.coding@gmail.com>
This reverts commit 9597962 in apache#1468
## What this PR do? This PR implements streaming mode deserialization, which make the protocol between `Fury#serialize(buffer, o)` and `Fury#serialize(stream, o)` consistent, and make the deserialization support real streaming mode to reduce latency, and facilitate the pipeline between deserialization and transfer. The changes inlucde: - Introduce a `FuryStreamReader` to allow buffered streaming reading. - Make `MemoryBuffer` support streaming reading without introduce cost on critical path. - Implements `InputStream` based `FuryStreamReader` for Fury. `java.nio.Channel` based zero-copy is not implemented in this PR. ## Related issues Closes apache#1435 ## Media Benchmark with this PR: ``` Benchmark (bufferType) (objectType) (references) Mode Cnt Score Error Units UserTypeDeserializeSuite.fury_deserialize array MEDIA_CONTENT false thrpt 50 2867721.514 ± 55102.830 ops/s UserTypeDeserializeSuite.fury_deserialize array MEDIA_CONTENT true thrpt 50 1841586.657 ± 46192.190 ops/s UserTypeDeserializeSuite.fury_deserialize directBuffer MEDIA_CONTENT false thrpt 50 2542987.512 ± 49835.129 ops/s UserTypeDeserializeSuite.fury_deserialize directBuffer MEDIA_CONTENT true thrpt 50 1712919.362 ± 26500.068 ops/s ``` Before this PR: 4c60eb9 ``` Benchmark (bufferType) (objectType) (references) Mode Cnt Score Error Units UserTypeDeserializeSuite.fury_deserialize array MEDIA_CONTENT false thrpt 50 2971081.559 ± 52348.975 ops/s UserTypeDeserializeSuite.fury_deserialize array MEDIA_CONTENT true thrpt 50 1938252.031 ± 25077.333 ops/s UserTypeDeserializeSuite.fury_deserialize directBuffer MEDIA_CONTENT false thrpt 50 2511729.964 ± 39278.437 ops/s UserTypeDeserializeSuite.fury_deserialize directBuffer MEDIA_CONTENT true thrpt 50 1714610.534 ± 29778.339 ops/s ``` Baseline(7bb21d0): ``` Benchmark (bufferType) (objectType) (references) Mode Cnt Score Error Units UserTypeDeserializeSuite.fury_deserialize array MEDIA_CONTENT false thrpt 50 2928012.663 ± 65849.325 ops/s UserTypeDeserializeSuite.fury_deserialize array MEDIA_CONTENT true thrpt 50 1937231.421 ± 24616.628 ops/s UserTypeDeserializeSuite.fury_deserialize directBuffer MEDIA_CONTENT false thrpt 50 2539983.720 ± 43379.751 ops/s UserTypeDeserializeSuite.fury_deserialize directBuffer MEDIA_CONTENT true thrpt 50 1711973.100 ± 19195.431 ops/s ``` ## Struct Benchmark This PR ``` Benchmark (bufferType) (objectType) (references) Mode Cnt Score Error Units UserTypeDeserializeSuite.fury_deserialize array STRUCT false thrpt 50 2603998.829 ± 35062.683 ops/s UserTypeDeserializeSuite.fury_deserialize array STRUCT true thrpt 50 3438343.670 ± 48911.824 ops/s UserTypeDeserializeSuite.fury_deserialize directBuffer STRUCT false thrpt 50 2631586.246 ± 34760.277 ops/s UserTypeDeserializeSuite.fury_deserialize directBuffer STRUCT true thrpt 50 3425883.355 ± 40436.642 ops/s ``` Before This PR： ``` Benchmark (bufferType) (objectType) (references) Mode Cnt Score Error Units UserTypeDeserializeSuite.fury_deserialize array STRUCT false thrpt 50 2589559.872 ± 28062.176 ops/s UserTypeDeserializeSuite.fury_deserialize array STRUCT true thrpt 50 3468978.381 ± 37684.130 ops/s UserTypeDeserializeSuite.fury_deserialize directBuffer STRUCT false thrpt 50 2640599.313 ± 36759.090 ops/s UserTypeDeserializeSuite.fury_deserialize directBuffer STRUCT true thrpt 50 3451992.441 ± 51964.331 ops/s ``` Baseline： ``` Benchmark (bufferType) (objectType) (references) Mode Cnt Score Error Units UserTypeDeserializeSuite.fury_deserialize array STRUCT false thrpt 50 1576536.898 ± 54571.741 ops/s UserTypeDeserializeSuite.fury_deserialize array STRUCT true thrpt 50 3194178.195 ± 60680.268 ops/s UserTypeDeserializeSuite.fury_deserialize directBuffer STRUCT false thrpt 50 1641797.023 ± 16222.155 ops/s UserTypeDeserializeSuite.fury_deserialize directBuffer STRUCT true thrpt 50 3163612.265 ± 49756.995 ops/s ```
<!-- **Thanks for contributing to Fury.** **If this is your first time opening a PR on fury, you can refer to [CONTRIBUTING.md](https://github.com/apache/incubator-fury/blob/main/CONTRIBUTING.md).** Contribution Checklist - The **Apache Fury (incubating)** community has restrictions on the naming of pr titles. You can also find instructions in [CONTRIBUTING.md](https://github.com/apache/incubator-fury/blob/main/CONTRIBUTING.md). - Fury has a strong focus on performance. If the PR you submit will have an impact on performance, please benchmark it first and provide the benchmark result here. --> ## What does this PR do? make slf4j optional <!-- Describe the purpose of this PR. --> ## Related issues apache#1485 <!-- Is there any related issue? Please attach here. - #xxxx0 - #xxxx1 - #xxxx2 --> ## Does this PR introduce any user-facing change? <!-- If any user-facing interface changes, please [open an issue](https://github.com/apache/incubator-fury/issues/new/choose) describing the need to do so and update the document if necessary. --> - [ ] Does this PR introduce any public API change? - [ ] Does this PR introduce any binary protocol compatibility change? ## Benchmark <!-- When the PR has an impact on performance (if you don't know whether the PR will have an impact on performance, you can submit the PR first, and if it will have impact on performance, the code reviewer will explain it), be sure to attach a benchmark data here. -->
<!-- **Thanks for contributing to Fury.** **If this is your first time opening a PR on fury, you can refer to [CONTRIBUTING.md](https://github.com/apache/incubator-fury/blob/main/CONTRIBUTING.md).** Contribution Checklist - The **Apache Fury (incubating)** community has restrictions on the naming of pr titles. You can also find instructions in [CONTRIBUTING.md](https://github.com/apache/incubator-fury/blob/main/CONTRIBUTING.md). - Fury has a strong focus on performance. If the PR you submit will have an impact on performance, please benchmark it first and provide the benchmark result here. --> ## What does this PR do? This PR reduce endian check code size in MemoryBuffer and fixed some internal API name. This PR introduce a 10% performance improvement <!-- Describe the purpose of this PR. --> ## Related issues <!-- Is there any related issue? Please attach here. - #xxxx0 - #xxxx1 - #xxxx2 --> ## Does this PR introduce any user-facing change? <!-- If any user-facing interface changes, please [open an issue](https://github.com/apache/incubator-fury/issues/new/choose) describing the need to do so and update the document if necessary. --> - [ ] Does this PR introduce any public API change? - [ ] Does this PR introduce any binary protocol compatibility change? ## Benchmark Serialization: ``` Before: Benchmark (bufferType) (objectType) (references) Mode Cnt Score Error Units UserTypeSerializeSuite.fury_serialize array MEDIA_CONTENT false thrpt 10 3363940.073 ± 86298.538 ops/s Now Benchmark (bufferType) (objectType) (references) Mode Cnt Score Error Units UserTypeSerializeSuite.fury_serialize array MEDIA_CONTENT false thrpt 10 3645145.088 ± 56496.783 ops/s ``` Deserialization: ``` Before: Benchmark (bufferType) (objectType) (references) Mode Cnt Score Error Units UserTypeDeserializeSuite.fury_deserialize array MEDIA_CONTENT false thrpt 10 2263144.547 ± 138780.480 ops/s Now: Benchmark (bufferType) (objectType) (references) Mode Cnt Score Error Units UserTypeDeserializeSuite.fury_deserialize array MEDIA_CONTENT false thrpt 10 2602900.001 ± 32584.112 ops/s ```
add type mapping for xlang serialization --------- Co-authored-by: LiangliangSui <116876207+LiangliangSui@users.noreply.github.com>
<!-- **Thanks for contributing to Fury.** **If this is your first time opening a PR on fury, you can refer to [CONTRIBUTING.md](https://github.com/apache/incubator-fury/blob/main/CONTRIBUTING.md).** Contribution Checklist - The **Apache Fury (incubating)** community has restrictions on the naming of pr titles. You can also find instructions in [CONTRIBUTING.md](https://github.com/apache/incubator-fury/blob/main/CONTRIBUTING.md). - Fury has a strong focus on performance. If the PR you submit will have an impact on performance, please benchmark it first and provide the benchmark result here. --> ## What does this PR do? reduce fury caller stack <!-- Describe the purpose of this PR. --> ## Related issues <!-- Is there any related issue? Please attach here. - #xxxx0 - #xxxx1 - #xxxx2 --> ## Does this PR introduce any user-facing change? <!-- If any user-facing interface changes, please [open an issue](https://github.com/apache/incubator-fury/issues/new/choose) describing the need to do so and update the document if necessary. --> - [ ] Does this PR introduce any public API change? - [ ] Does this PR introduce any binary protocol compatibility change? ## Benchmark <!-- When the PR has an impact on performance (if you don't know whether the PR will have an impact on performance, you can submit the PR first, and if it will have impact on performance, the code reviewer will explain it), be sure to attach a benchmark data here. -->
…ache#1503) ## What does this PR do? This PR add fastpath for read/write small varint in range `[0,127]`. This PR also fix wrong log level setting in apache#1492 <!-- Describe the purpose of this PR. --> ## Related issues <!-- Is there any related issue? Please attach here. - #xxxx0 - #xxxx1 - #xxxx2 --> ## Does this PR introduce any user-facing change? <!-- If any user-facing interface changes, please [open an issue](https://github.com/apache/incubator-fury/issues/new/choose) describing the need to do so and update the document if necessary. --> - [ ] Does this PR introduce any public API change? - [ ] Does this PR introduce any binary protocol compatibility change? ## Benchmark With this PR:  Before this PR:  <!-- When the PR has an impact on performance (if you don't know whether the PR will have an impact on performance, you can submit the PR first, and if it will have impact on performance, the code reviewer will explain it), be sure to attach a benchmark data here. -->
<!-- **Thanks for contributing to Fury.** **If this is your first time opening a PR on fury, you can refer to [CONTRIBUTING.md](https://github.com/apache/incubator-fury/blob/main/CONTRIBUTING.md).** Contribution Checklist - The **Apache Fury (incubating)** community has restrictions on the naming of pr titles. You can also find instructions in [CONTRIBUTING.md](https://github.com/apache/incubator-fury/blob/main/CONTRIBUTING.md). - Fury has a strong focus on performance. If the PR you submit will have an impact on performance, please benchmark it first and provide the benchmark result here. --> ## What does this PR do? <!-- Describe the purpose of this PR. --> Improve javadoc for logging module. ## Related issues <!-- Is there any related issue? Please attach here. - #xxxx0 - #xxxx1 - #xxxx2 --> apache#1497 ## Does this PR introduce any user-facing change? <!-- If any user-facing interface changes, please [open an issue](https://github.com/apache/incubator-fury/issues/new/choose) describing the need to do so and update the document if necessary. --> - [ ] Does this PR introduce any binary protocol compatibility change? - [ ] Does this PR introduce any public API change? ## Benchmark <!-- When the PR has an impact on performance (if you don't know whether the PR will have an impact on performance, you can submit the PR first, and if it will have impact on performance, the code reviewer will explain it), be sure to attach a benchmark data here. --> --------- Signed-off-by: LiangliangSui <coolsui.coding@gmail.com>
<!-- **Thanks for contributing to Fury.** **If this is your first time opening a PR on fury, you can refer to [CONTRIBUTING.md](https://github.com/apache/incubator-fury/blob/main/CONTRIBUTING.md).** Contribution Checklist - The **Apache Fury (incubating)** community has restrictions on the naming of pr titles. You can also find instructions in [CONTRIBUTING.md](https://github.com/apache/incubator-fury/blob/main/CONTRIBUTING.md). - Fury has a strong focus on performance. If the PR you submit will have an impact on performance, please benchmark it first and provide the benchmark result here. --> ## What does this PR do? **Reduce the use of closures to enhance code readability.** We used closures extensively in the past, which could easily trigger Inline-Cache in `V8`, but this practice was detrimental to code readability. Now that the infrastructure is stable, we can replace them with classes to enhance readability without affecting performance. ## Does this PR introduce any user-facing change? There are no user-facing change. ## Benchmark | (index) | serialize | deserialize | |----------|------------|--------------| | fury | 151 | 120 | | protobuf | 29 | 39 | | json | 28 | 35 | --------- Co-authored-by: wangweipeng <weipeng>
<!-- **Thanks for contributing to Fury.** **If this is your first time opening a PR on fury, you can refer to [CONTRIBUTING.md](https://github.com/apache/incubator-fury/blob/main/CONTRIBUTING.md).** Contribution Checklist - The **Apache Fury (incubating)** community has restrictions on the naming of pr titles. You can also find instructions in [CONTRIBUTING.md](https://github.com/apache/incubator-fury/blob/main/CONTRIBUTING.md). - Fury has a strong focus on performance. If the PR you submit will have an impact on performance, please benchmark it first and provide the benchmark result here. --> ## What does this PR do? THis PR fix fix NilLogger creation introduced in apache#1503 <!-- Describe the purpose of this PR. --> ## Related issues <!-- Is there any related issue? Please attach here. - #xxxx0 - #xxxx1 - #xxxx2 --> ## Does this PR introduce any user-facing change? <!-- If any user-facing interface changes, please [open an issue](https://github.com/apache/incubator-fury/issues/new/choose) describing the need to do so and update the document if necessary. --> - [ ] Does this PR introduce any public API change? - [ ] Does this PR introduce any binary protocol compatibility change? ## Benchmark <!-- When the PR has an impact on performance (if you don't know whether the PR will have an impact on performance, you can submit the PR first, and if it will have impact on performance, the code reviewer will explain it), be sure to attach a benchmark data here. -->
## What does this PR do? ## Related issues <!-- Is there any related issue? Please attach here. - #xxxx0 - #xxxx1 - #xxxx2 --> Closes apache#1510 ## Does this PR introduce any user-facing change? <!-- If any user-facing interface changes, please [open an issue](https://github.com/apache/incubator-fury/issues/new/choose) describing the need to do so and update the document if necessary. --> - [ ] Does this PR introduce any public API change? - [ ] Does this PR introduce any binary protocol compatibility change? ## Benchmark <!-- When the PR has an impact on performance (if you don't know whether the PR will have an impact on performance, you can submit the PR first, and if it will have impact on performance, the code reviewer will explain it), be sure to attach a benchmark data here. -->
## What does this PR do? This PR fix license and notice issue apache#1498 according to discussion thread https://lists.apache.org/list.html?general@incubator.apache.org ## Related issues Closes apache#1498 apache#1389 ## Does this PR introduce any user-facing change? <!-- If any user-facing interface changes, please [open an issue](https://github.com/apache/incubator-fury/issues/new/choose) describing the need to do so and update the document if necessary. --> - [ ] Does this PR introduce any public API change? - [ ] Does this PR introduce any binary protocol compatibility change? ## Benchmark <!-- When the PR has an impact on performance (if you don't know whether the PR will have an impact on performance, you can submit the PR first, and if it will have impact on performance, the code reviewer will explain it), be sure to attach a benchmark data here. -->
## What does this PR do? This PR moved ClassDef from `type` to `meta` package for bettern manage meta related classes ## Related issues <!-- Is there any related issue? Please attach here. - #xxxx0 - #xxxx1 - #xxxx2 --> ## Does this PR introduce any user-facing change? <!-- If any user-facing interface changes, please [open an issue](https://github.com/apache/incubator-fury/issues/new/choose) describing the need to do so and update the document if necessary. --> - [ ] Does this PR introduce any public API change? - [ ] Does this PR introduce any binary protocol compatibility change? ## Benchmark <!-- When the PR has an impact on performance (if you don't know whether the PR will have an impact on performance, you can submit the PR first, and if it will have impact on performance, the code reviewer will explain it), be sure to attach a benchmark data here. -->
…1513) ## What does this PR do? This PR fix special char overflow in meta string encoding. The `a-zA-Z._$` are 65 chars, which can't be expressed by 6 bits, which is range `0~63` ## Related issues <!-- Is there any related issue? Please attach here. - #xxxx0 - #xxxx1 - #xxxx2 --> ## Does this PR introduce any user-facing change? <!-- If any user-facing interface changes, please [open an issue](https://github.com/apache/incubator-fury/issues/new/choose) describing the need to do so and update the document if necessary. --> - [ ] Does this PR introduce any public API change? - [ ] Does this PR introduce any binary protocol compatibility change? ## Benchmark <!-- When the PR has an impact on performance (if you don't know whether the PR will have an impact on performance, you can submit the PR first, and if it will have impact on performance, the code reviewer will explain it), be sure to attach a benchmark data here. --> --------- Co-authored-by: LiangliangSui <116876207+LiangliangSui@users.noreply.github.com>
## What does this PR do? This PR renames EnumString to MetaString ## Related issues apache#1514 ## Does this PR introduce any user-facing change? <!-- If any user-facing interface changes, please [open an issue](https://github.com/apache/incubator-fury/issues/new/choose) describing the need to do so and update the document if necessary. --> - [ ] Does this PR introduce any public API change? - [ ] Does this PR introduce any binary protocol compatibility change? ## Benchmark <!-- When the PR has an impact on performance (if you don't know whether the PR will have an impact on performance, you can submit the PR first, and if it will have impact on performance, the code reviewer will explain it), be sure to attach a benchmark data here. -->
<!-- **Thanks for contributing to Fury.** **If this is your first time opening a PR on fury, you can refer to [CONTRIBUTING.md](https://github.com/apache/incubator-fury/blob/main/CONTRIBUTING.md).** Contribution Checklist - The **Apache Fury (incubating)** community has restrictions on the naming of pr titles. You can also find instructions in [CONTRIBUTING.md](https://github.com/apache/incubator-fury/blob/main/CONTRIBUTING.md). - Fury has a strong focus on performance. If the PR you submit will have an impact on performance, please benchmark it first and provide the benchmark result here. --> ## What does this PR do? This PR implements meta string encoding described in [fury java serialization spec](https://fury.apache.org/docs/specification/fury_java_serialization_spec#meta-string) and [xlang serialization spec](https://fury.apache.org/docs/specification/fury_xlang_serialization_spec#meta-string) We have `3/8` space saveing for most string: ```java // utf8 use 30 bytes, we use only 19 bytes assertEquals(encoder.encode("org.apache.fury.benchmark.data").getBytes().length, 19); // utf8 use 12 bytes, we use only 9 bytes. assertEquals(encoder.encode("MediaContent").getBytes().length, 9); ``` The integration with ClassResolver is left in another PR. ## Related issues apache#1240 apache#1413 ## Does this PR introduce any user-facing change? <!-- If any user-facing interface changes, please [open an issue](https://github.com/apache/incubator-fury/issues/new/choose) describing the need to do so and update the document if necessary. --> - [ ] Does this PR introduce any public API change? - [ ] Does this PR introduce any binary protocol compatibility change? ## Benchmark <!-- When the PR has an impact on performance (if you don't know whether the PR will have an impact on performance, you can submit the PR first, and if it will have impact on performance, the code reviewer will explain it), be sure to attach a benchmark data here. -->
) ## What does this PR do? This PR concats meta string len with flags and use varint encoding to encode such info, which can reduce 2 bytes for every meta string at most times. This PPR reduced serialized size a little: 415 bytes -> 407 bytes when class is not registered in java ``` Before this PR: Fury | MEDIA_CONTENT | false | array | 415 | With this PR: Fury | MEDIA_CONTENT | false | array | 407 | ``` ## Related issues Closes apache#1518 Closes apache#1519 Closes apache#1520 Closes apache#1521 ## Does this PR introduce any user-facing change? <!-- If any user-facing interface changes, please [open an issue](https://github.com/apache/incubator-fury/issues/new/choose) describing the need to do so and update the document if necessary. --> - [ ] Does this PR introduce any public API change? - [ ] Does this PR introduce any binary protocol compatibility change? ## Benchmark <!-- When the PR has an impact on performance (if you don't know whether the PR will have an impact on performance, you can submit the PR first, and if it will have impact on performance, the code reviewer will explain it), be sure to attach a benchmark data here. -->
## What does this PR do? This PR concats write classname flag with package name, which can save one byte for unregistred classname writing <!-- Describe the purpose of this PR. --> ## Related issues <!-- Is there any related issue? Please attach here. - #xxxx0 - #xxxx1 - #xxxx2 --> ## Does this PR introduce any user-facing change? <!-- If any user-facing interface changes, please [open an issue](https://github.com/apache/incubator-fury/issues/new/choose) describing the need to do so and update the document if necessary. --> - [ ] Does this PR introduce any public API change? - [ ] Does this PR introduce any binary protocol compatibility change? ## Benchmark <!-- When the PR has an impact on performance (if you don't know whether the PR will have an impact on performance, you can submit the PR first, and if it will have impact on performance, the code reviewer will explain it), be sure to attach a benchmark data here. -->
…he#1526) <!-- **Thanks for contributing to Fury.** **If this is your first time opening a PR on fury, you can refer to [CONTRIBUTING.md](https://github.com/apache/incubator-fury/blob/main/CONTRIBUTING.md).** Contribution Checklist - The **Apache Fury (incubating)** community has restrictions on the naming of pr titles. You can also find instructions in [CONTRIBUTING.md](https://github.com/apache/incubator-fury/blob/main/CONTRIBUTING.md). - Fury has a strong focus on performance. If the PR you submit will have an impact on performance, please benchmark it first and provide the benchmark result here. --> ## What does this PR do? <!-- Describe the purpose of this PR. --> Use "\<!--...--\>" to display the placeholder in [feature_request.md](https://github.com/apache/incubator-fury/blob/main/.github/ISSUE_TEMPLATE/feature_request.md), so that the placeholder does not need to be deleted when the corresponding content is not provided. ## Related issues <!-- Is there any related issue? Please attach here. - #xxxx0 - #xxxx1 - #xxxx2 --> ## Does this PR introduce any user-facing change? <!-- If any user-facing interface changes, please [open an issue](https://github.com/apache/incubator-fury/issues/new/choose) describing the need to do so and update the document if necessary. --> - [ ] Does this PR introduce any public API change? - [ ] Does this PR introduce any binary protocol compatibility change? ## Benchmark <!-- When the PR has an impact on performance (if you don't know whether the PR will have an impact on performance, you can submit the PR first, and if it will have impact on performance, the code reviewer will explain it), be sure to attach a benchmark data here. --> Signed-off-by: LiangliangSui <coolsui.coding@gmail.com>
…ectAndClass API (apache#1502) <!-- **Thanks for contributing to Fury.** **If this is your first time opening a PR on fury, you can refer to [CONTRIBUTING.md](https://github.com/apache/incubator-fury/blob/main/CONTRIBUTING.md).** Contribution Checklist - The **Apache Fury (incubating)** community has restrictions on the naming of pr titles. You can also find instructions in [CONTRIBUTING.md](https://github.com/apache/incubator-fury/blob/main/CONTRIBUTING.md). - Fury has a strong focus on performance. If the PR you submit will have an impact on performance, please benchmark it first and provide the benchmark result here. --> ## What does this PR do? This PR deprecates serializeJavaObjectAndClass/deserializeJavaObjectAndClass API. This APi is vargue and kind of duplicate with `Fury#serialize`. It doesn't provide `BufferCallback` support too. We should make our API minimal and does not bloat it <!-- Describe the purpose of this PR. --> ## Related issues <!-- Is there any related issue? Please attach here. - #xxxx0 - #xxxx1 - #xxxx2 --> ## Does this PR introduce any user-facing change? <!-- If any user-facing interface changes, please [open an issue](https://github.com/apache/incubator-fury/issues/new/choose) describing the need to do so and update the document if necessary. --> - [x] Does this PR introduce any public API change? - [ ] Does this PR introduce any binary protocol compatibility change? ## Benchmark <!-- When the PR has an impact on performance (if you don't know whether the PR will have an impact on performance, you can submit the PR first, and if it will have impact on performance, the code reviewer will explain it), be sure to attach a benchmark data here. -->
…me (apache#1527) ## What does this PR do? This PR supports meta string encoding for classname and package name in fury java serialization. Before this PR a unregistered class serialization: ``` Fury | MEDIA_CONTENT | false | array | 405 | ``` With this PR: ``` Fury | MEDIA_CONTENT | false | array | 390 | ``` This PR also fix buffer streaming readSliInt64 bugs introduced in apache#1451 ## Related issues <!-- Is there any related issue? Please attach here. - #xxxx0 - #xxxx1 - #xxxx2 --> ## Does this PR introduce any user-facing change? <!-- If any user-facing interface changes, please [open an issue](https://github.com/apache/incubator-fury/issues/new/choose) describing the need to do so and update the document if necessary. --> - [ ] Does this PR introduce any public API change? - [ ] Does this PR introduce any binary protocol compatibility change? ## Benchmark <!-- When the PR has an impact on performance (if you don't know whether the PR will have an impact on performance, you can submit the PR first, and if it will have impact on performance, the code reviewer will explain it), be sure to attach a benchmark data here. -->
<!-- **Thanks for contributing to Fury.** **If this is your first time opening a PR on fury, you can refer to [CONTRIBUTING.md](https://github.com/apache/incubator-fury/blob/main/CONTRIBUTING.md).** Contribution Checklist - The **Apache Fury (incubating)** community has restrictions on the naming of pr titles. You can also find instructions in [CONTRIBUTING.md](https://github.com/apache/incubator-fury/blob/main/CONTRIBUTING.md). - Fury has a strong focus on performance. If the PR you submit will have an impact on performance, please benchmark it first and provide the benchmark result here. --> ## What does this PR do? THis pr inline isDeclType in codegen before this pr: ```java boolean isDeclType0 = (value14 & 4) != 4; Serializer serializer2; if (isDeclType0) { serializer2 = serializer0; } else { serializer2 = classResolver.readClassInfo(memoryBuffer6, imageClassInfoHolder).getSerializer(); } ``` with this pr: ``` Serializer serializer2; if ( (value14 & 4) != 4) { serializer2 = serializer0; } else { serializer2 = classResolver.readClassInfo(memoryBuffer6, imageClassInfoHolder).getSerializer(); } ``` <!-- Describe the purpose of this PR. --> ## Related issues <!-- Is there any related issue? Please attach here. - #xxxx0 - #xxxx1 - #xxxx2 --> ## Does this PR introduce any user-facing change? <!-- If any user-facing interface changes, please [open an issue](https://github.com/apache/incubator-fury/issues/new/choose) describing the need to do so and update the document if necessary. --> - [ ] Does this PR introduce any public API change? - [ ] Does this PR introduce any binary protocol compatibility change? ## Benchmark <!-- When the PR has an impact on performance (if you don't know whether the PR will have an impact on performance, you can submit the PR first, and if it will have impact on performance, the code reviewer will explain it), be sure to attach a benchmark data here. -->
chaokunyang
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This PR update
releases-0.5.0branch to lastestmainbranch and bump version to0.5.0.This PR also diabled automatic python release since we don't release python wheel for 0.5.0
Related issues
#1389
Does this PR introduce any user-facing change?
Benchmark