[ISSUE #9396] Use fastjson2 in all modules

[yx9o]

Fixes #9396 .

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 63.82979% with 17 lines in your changes missing coverage. Please review.

Project coverage is 49.49%. Comparing base (53a3f69) to head (c5455a4).

Files with missing lines	Patch %	Lines
...mq/tools/command/export/ExportMetadataCommand.java	0.00%	3 Missing ⚠️
...mon/fastjson/GenericMapSuperclassDeserializer.java	85.71%	1 Missing and 1 partial ⚠️
...ache/rocketmq/common/utils/FastJsonSerializer.java	0.00%	2 Missing ⚠️
...tmq/remoting/protocol/heartbeat/HeartbeatData.java	0.00%	2 Missing ⚠️
...command/export/ExportMetadataInRocksDBCommand.java	0.00%	2 Missing ⚠️
...s/command/metadata/RocksDBConfigToJsonCommand.java	0.00%	2 Missing ⚠️
...mmand/broker/GetColdDataFlowCtrInfoSubCommand.java	0.00%	1 Missing ⚠️
...tmq/tools/command/export/ExportConfigsCommand.java	0.00%	1 Missing ⚠️
...tmq/tools/command/export/ExportMetricsCommand.java	0.00%	1 Missing ⚠️
.../tools/command/queue/QueryConsumeQueueCommand.java	0.00%	1 Missing ⚠️

Additional details and impacted files

@@              Coverage Diff              @@
##             develop    #9397      +/-   ##
=============================================
+ Coverage      48.14%   49.49%   +1.35%     
- Complexity     12022    12438     +416     
=============================================
  Files           1308     1307       -1     
  Lines          92253    92252       -1     
  Branches       11808    11810       +2     
=============================================
+ Hits           44413    45658    +1245     
+ Misses         42352    41032    -1320     
- Partials        5488     5562      +74     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[fuyou001]

1、Could you provide the performance benchmark data for the original Fastjson 1?
2、Could you provide compatibility test results with Fastjson 1, particularly when the object contains camel-case properties?

[yx9o]

1、Could you provide the performance benchmark data for the original Fastjson 1? 2、Could you provide compatibility test results with Fastjson 1, particularly when the object contains camel-case properties?

Thank you for your review. The data for the two questions are as follows:
1.

Regarding question 2, I encountered the camel case problem of getCId during the replacement test. The solution is to move the @JSONField(name = "c") annotation from the attribute to the get/set method. All the replaced codes are covered by unit tests.

[fuyou001]

Could you develop a tool to validate whether objects with camelCase properties are compatible with Fastjson 2, so we can check compatibility? @yx9o

[yx9o]

Could you develop a tool to validate whether objects with camelCase properties are compatible with Fastjson 2, so we can check compatibility? @yx9o

Let me understand. We need to develop a tool to test all objects in the project that have irregular camel case naming to ensure that fastjson1 and fastjson2 serialization are compatible, right?

[fuyou001]

Could you develop a tool to validate whether objects with camelCase properties are compatible with Fastjson 2, so we can check compatibility? @yx9o

Let me understand. We need to develop a tool to test all objects in the project that have irregular camel case naming to ensure that fastjson1 and fastjson2 serialization are compatible, right?

Yes, but we should only validate classes involved in JSON serialization/deserialization.

[yx9o]

Hi @fuyou001 , please help review again. According to your review, I added the following serialization compatibility test tool class: RemotingSerializableCompatTest. It mainly does the following:

1. Construct objects with default parameters for all subclasses of RemotingSerializable
2. Use fastjson1 to serialize the object obtained in the first step
3. Use fastjson2 to deserialize the json string obtained in the second step
4. Compare the object deserialized in the third step with the object constructed in the first step, and compare all attribute values. Their comparisons are all passed

[fuyou001]

Hi @fuyou001 , please help review again. According to your review, I added the following serialization compatibility test tool class: RemotingSerializableCompatTest. It mainly does the following:

1. Construct objects with default parameters for all subclasses of RemotingSerializable
2. Use fastjson1 to serialize the object obtained in the first step
3. Use fastjson2 to deserialize the json string obtained in the second step
4. Compare the object deserialized in the third step with the object constructed in the first step, and compare all attribute values. Their comparisons are all passed

When running RemotingSerializableCompatTest, I encountered failed classes. Here's the output for your review:
failed:class org.apache.rocketmq.remoting.protocol.body.RegisterBrokerBody
failed:class org.apache.rocketmq.remoting.protocol.header.GetConsumerListByGroupResponseBody
failed:class org.apache.rocketmq.remoting.protocol.route.TopicRouteData
failed:class org.apache.rocketmq.remoting.protocol.statictopic.TopicRemappingDetailWrapper
failed:class org.apache.rocketmq.remoting.protocol.body.SubscriptionGroupList
failed:class org.apache.rocketmq.remoting.protocol.body.GroupList
failed:class org.apache.rocketmq.remoting.protocol.body.ElectMasterResponseBody
failed:class org.apache.rocketmq.remoting.protocol.body.QueryAssignmentResponseBody
failed:class org.apache.rocketmq.remoting.protocol.body.BrokerMemberGroup
failed:class org.apache.rocketmq.remoting.protocol.heartbeat.HeartbeatData
failed:class org.apache.rocketmq.remoting.protocol.body.CheckClientRequestBody
failed:class org.apache.rocketmq.remoting.protocol.body.GetBrokerMemberGroupResponseBody
failed:class org.apache.rocketmq.remoting.protocol.body.ClusterInfo
failed:class org.apache.rocketmq.remoting.protocol.body.QueryConsumeQueueResponseBody
failed:class org.apache.rocketmq.remoting.protocol.body.TopicList
failed:class org.apache.rocketmq.remoting.protocol.body.QuerySubscriptionResponseBody
failed:class org.apache.rocketmq.remoting.protocol.body.ConsumeByWho
failed:class org.apache.rocketmq.remoting.protocol.body.ConsumerRunningInfo
failed:class org.apache.rocketmq.remoting.protocol.body.KVTable
failed:class org.apache.rocketmq.remoting.protocol.body.CreateTopicListRequestBody

[fuyou001]

@yx9o Subclasses of ConfigManager must also be checked

[yx9o]

Hi @fuyou001 , please help review again. According to your review, I added the following serialization compatibility test tool class: RemotingSerializableCompatTest. It mainly does the following:

1. Construct objects with default parameters for all subclasses of RemotingSerializable
2. Use fastjson1 to serialize the object obtained in the first step
3. Use fastjson2 to deserialize the json string obtained in the second step
4. Compare the object deserialized in the third step with the object constructed in the first step, and compare all attribute values. Their comparisons are all passed

When running RemotingSerializableCompatTest, I encountered failed classes. Here's the output for your review: failed:class org.apache.rocketmq.remoting.protocol.body.RegisterBrokerBody failed:class org.apache.rocketmq.remoting.protocol.header.GetConsumerListByGroupResponseBody failed:class org.apache.rocketmq.remoting.protocol.route.TopicRouteData failed:class org.apache.rocketmq.remoting.protocol.statictopic.TopicRemappingDetailWrapper failed:class org.apache.rocketmq.remoting.protocol.body.SubscriptionGroupList failed:class org.apache.rocketmq.remoting.protocol.body.GroupList failed:class org.apache.rocketmq.remoting.protocol.body.ElectMasterResponseBody failed:class org.apache.rocketmq.remoting.protocol.body.QueryAssignmentResponseBody failed:class org.apache.rocketmq.remoting.protocol.body.BrokerMemberGroup failed:class org.apache.rocketmq.remoting.protocol.heartbeat.HeartbeatData failed:class org.apache.rocketmq.remoting.protocol.body.CheckClientRequestBody failed:class org.apache.rocketmq.remoting.protocol.body.GetBrokerMemberGroupResponseBody failed:class org.apache.rocketmq.remoting.protocol.body.ClusterInfo failed:class org.apache.rocketmq.remoting.protocol.body.QueryConsumeQueueResponseBody failed:class org.apache.rocketmq.remoting.protocol.body.TopicList failed:class org.apache.rocketmq.remoting.protocol.body.QuerySubscriptionResponseBody failed:class org.apache.rocketmq.remoting.protocol.body.ConsumeByWho failed:class org.apache.rocketmq.remoting.protocol.body.ConsumerRunningInfo failed:class org.apache.rocketmq.remoting.protocol.body.KVTable failed:class org.apache.rocketmq.remoting.protocol.body.CreateTopicListRequestBody

Hello, I received your response. I have tested this class separately in the current branch and the develop branch. No errors occur. There is no "failed: xx" error output in the class itself. May I ask how you run this test?

[yx9o]

@yx9o Subclasses of ConfigManager must also be checked

ConfigManager's subclasses are in multiple different packages and cannot be tested uniformly using tool classes. I want to do fastjson1 serialization and fastjson2 deserialization compatibility verification in each subclass's Test. Is this OK?

[fuyou001]

Hi @fuyou001 , please help review again. According to your review, I added the following serialization compatibility test tool class: RemotingSerializableCompatTest. It mainly does the following:

1. Construct objects with default parameters for all subclasses of RemotingSerializable
2. Use fastjson1 to serialize the object obtained in the first step
3. Use fastjson2 to deserialize the json string obtained in the second step
4. Compare the object deserialized in the third step with the object constructed in the first step, and compare all attribute values. Their comparisons are all passed

When running RemotingSerializableCompatTest, I encountered failed classes. Here's the output for your review: failed:class org.apache.rocketmq.remoting.protocol.body.RegisterBrokerBody failed:class org.apache.rocketmq.remoting.protocol.header.GetConsumerListByGroupResponseBody failed:class org.apache.rocketmq.remoting.protocol.route.TopicRouteData failed:class org.apache.rocketmq.remoting.protocol.statictopic.TopicRemappingDetailWrapper failed:class org.apache.rocketmq.remoting.protocol.body.SubscriptionGroupList failed:class org.apache.rocketmq.remoting.protocol.body.GroupList failed:class org.apache.rocketmq.remoting.protocol.body.ElectMasterResponseBody failed:class org.apache.rocketmq.remoting.protocol.body.QueryAssignmentResponseBody failed:class org.apache.rocketmq.remoting.protocol.body.BrokerMemberGroup failed:class org.apache.rocketmq.remoting.protocol.heartbeat.HeartbeatData failed:class org.apache.rocketmq.remoting.protocol.body.CheckClientRequestBody failed:class org.apache.rocketmq.remoting.protocol.body.GetBrokerMemberGroupResponseBody failed:class org.apache.rocketmq.remoting.protocol.body.ClusterInfo failed:class org.apache.rocketmq.remoting.protocol.body.QueryConsumeQueueResponseBody failed:class org.apache.rocketmq.remoting.protocol.body.TopicList failed:class org.apache.rocketmq.remoting.protocol.body.QuerySubscriptionResponseBody failed:class org.apache.rocketmq.remoting.protocol.body.ConsumeByWho failed:class org.apache.rocketmq.remoting.protocol.body.ConsumerRunningInfo failed:class org.apache.rocketmq.remoting.protocol.body.KVTable failed:class org.apache.rocketmq.remoting.protocol.body.CreateTopicListRequestBody

Hello, I received your response. I have tested this class separately in the current branch and the develop branch. No errors occur. There is no "failed: xx" error output in the class itself. May I ask how you run this test?

RemotingSerializableCompatTest.testCompatibilityCheck method , I change :

            boolean isSuccess = checkCompatible(instance, clazz);
            if (!isSuccess) {
                System.out.println("failed:" + clazz);
            }