-
Notifications
You must be signed in to change notification settings - Fork 501
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HDDS-10542. Replace remaining GSON usage with Jackson. #6500
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @ArafatKhan2198 for working on this.
...main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainerMetadataInspector.java
Outdated
Show resolved
Hide resolved
...main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainerMetadataInspector.java
Outdated
Show resolved
Hide resolved
...main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainerMetadataInspector.java
Outdated
Show resolved
Hide resolved
...main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainerMetadataInspector.java
Outdated
Show resolved
Hide resolved
...main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainerMetadataInspector.java
Outdated
Show resolved
Hide resolved
.../java/org/apache/hadoop/ozone/container/keyvalue/TestKeyValueContainerMetadataInspector.java
Outdated
Show resolved
Hide resolved
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/events/EventQueue.java
Outdated
Show resolved
Hide resolved
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/events/EventQueue.java
Outdated
Show resolved
Hide resolved
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/events/EventQueue.java
Outdated
Show resolved
Hide resolved
hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/heatmap/TestHeatMapInfo.java
Outdated
Show resolved
Hide resolved
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/events/EventQueue.java
Outdated
Show resolved
Hide resolved
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/events/EventQueue.java
Outdated
Show resolved
Hide resolved
@ArafatKhan2198 Let's remove changes related to |
Ok will remove. But I did try a bunch of things, from the error The InvalidDefinitionException suggests that Jackson attempted to serialize an instance of UnknownFieldSet$Parser, which likely does not have any properties Jackson can serialize directly. Reading a little about it might happen because of protobuf classes. One quick fix i tried to implement was that ➖
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ArafatKhan2198 , thanks for working on this!
- We should add the test methods to
src/test
. - This change is very big (64kb). Please split it into at least two JIRAs. We may have the hadoop-hdds first and then then hadoop-ozone changes.
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/server/JsonUtils.java
Outdated
Show resolved
Hide resolved
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/server/JsonUtils.java
Outdated
Show resolved
Hide resolved
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/server/JsonUtils.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ArafatKhan2198 thanks for working on this patch, few comments. Pls check.
...main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainerMetadataInspector.java
Outdated
Show resolved
Hide resolved
...main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainerMetadataInspector.java
Outdated
Show resolved
Hide resolved
.../java/org/apache/hadoop/ozone/container/keyvalue/TestKeyValueContainerMetadataInspector.java
Show resolved
Hide resolved
.../integration-test/src/test/java/org/apache/hadoop/ozone/recon/TestReconWithOzoneManager.java
Outdated
Show resolved
Hide resolved
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/shell/TestOzoneShellHA.java
Outdated
Show resolved
Hide resolved
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/shell/TestOzoneShellHA.java
Outdated
Show resolved
Hide resolved
...s/src/main/java/org/apache/hadoop/ozone/admin/scm/ResetDeletedBlockRetryCountSubcommand.java
Outdated
Show resolved
Hide resolved
...org/apache/hadoop/hdds/scm/container/common/helpers/DeletedBlocksTransactionInfoWrapper.java
Outdated
Show resolved
Hide resolved
...main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainerMetadataInspector.java
Outdated
Show resolved
Hide resolved
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/conf/HddsConfServlet.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ArafatKhan2198 , thanks for the update! Please see the comments inlined and also https://issues.apache.org/jira/secure/attachment/13068634/6500_review.patch
BTW, please don't use wrap short lines (lines < 80 chars). Our max line length is 120 chars.
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/server/JsonUtils.java
Outdated
Show resolved
Hide resolved
...main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainerMetadataInspector.java
Outdated
Show resolved
Hide resolved
...main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainerMetadataInspector.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ArafatKhan2198 , thanks for the update. The change looks good.
- Please address @devmadhuu 's comment on
isMissing()
. I also have a question there. - Please don't use short lines for the new code.
...main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainerMetadataInspector.java
Outdated
Show resolved
Hide resolved
...main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainerMetadataInspector.java
Outdated
Show resolved
Hide resolved
...main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainerMetadataInspector.java
Outdated
Show resolved
Hide resolved
...main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainerMetadataInspector.java
Outdated
Show resolved
Hide resolved
...main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainerMetadataInspector.java
Outdated
Show resolved
Hide resolved
...main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainerMetadataInspector.java
Outdated
Show resolved
Hide resolved
...main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainerMetadataInspector.java
Outdated
Show resolved
Hide resolved
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/conf/HddsConfServlet.java
Show resolved
Hide resolved
@szetszwo @devmadhuu We should be good to go on this, let me know if there are any other changes. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @ArafatKhan2198 for working on this patch. LGTM +1
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1 the change looks good.
@ArafatKhan2198 , thanks for your contribution! @devmadhuu , thanks a lot for reviewing this! |
(cherry picked from commit f61f56d)
What changes were proposed in this pull request?
This pull request introduces a migration from Gson to Jackson, specifically focusing on replacing Gson's JsonParser with Jackson's
ObjectMapper
. It involves using ObjectMapper'sreadValue()
method to deserialize JSON strings into Java objects. Additionally, the code utilizes Jackson'sJsonNode
to navigate and extract data from JSON structures.Important mention ➖
We also improved the
JsonUtils
class by introducing new utility methods for more flexible JSON processing with Jackson. We addedreadArrayFromReader
to deserialize JSON content from aReader
into Java arrays, andconvertValueOrObjectNode
to efficiently convert any given value into aJsonNode
, ensuring compatibility with various JSON structures. These improvements streamline JSON operations across our project, for both serialization and deserialization tasks.What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-10542
How was this patch tested?
Ran the Unit Tests for the changed classes.
The forked CI also ran green!