Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade jackson and jsonrpc libs #1883

Merged
merged 3 commits into from Nov 17, 2022
Merged

Conversation

rmoreliovlabs
Copy link
Contributor

@rmoreliovlabs rmoreliovlabs commented Sep 18, 2022

Description

Upgrade jackson and jsonrpc libs.

Jackson ObjectMapper is handling null values in this new version and some tests were expecting NullPointerExceptions and JsonMappingExceptions. I just made those methods expecting Jackson Object Mapper exceptions to have a similar behavior to keep it working as previously expected.

Motivation and Context

We are using an old version of com.fasterxml.jackson.core:jackson-databind, currently set to 2.8.7, being updated to 2.8.11 on scope of Java upgrade effort. There is already a 2.13 version and 2.14 is coming soon.

The reason for not updating now is that we’ve detected some changes on method outputs. As an example, ObjectMapper.readTree is now returning null in cases when it was throwing an NullPointerException (tested on co.rsk.config.RemascConfigFactoryTest#createRemascConfigInvalidConfig). Even if code compiles and tests pass (after fixing the previously mentioned difference), we could be creating some bugs that would remain unnoticed until runtime, as we do a wide usage of this library.

It looks like the change was introduced on 2.10.0, as everything seemed to be working fine up to 2.9.10. According to Jackson release strategy, minor versions should not introduce any breaking change, so probably this was a bugfix or alike. However, we think this upgrade would need further and deeper review just in case.

Update

After investigating a bit, it looks like we might also have an incompatibility with com.github.briandilley.jsonrpc4j, that depends on Jackson. This library could only be updated to 1.5.3, as with latest 1.6 it was depending on Jackson's 2.10.2 and causing the same issue mentioned above even when keeping Jackson’s declared dependency at original 2.8.7

How Has This Been Tested?

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • Tests for the changes have been added (for bug fixes / features)
  • Requires Activation Code (Hard Fork)

fed:upgrade-jackson-and-json-rpc-libs

Copy link

@illuque-iov illuque-iov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you run the tests after the change?

This one, among other, is failing for me: co.rsk.config.RemascConfigFactoryTest#createRemascConfigInvalidConfig

It most probably will fail also in CI/CD when the Gradle verification issue is fixed.

@rmoreliovlabs rmoreliovlabs force-pushed the upgrade-jackson-and-json-rpc-libs branch from d4a6f6b to e54d3ae Compare September 20, 2022 04:35
@rmoreliovlabs rmoreliovlabs added the WIP Work in progress label Sep 22, 2022
@rmoreliovlabs rmoreliovlabs force-pushed the upgrade-jackson-and-json-rpc-libs branch from e54d3ae to 314c556 Compare September 22, 2022 02:24
@rmoreliovlabs rmoreliovlabs removed the WIP Work in progress label Sep 22, 2022
@rmoreliovlabs
Copy link
Contributor Author

Have you run the tests after the change?

This one, among other, is failing for me: co.rsk.config.RemascConfigFactoryTest#createRemascConfigInvalidConfig

It most probably will fail also in CI/CD when the Gradle verification issue is fixed.

The previous behavior of the library produced a nullpointer exception when you sent null to public <T> T treeToValue(TreeNode n, Class<T> valueType) in the n argument. New version's implementation checks for null and if n is null then returns null hence avoid NullPointerException to happen.

I tried to fix this issue by following the same behavior after treeToValue is executed.

@rmoreliovlabs rmoreliovlabs force-pushed the upgrade-jackson-and-json-rpc-libs branch 3 times, most recently from bb1b60b to 8d95b96 Compare September 22, 2022 04:42
@rmoreliovlabs
Copy link
Contributor Author

pipeline:run

Copy link

@illuque-iov illuque-iov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a general note, IMO the wrapper we discussed is not needed, we only call mapper.treeToValue from 3 places in the production code, I think we should deal with the new behavior from Jackson.

@rmoreliovlabs rmoreliovlabs force-pushed the upgrade-jackson-and-json-rpc-libs branch from e983963 to 4c18c64 Compare September 22, 2022 20:42
Copy link

@illuque-iov illuque-iov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO wrapping just to keep previous library behavior is not ideal, if Jackson changed it is probably for a reason. I would instead let each calling place decide what to do based on business rules. But no big deal, if you prefer this approach I'm ok.

But I still think we could now fix the "problem" with the NPE I've mentioned here.

@Vovchyk
Copy link
Contributor

Vovchyk commented Sep 23, 2022

IMO wrapping just to keep previous library behavior is not ideal, if Jackson changed it is probably for a reason. I would instead let each calling place decide what to do based on business rules. But no big deal, if you prefer this approach I'm ok.

But I still think we could now fix the "problem" with the NPE I've mentioned here.

imho, I'd try to preserve compatibility with the previous versions of the api, unless there's another reason not to do it.

rewriting each place where null's may occur is error prone, so perhaps keeping this check for null in one place is simpler

@illuque-iov
Copy link

illuque-iov commented Sep 23, 2022

IMO wrapping just to keep previous library behavior is not ideal, if Jackson changed it is probably for a reason. I would instead let each calling place decide what to do based on business rules. But no big deal, if you prefer this approach I'm ok.
But I still think we could now fix the "problem" with the NPE I've mentioned here.

imho, I'd try to preserve compatibility with the previous versions of the api, unless there's another reason not to do it.

rewriting each place where null's may occur is error prone, so perhaps keeping this check for null in one place is simpler

Yeah, I generally agree, I explained myself wrong probably. I meant wrapping was not ideal in this scenario, where wrapped method is only being called in 3 places and where the check for null that we are doing is just throwing manually a NPE because the library was doing that before. For example, doing this is preventing us from properly handling the exception in these two places, that's why I thing it should me manually handled:

  1. Not releasing content as mentioned here
  2. Not returning anything to the caller as mentioned here

@rmoreliovlabs rmoreliovlabs force-pushed the upgrade-jackson-and-json-rpc-libs branch from 4c18c64 to b40e150 Compare September 23, 2022 13:39
@rmoreliovlabs rmoreliovlabs force-pushed the upgrade-jackson-and-json-rpc-libs branch from 251b033 to 1b88f83 Compare September 26, 2022 15:46
@sonarcloud
Copy link

sonarcloud bot commented Sep 26, 2022

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 1 Code Smell

94.4% 94.4% Coverage
0.0% 0.0% Duplication

@rmoreliovlabs rmoreliovlabs force-pushed the upgrade-jackson-and-json-rpc-libs branch 2 times, most recently from ab642b9 to bc4065c Compare November 9, 2022 20:50
@Vovchyk
Copy link
Contributor

Vovchyk commented Nov 10, 2022

pipeline:run

@rmoreliovlabs
Copy link
Contributor Author

pipeline:run

@rmoreliovlabs rmoreliovlabs force-pushed the upgrade-jackson-and-json-rpc-libs branch from bc4065c to d305110 Compare November 10, 2022 22:43
@sonarcloud
Copy link

sonarcloud bot commented Nov 10, 2022

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 15 Code Smells

94.4% 94.4% Coverage
0.0% 0.0% Duplication

@Vovchyk
Copy link
Contributor

Vovchyk commented Nov 11, 2022

pipeline:run

9 similar comments
@rmoreliovlabs
Copy link
Contributor Author

pipeline:run

@rmoreliovlabs
Copy link
Contributor Author

pipeline:run

@rmoreliovlabs
Copy link
Contributor Author

pipeline:run

@Vovchyk
Copy link
Contributor

Vovchyk commented Nov 14, 2022

pipeline:run

@rmoreliovlabs
Copy link
Contributor Author

pipeline:run

@Vovchyk
Copy link
Contributor

Vovchyk commented Nov 15, 2022

pipeline:run

@rmoreliovlabs
Copy link
Contributor Author

pipeline:run

@rmoreliovlabs
Copy link
Contributor Author

pipeline:run

@rmoreliovlabs
Copy link
Contributor Author

pipeline:run

@Vovchyk Vovchyk merged commit 511b5be into master Nov 17, 2022
@Vovchyk Vovchyk deleted the upgrade-jackson-and-json-rpc-libs branch November 17, 2022 09:28
@aeidelman aeidelman added this to the Hop 4.2.0 milestone Jan 26, 2023
@aeidelman aeidelman modified the milestones: Hop 4.2.0, Hop 4.3.0 Mar 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants