Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid introducing bookkeeper-common into the pulsar-common #9551

Merged
merged 3 commits into from
Feb 12, 2021

Conversation

zymap
Copy link
Member

@zymap zymap commented Feb 10, 2021


Motivation

Direct using jackson to parse json to avoid introduce the bookkeeper-common
into the pulsar-common. That will make other modules which are using pulsar-common
has some unused bookkeeper dependencies.

---

*Motivation*

Direct using jackson to parse json to avoid introduce the bookkeeper-common
into the pulsar-common. That will make other modules which are using pulsar-common
has some unused bookkeeper dependencies.
@zymap zymap self-assigned this Feb 10, 2021
@zymap zymap changed the title Avoid introduce bookkeeper-common into the pulsar-common Avoid introducing bookkeeper-common into the pulsar-common Feb 10, 2021
@zymap zymap added this to the 2.8.0 milestone Feb 10, 2021
@@ -30,6 +31,8 @@
private final Class policyClass;
private final Map<String, Object> properties;

private static ObjectMapper mapper = new ObjectMapper();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not thread safe and also is not used the preferred configuration for Jackson. pulsar-common has already ObjectMapperFactory.getThreadLocal() which is used for this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the reminder. I change to use the ObjectMapperFactory.getThreadLocal(). Please take a look again.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@merlimat I am not sure I get your point.
AFAIK it is very common to have static fields holding ObjectMapper if you do not need to change the configuration.
Usually you create one ObjectMapper per class or per group of classes (internally it holds a cache)

https://stackoverflow.com/questions/3907929/should-i-declare-jacksons-objectmapper-as-a-static-field

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

btw using consistently ObjectMapperFactory.getThreadLocal() is good to me

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've seen concurrency issues in the past resulting in random exceptions. If I'm not remembering wrong, they were related to the caching of the classes introspection in the object mapper.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@merlimat that makes sense to me.
In fact usually you create a static ObjectMapper when you are sure that it will deal only with a limited and fixed set classes .

in this case the ObjectMapper will deal only with this class, and thus it is safe to use a static instance.,

As just said before, I prefer to have a consistent way of coding, so here in Pulsar is it fine to me to use ObjectManagerFactory. The overhead is to create a ObjectMapper (and the internal cache) per thread instead of having just one per target class.

Copy link
Contributor

@eolivelli eolivelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
I left a suggestion, please take it into consideration


EnsemblePlacementPolicyConfig originalConfig =
new EnsemblePlacementPolicyConfig(MockedEnsemblePlacementPolicy.class, Collections.EMPTY_MAP);
byte[] encodedConfig = originalConfig.encode();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in order to confirm backward compatibility in the future I feel it may have a good value to try to also deserialise a fixed string constant, I mean a value that has not been produced but current version of the code

what about adding such a test ?

  • String encoded = "{.....}"
  • decode
  • assertEquals

@eolivelli
Copy link
Contributor

/pulsarbot run-failure-tests

@merlimat merlimat merged commit 18e61b3 into apache:master Feb 12, 2021
@codelipenghui codelipenghui added the cherry-picked/branch-2.7 Archived: 2.7 is end of life label Feb 18, 2021
codelipenghui pushed a commit that referenced this pull request Feb 18, 2021
* Avoid introduce bookkeeper-common into the pulsar-common
---

*Motivation*

Direct using jackson to parse json to avoid introduce the bookkeeper-common
into the pulsar-common. That will make other modules which are using pulsar-common
has some unused bookkeeper dependencies.

* Fix the build and add some tests

* Address comments

(cherry picked from commit 18e61b3)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cherry-picked/branch-2.7 Archived: 2.7 is end of life release/2.7.1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants