-
Notifications
You must be signed in to change notification settings - Fork 3.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a JSON RecordBuilder to the GenericJsonSchema #10052
Add a JSON RecordBuilder to the GenericJsonSchema #10052
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nice work
I left a few comments
...client/src/main/java/org/apache/pulsar/client/impl/schema/generic/JsonRecordBuilderImpl.java
Show resolved
Hide resolved
...client/src/main/java/org/apache/pulsar/client/impl/schema/generic/JsonRecordBuilderImpl.java
Outdated
Show resolved
Hide resolved
...client/src/main/java/org/apache/pulsar/client/impl/schema/generic/JsonRecordBuilderImpl.java
Outdated
Show resolved
Hide resolved
pulsar-client/src/test/java/org/apache/pulsar/client/impl/schema/JSONSchemaTest.java
Outdated
Show resolved
Hide resolved
pulsar-client/src/test/java/org/apache/pulsar/client/impl/schema/JSONSchemaTest.java
Show resolved
Hide resolved
...client/src/main/java/org/apache/pulsar/client/impl/schema/generic/JsonRecordBuilderImpl.java
Outdated
Show resolved
Hide resolved
...client/src/main/java/org/apache/pulsar/client/impl/schema/generic/JsonRecordBuilderImpl.java
Outdated
Show resolved
Hide resolved
pulsar-client/src/test/java/org/apache/pulsar/client/impl/schema/JSONSchemaTest.java
Outdated
Show resolved
Hide resolved
pulsar-client/src/test/java/org/apache/pulsar/client/impl/schema/JSONSchemaTest.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
please remove the "star import", then the patch is good to go from my point of view.
@sijie @codelipenghui can you please take a look ?
pulsar-client/src/test/java/org/apache/pulsar/client/impl/schema/JSONSchemaTest.java
Outdated
Show resolved
Hide resolved
10cc372
to
44759d9
Compare
private static ObjectMapper objectMapper = new ObjectMapper(); | ||
|
||
private final GenericSchemaImpl genericSchema; | ||
private Map<String, Object> map = new HashMap<>(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not use the ObjectNode directly? Use the ObjectMapper to generate the JsonNode also create an internal map there,
Looks like:
ObjectNode node = JsonNodeFactory.instance.objectNode();
node.putPOJO("a", "a");
So that we don't need to create a HashMap for each JsonRecoredBuilder.
And we can reuse the ObjectNode by adding it to the ThreadLocal? We can call ObjectNode.removeAll() after build the GenericRecord.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@codelipenghui
I prefer to not use ThreadLocal, as it will complicate the usage, as the user is not able to have two builder on the same thread, with the risk of mixing data from two different records.
We can have a simple implementation now and enhance it when needed.
Creating just one object is not a big deal for the JVM.
we do not have such complexity for the AVRO record builder
Line 29 in 5955930
class AvroRecordBuilderImpl implements GenericRecordBuilder { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Jackson already have a pretty good json node factory to build json node. We should use it instead of building the logic again that Jackson already did.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The ObjectNode builder also uses a Map to build a JsonNode (a LinkedHashMap), but requires to call the a dedicated put() method for each native types or putPOJO() for Objects (like in such a code).
The ObjectMapper.valueToTree(map) produce the same result, a JsonNode, and the underlying serialiser properly deals with native and POJO objects in the same way without serializing it into "basic" nodes. The ObjectNode allows to filter or alter nodes, but we don't need here in the build() method, so the ObjectMapper.valueToTree() it is simple and efficient here.
Moreover, the ObjectMapper is thread-safe and a static instance is enough to do the job.
...client/src/main/java/org/apache/pulsar/client/impl/schema/generic/JsonRecordBuilderImpl.java
Show resolved
Hide resolved
private static ObjectMapper objectMapper = new ObjectMapper(); | ||
|
||
private final GenericSchemaImpl genericSchema; | ||
private Map<String, Object> map = new HashMap<>(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Jackson already have a pretty good json node factory to build json node. We should use it instead of building the logic again that Jackson already did.
7977f0e
to
ae19f43
Compare
@sijie as you told that you are okay with this patch, can you please click on "Approve" ? otherwise we cannot merge it |
6558e5b
to
23cb3ad
Compare
23cb3ad
to
40889a5
Compare
Provide a JSON GenericRecord builder allowing to produce JsonGenericRecord from the GenericJsonSchema. Add a new class org.apache.pulsar.client.impl.schema.generic.JsonRecordBuilderImpl Modify the org.apache.pulsar.client.impl.schema.generic.GenericJsonSchema.newRecordBuilder() Add a unit test in org.apache.pulsar.client.impl.schema.JSONSchemaTest (cherry picked from commit 08cbdc7)
Provide a JSON GenericRecord builder allowing to produce JsonGenericRecord from the GenericJsonSchema. Add a new class org.apache.pulsar.client.impl.schema.generic.JsonRecordBuilderImpl Modify the org.apache.pulsar.client.impl.schema.generic.GenericJsonSchema.newRecordBuilder() Add a unit test in org.apache.pulsar.client.impl.schema.JSONSchemaTest
Motivation
Provide a JSON GenericRecord builder allowing to produce JsonGenericRecord from the GenericJsonSchema.
Modifications
Add a new class org.apache.pulsar.client.impl.schema.generic.JsonRecordBuilderImpl
Modify the org.apache.pulsar.client.impl.schema.generic.GenericJsonSchema.newRecordBuilder()
Verifying this change
Add a unit test in org.apache.pulsar.client.impl.schema.JSONSchemaTest