New property (wrap as array) in avro2json converter#85
Closed
joemeszaros wants to merge 1 commit intoapache:masterfrom
Closed
New property (wrap as array) in avro2json converter#85joemeszaros wants to merge 1 commit intoapache:masterfrom
joemeszaros wants to merge 1 commit intoapache:masterfrom
Conversation
Create a new property (wrap as array) in ConvertAvroToJson, which determines how stream of records is exposed: either as a sequence of single Objects (false), writing every Object to a new line, or as an array of Objects. Default value is true, meaning that the Avro content is exposed as a sequence of root-level Object entries. False value is useful, when you want to write your records as single intances to a target component (e.g. Kafka). Let's assume you have an Avro content as stream of records (record1, record2, ...). If wrap as array is false, the converter will expose the records as sequence of single JSON objects: record1 record2 ... recordN Please bear in mind, that the final output is not a valid JSON content. You can then forward this content e.g. to Kafka, where every record will be a single Kafka message. If wrap as array is true, the output looks like this: [record1,record2,...,recordN] It is useful when you want to convert your Avro content to a valid JSON array.
JPercivall
pushed a commit
to JPercivall/nifi
that referenced
this pull request
Apr 23, 2018
This closes apache#85. Signed-off-by: Aldrin Piri <aldrin@apache.org>
iadamcsik
pushed a commit
to iadamcsik/nifi
that referenced
this pull request
Oct 22, 2025
CDPDFX-6143 Add control channel offset and flow change error stream (apache#81) * CDPDFX-6143 Add ability to specify initial offset for control channel * CDPDFX-6143 Adding an error stream for flow change events and allow poll task to move on after some number of failed attempts * Remove requirement of initial control channel offset * Add optional changeChannelOffset to FlowCreatedEvent to allow FD to specify the offset to start from CDPDFX-6146: Adding ProcessGroupRequestRefreshEvent as a FlowControlEvent (apache#82) CDPDFX-6209 Publish component ids after processing flow-created control event (apache#83) CDPDFX-6209 Publish component ids after processing flow-created contr… This closes apache#83 CDPDFX - Additional logging for flow designer debugging CDPDFX-6284: (apache#85) - Ensuring proper PG is used when publishing validation and status. CDPDFX-6258 Switch to using Redis Pub/Sub for publishing feedback (apache#86) * CDPDFX-6258 Switch to using Redis Pub/Sub for publishing feedback * Comment out System.println CDPDFX-6256 - don't reflect flow change events on component removal (apache#87) * CDPDFX-6256 - don't reflect flow change events on component removal * CDPDFX-6256 - remove generics usages; coverage for ports and controller services CDPDFX-6318 Add new feedback event for reporting multiple controller service statuses in one event (apache#88) * CDPDFX-6318 Add new feedback event for reporting multiple controller service statuses in one event * Include referencing services when publishing updated status of the current service status being modified CDPDFX-6257: Removing runtime manifest event (apache#89) * CDPDFX-6257: Removing RuntimeManifestEvent * Finishing removal of runtime manifest publishing * Removing more runtime manifest publishing references * Resolving compile errors This closes apache#89 CDPDFX-6082 Fix gcs dependencies in Iceberg pom and ignore test CDPDFX-6358 Resolve compatible bundles when processing control and change events to avoid ghosted components (apache#90) * CDPDFX-6358 Resolve compatible bundles when processing control and change events to avoid ghosted components * Use current event revision when publishing state change events CDPDFX-6374 Specify mapping/sync options in control handler to sync with the default scheduler instead of retain scheduler (apache#91) This closes apache#91 CDPDFX Rename version control system for NIFI_REGISTRY to REGISTRY_CLIENT (apache#92) This closes apache#92 CDPDFX-6499: Process Group Sync (apache#95) * CDPDFX-6499: Adding process group synchronization event for copy/paste * Stopping components prior to process group synchronization to account for Stop Process Group action * Removing components from flow diff comparison when components are new CDPDFX-6700 Set topLevelGroupId in sync options to the FD process group id (apache#98) CDPDFX - Fix incorrect recursion in StandardFlowChangeErrorHandler that could produce a stack overflow (apache#99) CDPDFX-6783 Allow null to be returned when retrieving a sensitive param value (apache#100) This closes apache#100
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Create a new property (wrap as array) in ConvertAvroToJson, which determines how stream of records is exposed: either as a sequence of single Objects (false), writing every Object to a new line, or as an array of Objects. Default value is true, meaning that the Avro content is exposed as a sequence of root-level Object entries. False value is useful, when you want to write your records as single instances to a target component (e.g. Kafka).
Let's assume you have an Avro content as stream of records (record1, record2, ...). If wrap as array is false, the converter will expose the records as sequence of single JSON objects:
Please bear in mind, that the final output is not a valid JSON content. You can then forward this content e.g. to Kafka, where every record will be a single Kafka message.
If wrap as array is true, the output looks like this:
It is useful when you want to convert your Avro content to a valid JSON array.