Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NIFI-10865 allow RecordPath's unescapeJson to convert de-serialised JSON Objects into Records #6708

Closed
wants to merge 1 commit into from

Conversation

ChrisSamo632
Copy link
Contributor

@ChrisSamo632 ChrisSamo632 commented Nov 22, 2022

Summary

NIFI-10865 allow RecordPath's unescapeJson to convert de-serialised JSON Objects into Records

NIFI-10865 allow UpdateRecord to replace the Record root for relative paths, e.g. when a RecordPath function is used to modify selected field(s)

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

Pull Request Tracking

  • Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
  • Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

  • Pull Request based on current revision of the main branch
  • Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

  • Build completed using mvn clean install -P contrib-check
    • JDK 8
    • JDK 11
    • JDK 17

Licensing

  • [ ] New dependencies are compatible with the Apache License 2.0 according to the License Policy
  • [ ] New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

  • Documentation formatting appears as expected in rendered files`

Copy link
Contributor

@MikeThomsen MikeThomsen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does not appear to work if the schema contains a child element. Tested this schema and JSON with UpdateRecord using /embedded and unescapeJson(/str) as the configured update and got an exception:

Schema:

{
	"name": "Parent",
	"type": "record",
	"fields": [{
			"name": "str",
			"type": "string"
		},
		{
			"name": "embedded",
			"type": [{
				"name": "EmbeddedRecord",
				"type": "record",
				"fields": [{
						"name": "label",
						"type": "string"
					},
					{
						"name": "child",
						"type": {
							"name": "ChildRecord",
							"type": "record",
							"fields": [{
								"name": "name",
								"type": "string"
							}]
						}
					}
				]
			}, "null"]
		}
	]
}

Data:

{
	"str": "{\"label\":\"Test!\",\"child\":{\"name\":\"Child record!\"}}",
	"embedded": null
}

@ChrisSamo632
Copy link
Contributor Author

ChrisSamo632 commented Jan 14, 2023

@MikeThomsen I rebased on main latest and added a new TestUpdateRecord unit test (testSetNestedRecordWithUnescapeJsonCall) based on the example you provided and it seems to work without issue. Can you take a look and see whether the test is setup any differently to what you were trying?

If you still get an error, can you elaborate with the meagre and/or stacktrace?

…SON Objects into Records

NIFI-10865 allow UpdateRecord to replace the Record root for relative paths, e.g. when a RecordPath function is used to modify selected field(s)
@MikeThomsen
Copy link
Contributor

I don't know what I did that first round, but you were right. I went back and did it over from scratch, and it worked.

asfgit pushed a commit that referenced this pull request Apr 29, 2023
…SON Objects into Records

NIFI-10865 allow UpdateRecord to replace the Record root for relative paths, e.g. when a RecordPath function is used to modify selected field(s)

This closes #6708

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
@asfgit asfgit closed this in 9274d0c Apr 29, 2023
@ChrisSamo632 ChrisSamo632 deleted the NIFI-10865 branch May 2, 2023 16:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants