Extend sanitiseRow to recurse into arrays of structs#2599
Merged
Conversation
Collaborator
Author
Code reviewNo issues found. Checked for bugs and CLAUDE.md compliance. 🤖 Generated with Claude Code - If this code review was useful, please react with 👍. Otherwise, react with 👎. |
Member
|
@piotrszul Why does this PR include a change to the R library POM? |
johngrimes
added
bug
johngrimes
changed the title
Member
|
@piotrszul Are you finished with this? It looks fine to me, should we merge to |
`sanitiseRow` only handled nested `Row` values but fell through for `scala.collection.Seq` values (how Spark represents array fields), so synthetic fields like `_fid` and null-valued fields leaked into the JSON output whenever a FHIRPath expression returned a type containing an array of structs (e.g. `CodeableConcept.coding`). Adds a new branch that iterates over `Seq` elements, recursively sanitises any `Row` elements, and updates the parent field's `ArrayType` elementType to the sanitised element schema so that `Row.json()` positional mapping remains correct. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Pre-size the ArrayList with the known sequence length, remove a redundant what-comment, and extract the shared coding row fixture into a helper to eliminate copy-paste between two test classes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Locks in that sanitiseRow correctly renders JSON for an array of structs where elements differ in which fields are null, and therefore have different post-sanitisation schemas.
Extract switch-based field dispatch and the array-of-struct branch into separate helper methods so sanitiseRow itself stays simple and the type-dispatch chain is expressed as a Java 21 switch expression. Resolves SonarCloud java:S3776 and java:S6880 findings on SingleInstanceEvaluator.
|
johngrimes
approved these changes
May 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
sanitiseRowinSingleInstanceEvaluatorfell through to theelsebranch forscala.collection.Seqvalues (Spark's representation of array fields), so synthetic fields like_fidand null-valued fields were never stripped from array-of-struct elementsSeqelements, recurses into anyRowinstances, and updates the parent field'sArrayTypeelementType to the sanitised element schema soRow.json()positional mapping stays correctCodeableConcept.coding), schema propagation for arrays, and end-to-end JSON outputTest plan
SingleInstanceEvaluatorTest$SanitiseRowTests.sanitisesElementsInArrayOfStructs— verifies synthetic and null-valued fields are stripped from array-of-struct elementsSingleInstanceEvaluatorTest$SanitiseRowTests.updatesParentSchemaForSanitisedArrayOfStructs— verifies the parentArrayTypeelementType is updated after sanitisationSingleInstanceEvaluatorTest$RowToJsonTests.jsonCorrectlyRendersArrayOfStructsAfterSanitisation— verifies the JSON output excludes all synthetic and null-valued fields from array elementsSingleInstanceEvaluatorTesttests continue to pass (35/35)🤖 Generated with Claude Code