Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed bugs in recursion depth and added a path example #761

Merged
merged 1 commit into from
Jul 21, 2023

Conversation

bashir2
Copy link
Collaborator

@bashir2 bashir2 commented Jul 19, 2023

Description of what I changed

This fixes some of the issues mentioned in #757. It also adds a path example where the recursion depth limit is disabled. The example is commented out until we decide we really want to support this pattern. If we do, then those exceptional paths should be exposed as configuration parameters.

E2E test

TESTED:

Ran the pipeline with a sample QuestionnaireResponse that had deep item.item.item... recursions and confirmed that all of the answer fields are transformed into the Parquet file without increasing the general recursion max-depth, if that path is provided as an exception.

Also tested the same scenario by increasing max-depth and no exceptional paths; at levels 1, 2, 3, etc. the expected fields were present.

Checklist: I completed these to help reviewers :)

  • I have read and will follow the review process.

  • I am familiar with Google Style Guides for the language I have coded in.

    No? Please take some time and review Java and Python style guides.

  • My IDE is configured to follow the Google code styles.

    No? Unsure? -> configure your IDE.

  • I have added tests to cover my changes. (If you refactored existing code that was well tested you do not have to add tests)

  • I ran mvn clean package right before creating this pull request and added all formatting changes to my commit.

  • All new and existing tests passed.

  • My pull request is based on the latest changes of the master branch.

    No? Unsure? -> execute command git pull --rebase upstream master

@bashir2 bashir2 requested a review from atulai-sg July 19, 2023 19:14
Copy link
Collaborator Author

@bashir2 bashir2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review.

Copy link
Collaborator

@atulai-sg atulai-sg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this fix, Bashir!

@bashir2 bashir2 merged commit 892ad88 into google:master Jul 21, 2023
5 checks passed
@bashir2 bashir2 deleted the maxDepth branch July 25, 2023 19:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants