Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Address boundary handling issues in non-blocking raw text reader, and its blocking wrapper #488

Merged
merged 5 commits into from
Mar 17, 2023

Conversation

nirosys
Copy link
Contributor

@nirosys nirosys commented Mar 17, 2023

Issue #, if available: n/a

Description of changes:
Previous implementation of the non-blocking reader, and its blocking wrapper, had two bugs when handling incomplete errors, and continuing after an incomplete error.

The non-blocking reader did not take into account the step_out nesting when unsetting the need_continue. When stepping out of a container that contained a container that had not yet been read, this would result in the reader stepping out beyond the target depth.

The blocking reader did not properly mark the stream as complete when stepping out of a container. If a step_out caused the last remaining data from the stream to be read (completing the stream) root level values at the end of the stream could be ambiguous and continue to cause incomplete data errors, until the stream had been marked complete.


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@nirosys nirosys changed the title Fix two bugs related to read boundary handling Address boundary handling issues in non-blocking raw text reader, and its blocking wrapper Mar 17, 2023
@nirosys nirosys force-pushed the rgiliam/reader_boundary_bugs branch from 4b27705 to 50640bb Compare March 17, 2023 18:10
Copy link
Contributor

@zslayton zslayton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This diff looks good, could you add 1+ unit tests showing it's fixed?

@nirosys
Copy link
Contributor Author

nirosys commented Mar 17, 2023

Absolutely, porting them over from my other branch now.

@codecov
Copy link

codecov bot commented Mar 17, 2023

Codecov Report

Patch coverage: 97.91% and project coverage change: +0.04 🎉

Comparison is base (cffa745) 89.47% compared to head (f8b43e2) 89.52%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #488      +/-   ##
==========================================
+ Coverage   89.47%   89.52%   +0.04%     
==========================================
  Files          76       76              
  Lines       13704    13747      +43     
==========================================
+ Hits        12262    12307      +45     
+ Misses       1442     1440       -2     
Impacted Files Coverage Δ
src/text/raw_text_reader.rs 97.85% <97.87%> (-0.02%) ⬇️
src/text/non_blocking/raw_text_reader.rs 89.51% <100.00%> (+0.34%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@nirosys
Copy link
Contributor Author

nirosys commented Mar 17, 2023

Added a unit test which currently validates the step_out nesting fix when continuing from an incomplete. I do not think it validates the end of stream on step_out anymore (It has been modified since I found the issue). It will take me a bit to repro, in order to get the data, and read sizes, to line up so it exhausts on step_out.

I'll make sure to include that in the format agnostic PR if no one minds its absence here.

Copy link
Contributor

@zslayton zslayton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Can you address the clippy warnings before merging?

@nirosys
Copy link
Contributor Author

nirosys commented Mar 17, 2023

tyvm! Added commit for clippy issues.

@nirosys nirosys merged commit 0c4858c into amazon-ion:main Mar 17, 2023
@nirosys nirosys deleted the rgiliam/reader_boundary_bugs branch April 24, 2024 22:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants