Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Handle incomplete errors when parsing escaped string sequences #495

Merged
merged 2 commits into from
Apr 5, 2023
Merged

Handle incomplete errors when parsing escaped string sequences #495

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
aba6b0b
Handle incomplete errors when parsing escaped string sequences
nirosys Mar 28, 2023
3655857
Merge branch 'main' into rgiliam/fatal_escaped_string_error
zslayton Apr 5, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions src/text/non_blocking/raw_text_reader.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1795,4 +1795,32 @@ mod reader_tests {

Ok(())
}

#[test]
fn generate_incomplete_on_truncated_escape() -> IonResult<()> {
let source = "\"123456\\u269b\"";
// ^-- First read stops here. (offset 9)
let mut reader = RawTextReader::new(source.as_bytes()[..10].to_owned());

match reader.next() {
Err(IonError::Incomplete {
position:
Position {
line_column: Some((line, column)),
..
},
..
}) => {
assert_eq!(line, 0); // Line is still 0 since we haven't actually seen a '\n' yet.
assert_eq!(column, 0); // start of the string; the value being parsed.
}
Err(e) => panic!("unexpected error after partial escaped sequence: {e}"),
Ok(item) => {
panic!("unexpected successful parsing of partial escaped sequence data: {item:?}")
}
}
reader.append_bytes(source[10..].as_bytes())?;
next_type(&mut reader, IonType::String, false);
Comment on lines +1824 to +1825
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Am I understanding this correctly? These lines are testing that you can append more data and have the parsing continue successfully, right? If so, it's probably a good idea to assert that we are getting the expected value, not just the expected type.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, I could add that. The primary purpose of the unit test though is to ensure the incomplete error is returned, since that is what the PR is changing. There are other unit tests that validate the parser's behavior when continuing from an incomplete, so imo the append_bytes, and next_type, could even be removed.

This test might be better off testing the escaped sequence (or string) parser directly rather than the reader.

Ok(())
}
}
2 changes: 1 addition & 1 deletion src/text/non_blocking/text_buffer.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -345,7 +345,7 @@ impl TextBuffer<Vec<u8>> {
self.restack();
self.reserve_capacity(length);

let read_buffer = &mut self.data.as_mut_slice()[self.data_end..];
let read_buffer = &mut self.data.as_mut_slice()[self.data_end..(self.data_end + length)];
let bytes_read = source.read(read_buffer)?;
self.data_end += bytes_read;

Expand Down
1 change: 1 addition & 0 deletions src/text/parsers/text_support.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ where
// Return a fatal error.
match parser.parse(remaining) {
Ok((remaining, string_fragment)) => Ok((remaining, string_fragment)),
Err(e @ nom::Err::Incomplete(_)) => Err(e),
Err(e) => fatal_parse_error(remaining, format!("could not parse {label}: {e}")),
}
}
Expand Down