Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Xml_generator: prevent unnecessary new lines #2967

Closed
m-stein opened this issue Sep 7, 2018 · 9 comments
Closed

Xml_generator: prevent unnecessary new lines #2967

m-stein opened this issue Sep 7, 2018 · 9 comments
Assignees
Labels

Comments

@m-stein
Copy link
Member

m-stein commented Sep 7, 2018

The handling of exceptions in the Xml_generator::Node constructor leaves unwanted new lines in the XML string.

@m-stein m-stein added the bug label Sep 7, 2018
@m-stein m-stein self-assigned this Sep 7, 2018
@nfeske
Copy link
Member

nfeske commented Sep 7, 2018

This issue title, description, and label leaves me puzzled. It suggests that something fundamental is broken and must be fixed but remains completely unspecific about the symptom and nature of the problem.

If the issue is about the topic we discussed offline just yesterday, an appropriate title would be "Xml_generator: improve formatting on exception" with the label "cleanup" attached. From what I know, the issue is merely about a formatting improvement in the exception case, which happened to become a relevant in your case where exceptions are used in a non-exceptional way.

I want to encourage you to spend a little more effort on meaningful issue titles, descriptions, and labels. An issue should at least be specific about the symptom and the expected behavior. Please keep in mind that the issue tracker is part of our project's documentation.

@m-stein m-stein added the fixed label Sep 7, 2018
m-stein added a commit to m-stein/genode that referenced this issue Sep 7, 2018
1) The loop for determining the line length read from a character offset
   before checking whether the offset is smaller than the given string
   length. This could have caused access outside the string buffer.

2) The routine for determining the line length first seeked for the
   offset of the last real character of the line and than added one for
   getting the length but only if the following character was '\n'. This
   has to be done for any other line-terminating character too. The only
   case where you don't want to do this is when the end of the whole
   string is reached.

Issue genodelabs#2967
m-stein added a commit to m-stein/genode that referenced this issue Sep 7, 2018
When the functor provided to the Node constructor throws an exception,
do revert all changes in reverse order. Previously, the changes made
to the parent node were not considered by the exception handler which
caused unnecessary characters to remain in the out buffer for each
reverted node.

Issue genodelabs#2967
m-stein added a commit to m-stein/genode that referenced this issue Sep 7, 2018
Previously, a '\0' was appended to the out buffer at the end of each
call of the Node constructor. At the end of the Xml_generator
constructor, at the other hand, '\n' was appended. Without exceptions in
the Node constructor this led to "\0\n" at the end of generated XML.
However, with exceptions, the '\0' of the last Node constructor call was
sometimes overwritten, leading to a mere "\n" at the end. It is better
to append '\0' only in one place, namely at the point where the '\n' was
appended in the Xml_generator constructor. This also fixes the problem
with the exceptions.

This also extends the xml_generator test to drive a harder test on
exceptions in the Xml_generator.

Fixes genodelabs#2967
@m-stein
Copy link
Member Author

m-stein commented Sep 7, 2018

Fixed:
ad988b1 Xml_generator: fix and test missing '\0'
fc88d6f Xml_generator: fix exception handling in Node(...)
dc32bf4 print_lines: fix bugs in line length calculation

@m-stein
Copy link
Member Author

m-stein commented Sep 7, 2018

@nfeske Sorry for that. I thought it would be fixed right away but it turned out to be more time-intensive.

@m-stein m-stein changed the title Xml_generator: fix exception handling in Node(...) Xml_generator: fix unnecessary new lines Sep 7, 2018
@nfeske
Copy link
Member

nfeske commented Sep 7, 2018

The commits are very insightful. Thanks for the good commit messages and the extended tests.

chelmuth pushed a commit that referenced this issue Sep 10, 2018
1) The loop for determining the line length read from a character offset
   before checking whether the offset is smaller than the given string
   length. This could have caused access outside the string buffer.

2) The routine for determining the line length first seeked for the
   offset of the last real character of the line and than added one for
   getting the length but only if the following character was '\n'. This
   has to be done for any other line-terminating character too. The only
   case where you don't want to do this is when the end of the whole
   string is reached.

Issue #2967
chelmuth pushed a commit that referenced this issue Sep 10, 2018
When the functor provided to the Node constructor throws an exception,
do revert all changes in reverse order. Previously, the changes made
to the parent node were not considered by the exception handler which
caused unnecessary characters to remain in the out buffer for each
reverted node.

Issue #2967
chelmuth pushed a commit that referenced this issue Sep 10, 2018
Previously, a '\0' was appended to the out buffer at the end of each
call of the Node constructor. At the end of the Xml_generator
constructor, at the other hand, '\n' was appended. Without exceptions in
the Node constructor this led to "\0\n" at the end of generated XML.
However, with exceptions, the '\0' of the last Node constructor call was
sometimes overwritten, leading to a mere "\n" at the end. It is better
to append '\0' only in one place, namely at the point where the '\n' was
appended in the Xml_generator constructor. This also fixes the problem
with the exceptions.

This also extends the xml_generator test to drive a harder test on
exceptions in the Xml_generator.

Fixes #2967
@chelmuth
Copy link
Member

Related to the discussion of _undo_content_buffer I'm wondering how the util should work if the top-level node is aborted by throwing an exception. (This is the only case with indented==false as far as I understand.) One option is to generate an empty top-level node which would be valid XML. Currently, the test case generates <config when throwing after

static size_t xml_with_exceptions(char *dst, size_t dst_len)
{
Genode::Xml_generator xml(dst, dst_len, "config", [&]
{

@chelmuth
Copy link
Member

I suspect the commit series breaks make run/report_rom, please check.

@nfeske
Copy link
Member

nfeske commented Sep 11, 2018

The commit Xml_generator: fix and test missing '\0' causes a segfault in init in the leitzentrale.run scenario on base-linux.

@chelmuth chelmuth changed the title Xml_generator: fix unnecessary new lines Xml_generator: prevent unnecessary new lines Sep 11, 2018
@chelmuth chelmuth removed the fixed label Sep 11, 2018
@nfeske
Copy link
Member

nfeske commented Sep 11, 2018

Let's keep this line of work for merging at a later point because it deserves more attention than I'm able to provide right now.

@m-stein I'm wondering, in the commit message you state "Without exceptions in the Node constructor this led to "\0\n" at the end of generated XML." as a fact. But have you actually witnessed this sequence in the wild? I'm asking because I haven't (looking at the reports in sculpt's report fs), and when looking at the code, I would not expect it either.

Let my try to clarify, the _out_buffer.append('\') operation is applied to a local copy of the _out_buffer after committing the content. This local copy (the buffer meta data, not the characters) is discarded afterwards. Hence, the zero-character is inserted but it stays outside the committed character sequence.

BTW, The \n at the end ensures that the generated data ends with a newline when printed, e.g., by cat /report/runtime/state. I would like to preserve it. (I noticed that you added a manual \n to the test, which I find odd). But it actually overwrites the last written \0. So it might be sensible to also append a \0 here. That said, we normally don't depend on the zero-termination of the generated string but rather propagate the number of generated characters to the user via the user() accessor (see the use in repos/os/include/os/reporter.h). So I'm still puzzled how your patch could introduce the segfault problem I reported for leitzentrale.run.

...to be investigated later.

chelmuth pushed a commit that referenced this issue Sep 13, 2018
1) The loop for determining the line length read from a character offset
   before checking whether the offset is smaller than the given string
   length. This could have caused access outside the string buffer.

2) The routine for determining the line length first seeked for the
   offset of the last real character of the line and than added one for
   getting the length but only if the following character was '\n'. This
   has to be done for any other line-terminating character too. The only
   case where you don't want to do this is when the end of the whole
   string is reached.

Issue #2967
chelmuth pushed a commit that referenced this issue Sep 13, 2018
1) The loop for determining the line length read from a character offset
   before checking whether the offset is smaller than the given string
   length. This could have caused access outside the string buffer.

2) The routine for determining the line length first seeked for the
   offset of the last real character of the line and than added one for
   getting the length but only if the following character was '\n'. This
   has to be done for any other line-terminating character too. The only
   case where you don't want to do this is when the end of the whole
   string is reached.

Issue #2967
m-stein added a commit to m-stein/genode that referenced this issue Sep 16, 2018
When the functor provided to the Node constructor throws an exception,
do revert all changes in reverse order. Previously, the changes made
to the parent node were not considered by the exception handler which
caused unnecessary characters to remain in the out buffer for each
reverted node.

Issue genodelabs#2967
m-stein added a commit to m-stein/genode that referenced this issue Sep 16, 2018
Previously, a '\0' was appended to the out buffer at the end of each
call of the Node constructor. At the end of the Xml_generator
constructor, at the other hand, '\n' was appended. Without exceptions in
the Node constructor this led to "\0\n" at the end of generated XML.
However, with exceptions, the '\0' of the last Node constructor call was
sometimes overwritten, leading to a mere "\n" at the end. It is better
to append '\0' only in one place, namely at the point where the '\n' was
appended in the Xml_generator constructor. This also fixes the problem
with the exceptions.

This also extends the xml_generator test to drive a harder test on
exceptions in the Xml_generator.

Fixes genodelabs#2967
m-stein added a commit to m-stein/genode that referenced this issue Sep 16, 2018
When the functor provided to the Node constructor throws an exception,
do revert all changes in reverse order. Previously, the changes made
to the parent node were not considered by the exception handler which
caused unnecessary characters to remain in the out buffer for each
reverted node.

Issue genodelabs#2967
m-stein added a commit to m-stein/genode that referenced this issue Sep 16, 2018
Previously, a '\0' was appended to the out buffer at the end of each
call of the Node constructor. At the end of the Xml_generator
constructor, at the other hand, '\n' was appended. Without exceptions in
the Node constructor this led to "\0\n" at the end of generated XML.
However, with exceptions, the '\0' of the last Node constructor call was
sometimes overwritten, leading to a mere "\n" at the end. It is better
to append '\0' only in one place, namely at the point where the '\n' was
appended in the Xml_generator constructor. This also fixes the problem
with the exceptions.

This also extends the xml_generator test to drive a harder test on
exceptions in the Xml_generator.

Fixes genodelabs#2967
m-stein added a commit to m-stein/genode that referenced this issue Sep 27, 2018
When the functor provided to the Node constructor throws an exception,
do revert all changes in reverse order. Previously, the changes made
to the parent node were not considered by the exception handler which
caused unnecessary characters to remain in the out buffer for each
reverted node.

Issue genodelabs#2967
m-stein added a commit to m-stein/genode that referenced this issue Sep 27, 2018
Previously, a '\0' was appended to the out buffer at the end of each
call of the Node constructor. At the end of the Xml_generator
constructor, at the other hand, '\n' was appended. Without exceptions in
the Node constructor this led to "\0\n" at the end of generated XML.
However, with exceptions, the '\0' of the last Node constructor call was
sometimes overwritten, leading to a mere "\n" at the end. It is better
to append '\0' only in one place, namely at the point where the '\n' was
appended in the Xml_generator constructor. This also fixes the problem
with the exceptions.

This also extends the xml_generator test to drive a harder test on
exceptions in the Xml_generator.

Fixes genodelabs#2967
m-stein added a commit to m-stein/genode that referenced this issue Sep 28, 2018
When the functor provided to the Node constructor throws an exception,
do revert all changes in reverse order. Previously, the changes made
to the parent node were not considered by the exception handler which
caused unnecessary characters to remain in the out buffer for each
reverted node.

Issue genodelabs#2967
m-stein added a commit to m-stein/genode that referenced this issue Sep 28, 2018
Previously, a '\0' was appended to the out buffer at the end of each
call of the Node constructor. At the end of the Xml_generator
constructor, at the other hand, '\n' was appended. Without exceptions in
the Node constructor this led to "\0\n" at the end of generated XML.
However, with exceptions, the '\0' of the last Node constructor call was
sometimes overwritten, leading to a mere "\n" at the end. It is better
to append '\0' only in one place, namely at the point where the '\n' was
appended in the Xml_generator constructor. This also fixes the problem
with the exceptions.

This also extends the xml_generator test to drive a harder test on
exceptions in the Xml_generator.

Fixes genodelabs#2967
@nfeske
Copy link
Member

nfeske commented Jan 30, 2019

Fixed in master.

@nfeske nfeske closed this as completed Jan 30, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants