-
Notifications
You must be signed in to change notification settings - Fork 5.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
8271003: hs_err improvement: handle CLASSPATH env setting longer than O_BUFLEN #4947
8271003: hs_err improvement: handle CLASSPATH env setting longer than O_BUFLEN #4947
Conversation
/label add hotspot-runtime |
👋 Welcome back ccheung! A progress list of the required criteria for merging this PR into |
@calvinccheung |
Webrevs
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi Calvin,
Not sure about this ... still thinking.
In the meantime see couple of comments below.
Thanks,
David
// constant format string | ||
str = format; | ||
len = strlen(str); | ||
} else if (format[0] == '%' && format[1] == 's' && format[2] == '\0') { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why are you assuming the %s must be at the beginning rather than "blah blah %s blah" ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah I see what you are doing now - the code just special cases the two cases of a string with no format specifiers, and a string that is only "%s". Anything else needs full handling.
private static final String cp_env = "CLASSPATH"; | ||
private static final String end_path = "end-path"; | ||
|
||
private static class Crasher { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can't you just use one of the existing -XX:ErrorHandlerTest=n values to trigger the crash?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, using -XX:ErrorHandlerTest=1 should trigger the crash. I was just reusing the Crasher from another test case.
David |
I think we already do almost that. jdk/src/hotspot/share/utilities/ostream.cpp Lines 94 to 104 in 95f0fd6
Only thing, for print_cr() we still use the scratch buffer since we need to append \n. That could be done smarter. Beyond that, I think the O_BUFLEN thing is seriously annoying and I would like to have a better solution. One where we start with a preallocated buffer as we do now, but expand it dynamically (using raw ::malloc()) if necessary. That should still be reasonably safe. |
I was thinking of changing |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi Calvin,
Sorry, I think we cannot do it the way you propose.
The problem is that the print_cr() functions are supposed to write the newline with the same write(2)
call. That is not documented in do_vsnprintf(), it's an easy to miss a detail.
Writing with one single write(2)
call guarantees some atomicity when writing concurrently from several threads. If you write the newline separately, you risk line tear - the newline appearing away from its line.
This means that print_cr("%s", s)
should not write the newline separately and therefore would not benefit from your improvement. And print("%s", s)
would already do exactly what you propose, since in do_vsnprintf()
we already handle the two special cases.
I propose a specific fix for your problem in os::print_environment_variables()
by replacing st->print_cr("%s", envvar);
with either one of
st->print("%s", envvar);
st->cr()
or
st->print_raw(envvar);
st->cr()
The test is useful though. But you can simplify it:
- as David said, use
-XX:ErrorHandlerTest
. For better realistic testing, I'd use an option which gives you a real signal, e.g.-XX:ErrorHandlerTest=14
- just start the test with
-XX:+ErrorFileToStdout
. We added this option to pipe the error file to stdout. That way you can parse stdout directly, using OutputScanner methods and regex, no need to open and parse the hs-err file:
thomas@starfish$ CLASSPATH=abcabc ./images/jdk/bin/java -XX:ErrorHandlerTest=14 -XX:+ErrorFileToStdout | grep CLASSPATH
CLASSPATH=abcabc
Cheers, Thomas
Hi Thomas, Thanks for your review.
I picked the above since it is more consistent with another
I've updated the test and using Thanks,
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, Calvin. Your more general approach would have been nice, but thanks for considering my log tearing worries and for taking my suggestion.
Cheers, Thomas
@calvinccheung This change now passes all automated pre-integration checks. ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details. After integration, the commit message for the final commit will be:
You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed. At the time when this comment was updated there had been 39 new commits pushed to the
As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details. ➡️ To integrate this PR with the above commit message to the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good.
One comment request.
Thanks,
David
@@ -989,7 +989,8 @@ void os::print_environment_variables(outputStream* st, const char** env_list) { | |||
if (envvar != NULL) { | |||
st->print("%s", env_list[i]); | |||
st->print("="); | |||
st->print_cr("%s", envvar); | |||
st->print("%s", envvar); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suggest adding a comment like:
// Use separate cr() printing to avoid unnecessary buffer operations that might cause truncation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for taking another look.
I've added the comment.
@dholmes-ora, @tstuefe, @iklam, @yminqi Thanks for the review. |
/integrate |
Going to push as commit 3435d29.
Your commit was automatically rebased without conflicts. |
@calvinccheung Pushed as commit 3435d29. 💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored. |
Mailing list message from David Holmes on hotspot-runtime-dev: On 2/08/2021 6:26 pm, Thomas Stuefe wrote:
Right I missed that bit - but I guess we can do that as a separate
Raw malloc in the error handler is not safe if we're processing a Cheers, |
Mailing list message from Thomas Stüfe on hotspot-runtime-dev: On Mon, Aug 2, 2021 at 11:59 AM David Holmes <david.holmes at oracle.com>
I thought it would be a calculated risk. You only run into problems if Cheers, Thomas
|
Mailing list message from David Holmes on hotspot-runtime-dev: On 3/08/2021 3:40 am, Calvin Cheung wrote:
Sorry Calvin but I'd rather see a single complete solution in one step. IIUC the only time we have a problem with truncation is when we have void outputStream::vprint_cr(const char* format, va_list argptr) { to void outputStream::vprint_cr(const char* format, va_list argptr) { or add the "add_cr" to the write() method and let it handle it without a Thanks, |
Mailing list message from Thomas Stüfe on hotspot-runtime-dev: On Tue, Aug 3, 2021 at 12:53 AM David Holmes <david.holmes at oracle.com>
Missed your mail since the duplication of ML mails into github See my answer to Calvin, I think we want to write the output with a single ..Thomas |
Mailing list message from David Holmes on hotspot-runtime-dev: Hi Thomas, On 3/08/2021 2:00 pm, Thomas Stuefe wrote:
Do we really rely on the write(2) call for atomicity? The defaultStream Cheers, |
Mailing list message from Thomas Stüfe on hotspot-runtime-dev: Hi David,
That would only work for the defaultStream, no? What about other streams? Cheers,
..Thomas |
Mailing list message from David Holmes on hotspot-runtime-dev: On 3/08/2021 2:38 pm, Thomas St?fe wrote:
Do the other streams have atomicity in the first place? It's not an David
|
Mailing list message from Thomas Stüfe on hotspot-runtime-dev: On Tue, Aug 3, 2021 at 7:32 AM David Holmes <david.holmes at oracle.com> wrote:
We have fdStream and fileStream. Both are calling write(2) resp. fwrite(3) ..Thomas |
Please review this small enhancement for addressing the problem of the CLASSPATH env variable setting being truncated in a hs err log.
For printing a char string, it doesn't need to go through
do_vsnprintf()
which does the truncation based on the input buffer length. The change is local to the code path pertaining to hs err log.Testing:
Progress
Issue
Reviewers
Reviewing
Using
git
Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/4947/head:pull/4947
$ git checkout pull/4947
Update a local copy of the PR:
$ git checkout pull/4947
$ git pull https://git.openjdk.java.net/jdk pull/4947/head
Using Skara CLI tools
Checkout this PR locally:
$ git pr checkout 4947
View PR using the GUI difftool:
$ git pr show -t 4947
Using diff file
Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/4947.diff