-
Notifications
You must be signed in to change notification settings - Fork 6.1k
8347707: Standardise the use of os::snprintf and os::snprintf_checked #26849
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Make os::vsnprintf responsible for error checking.
Change callers that need the return value to use os::snprintf, and add truncation checks.
…ld not cause a problem during normal execution in debug testing.
|
👋 Welcome back dholmes! A progress list of the required criteria for merging this PR into |
|
@dholmes-ora This change now passes all automated pre-integration checks. ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details. After integration, the commit message for the final commit will be: You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed. At the time when this comment was updated there had been 87 new commits pushed to the
As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details. ➡️ To integrate this PR with the above commit message to the |
|
@dholmes-ora The following labels will be automatically applied to this pull request:
When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command. |
Webrevs
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I noticed there are a number of calls with buflen - 1. Probably from the bad
old days when some snprintf variants didn't guarantee null termination. I
didn't see forced null terminations for the few I spot-checked though, and
it's not obvious in some cases that the final byte is certain to be null going
in. Nothing to be done about that here, but there might be some following
cleanup work called for.
| int written_chars = os::snprintf_checked(buffer, sizeof(buffer), "%d", value); | ||
| if (written_chars <= 4) { | ||
| int written_chars = os::snprintf(buffer, sizeof(buffer), "%d", value); | ||
| if (written_chars > 0 && written_chars <= 4) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't understand the addition of written_chars > 0 here and line 658?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well in theory, in a release build, os::snprintf could return -1 and I didn't want any static analysis tools pointing this out to me later.
src/hotspot/share/runtime/os.hpp
Outdated
| static int snprintf_checked(char* buf, size_t len, const char* fmt, ...) ATTRIBUTE_PRINTF(3, 4); | ||
| // Performs vsnprintf and asserts the result is non-negative (so there was not | ||
| // an encoding error or any other kind of usage error). | ||
| [[nodiscard]] static int vsnprintf(char* buf, size_t len, const char* fmt, va_list args) ATTRIBUTE_PRINTF(3, 0); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consider moving the ATTRIBUTE_PRINTF to the front so all the attributes are together?
And maybe a line break between the attributes and the signature, just to avoid pushing the
signature way over to the right.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have done that and placed each attribute on its own line (we should have a style guide entry for this :) ). But I note that all the other ATTRIBUTE_PRINTF's are placed after the function.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What you've done here matches what I did when adding [[noreturn]] to report_xxx functions
in debug.hpp. So I'm good with that. The style guide already has this guidance:
https://github.com/openjdk/jdk/blame/02fe095d29994bec28c85beb6bf2a69b0f49b206/doc/hotspot-style.md#L1120-L1122
There's just lots of old, non-conforming code. :)
I forgot about that when I said "Consider moving", and should have pointed to the style guide.
Whether and where there should be line breaks is something the style guide leaves to authors and reviewers.
|
Could we map snprintf to os::snprintf_checked with a macro or some kind of namespace magic? It would reduce the number of files changed and catch any new cases of snprintf that might get accidentally added in the future. |
| if (sg_error == VMGUESTLIB_ERROR_SUCCESS) { | ||
| has_host_information = true; | ||
| os::snprintf(host_information, sizeof(host_information), "%s", result_info); | ||
| os::snprintf_checked(host_information, sizeof(host_information), "%s", result_info); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are these two guaranteed not to overflow/truncate?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The issue is not whether they can overflow, but if they do is it something we want to detect during testing so we can take action - e.g. by increasing the buffer size. This is very subjective, but my initial position in most cases has been yes.
| snprintf(fn, sizeof(fn), "%s/.attach_pid%d", | ||
| os::get_temp_directory(), os::current_process_id()); | ||
| os::snprintf_checked(fn, sizeof(fn), "%s/.attach_pid%d", | ||
| os::get_temp_directory(), os::current_process_id()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This could fail if os::get_temp_directory() returns an extremely long path. How about doing a truncation check like at line 354?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nevermind, same thing. We would need to fix a lot of code if os::get_temp_directory() returned a pathologically long string.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've changed line 352 as per Kim's comment above because snprintf followed by an assert for truncation is what snprintf_checked does.
Again the question to ask is: if we hit this during testing do we think it indicates we need to increase the buffer size. Again I am initially in the yes camp.
|
Thanks for looking at this @dean-long !
Raw |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good.
|
Thanks for the Review Kim! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A good looking PR, and a good step in the right direction. Nice!
But as @kimbarrett I also noticed there are a number of calls with buflen - 1, which in my case consumed too mush of the review time. So it would be nice to see a follow up that deals with code like this.
|
Thanks for the review @fbredber . I have added your cleanup suggestion to JDK-8365896 as well. |
|
/integrate |
|
Going to push as commit 80ab094.
Your commit was automatically rebased without conflicts. |
|
@dholmes-ora Pushed as commit 80ab094. 💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored. |
|
Note, before integrating I merged locally with master, checked the incoming changes and re-ran tier 1-3 builds. |
This is a proposal to standardize on the use of
os::snprintfandos::snprintf_checked across the hotspot code base, and to disallow use of the C library variants. (It does not touch use ofjio_printfat all.)From: https://bugs.openjdk.org/browse/JDK-8347707
The platform
snprintf/vsnprintfreturns -1 on error, else if the buffer is large enough returns the number of bytes written (excluding the null byte), else (buffer is too small) the number of characters (excluding the terminating null byte) which would have been written to the final string if enough space had been available. Thus, a return value of size or more means that the output was truncated.To provide a consistent approach to error handling and truncation management, we provide
os::xxxwrapper functions as described below and forbid the use of the library::vsnprintfand::snprintf.The potential errors are, generally speaking, not something we should encounter in our own well-written code:
int).As these should simply never occur, we handle the checks for -1 at the lowest-level (
os::vsnprintf) with an assertion, and accompanying precondition assertions.The potential clients of this API then fall into a number of camps:
For these clients we have
void os::snprintf_checked- which returns nothing and asserts on truncation.snprintfwhere you advance the buffer pointer based on previous writes), but again for whom truncation should never happen.For these clients we have
os::snprintf, but they have to add their own assertion for no truncation.These clients also use
os::snprintf_checked. The truncation assertion can be useful for guiding buffer sizing decisions, but in product mode truncation is not an error.These clients are also directed to use
os::snprintf.In summary we provide the following API:
[[nodiscard]] int os::vsnprintfis the building block for the other methods, it:[[nodiscard[]]so that callers cannot ignore the return value (they can explicitly cast tovoidto indicate they don't need it)void os::snprintf_checked[[nodiscard]] int os::snprintfos::vnsprintfso asserts on errorsIn terms of the effects on the existing code we:
::snprintf/os::snprintfthat ignore the return value and ensure the buffer is large enough to useos::snprintf_checkedos::snprintf.::snprintf/os::snprintfthat use the return value to useos::snprintf, plus any additional assertions neededos::snprintf_checkedthat do use the return value, to useos::snprintfwith their own assertions addedos::vnsprintfare adjusted as neededThe PR is comprising multiple dependent commits so that you can view things in stages. There are 46 modified files. The bulk of the changes replace calls to
snprintf/os::snprintfwith calls toos::snprintf_checked.Note the use of
[[nodiscard]]is permitted but not yet documented as such in the style-guide.Progress
Issue
Reviewers
Reviewing
Using
gitCheckout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/26849/head:pull/26849$ git checkout pull/26849Update a local copy of the PR:
$ git checkout pull/26849$ git pull https://git.openjdk.org/jdk.git pull/26849/headUsing Skara CLI tools
Checkout this PR locally:
$ git pr checkout 26849View PR using the GUI difftool:
$ git pr show -t 26849Using diff file
Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/26849.diff
Using Webrev
Link to Webrev Comment