8311906: Improve robustness of String constructors with mutable array inputs #16425

RogerRiggs · 2023-10-30T18:34:44Z

Strings, after construction, are immutable but may be constructed from mutable arrays of bytes, characters, or integers.
The string constructors should guard against the effects of mutating the arrays during construction that might invalidate internal invariants for the correct behavior of operations on the resulting strings. In particular, a number of operations have optimizations for operations on pairs of latin1 strings and pairs of non-latin1 strings, while operations between latin1 and non-latin1 strings use a more general implementation.

The changes include:

Adding a warning to each constructor with an array as an argument to indicate that the results are indeterminate
if the input array is modified before the constructor returns.
The resulting string may contain any combination of characters sampled from the input array.
Ensure that strings that are represented as non-latin1 contain at least one non-latin1 character.
For latin1 inputs, whether the arrays contain ASCII, ISO-8859-1, UTF8, or another encoding decoded to latin1 the scanning and compression is unchanged.
If a non-latin1 character is found, the string is represented as non-latin1 with the added verification that a non-latin1 character is present at the same index.
If that character is found to be latin1, then the input array has been modified and the result of the scan may be incorrect.
Though a ConcurrentModificationException could be thrown, the risk to an existing application of an unexpected exception should be avoided.
Instead, the non-latin1 copy of the input is re-scanned and compressed; that scan determines whether the latin1 or the non-latin1 representation is returned.
The methods that scan for non-latin1 characters and their intrinsic implementations are updated to return the index of the non-latin1 character.
String construction from StringBuilder and CharSequence must also be guarded as their contents may be modified during construction.

Progress

Change must be properly reviewed (1 review required, with at least 1 Reviewer)
Change must not contain extraneous whitespace
Commit message must refer to an issue
Change requires CSR request JDK-8319228 to be approved

Issues

JDK-8311906: Improve robustness of String constructors with mutable array inputs (Bug - P4)
JDK-8319228: Improve robustness of String constructors with mutable array inputs (CSR)

Reviewers

Raffaello Giulietti (@rgiulietti - Reviewer) ⚠️ Review applies to 5299c43b
Tobias Hartmann (@TobiHartmann - Reviewer) ⚠️ Review applies to ad73a2a6
Claes Redestad (@cl4es - Reviewer)
Damon Fenacci (@dafedafe - Committer)

Contributors

Damon Fenacci <dfenacci@openjdk.org>
Claes Redestad <redestad@openjdk.org>
Amit Kumar <amitkumar@openjdk.org>
Martin Doerr <mdoerr@openjdk.org>

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/16425/head:pull/16425
$ git checkout pull/16425

Update a local copy of the PR:
$ git checkout pull/16425
$ git pull https://git.openjdk.org/jdk.git pull/16425/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 16425

View PR using the GUI difftool:
$ git pr show -t 16425

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/16425.diff

Webrev

Link to Webrev Comment

… arguments Ensure that mutations of input arguments to constructors can not invalidate the latin1 vs non-latin1 coding. 8313982: Non-conforming CharSequence implementation can break StringBuilder Increase AbstractStringBuilder/StringBuilder robustness when appending a CharSequence that may throw exceptions while returning characters from the sequence.

RogerRiggs · 2023-10-30T18:34:58Z

/csr

bridgekeeper · 2023-10-30T18:35:54Z

👋 Welcome back rriggs! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2023-10-30T18:37:18Z

@RogerRiggs has indicated that a compatibility and specification (CSR) request is needed for this pull request.

@RogerRiggs this pull request must refer to an issue in JBS to be able to link it to a CSR request. To refer this pull request to an issue in JBS, please update the title of this pull request to just the issue ID.

openjdk · 2023-10-30T18:37:40Z

@RogerRiggs The following labels will be automatically applied to this pull request:

core-libs
hotspot

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

…mpress to return the index of the first non-latin1 character or the length if none. Not yet built or tested on RiscV. The test StringRacyConstructor verifies the behavior of the compress intrinsic. Updated comments on intrinsic implementations of char_array_compress to reflect the change in the behavior of StringUTF16.compress. Damon Fenacci (@damon_fenacci) authored the changes to macroAssembler_aarch64.cpp and macroAssembler_aarch64.cpp.

liach · 2023-11-04T00:07:33Z

src/java.base/share/classes/java/lang/StringUTF16.java

+     */
+    @ForceInline
+    public static byte[] compress(final char[] val, final int off, final int count) {
+        byte[] latin1 = new byte[count];


Will this redundant array allocation be costly if we are working with mostly-utf16 strings, such as CJK strings with no latin characters?

I suggest we can use a heuristic to read the initial char; if it's utf16 then we skip the latin-1 process altogether (and we can assign the utf16 value to the initial index to ensure it's non-latin-1 compressible.

We can reconsider this design as a separate PR.
Every additional check has a performance impact and in this bug the goal is to avoid any regression.

We'll need to gain some insight into the distribution of strings when used in a non-latin1 application.
How many of the strings are latin1 vs non-latin1, what is the distribution of string lengths and which APIs are in use in the applications. The implementation is already pretty good about working with strings of different coders
but there may be some different choices when converting between char arrays and int arrays and strings.

Just curious, how does benchmark StringConstructor.newStringFromCharsMixedBegin change before and after this patch? If we can see how much of an impact this has on CJK strings it would be appreciated.

Just curious, how does benchmark StringConstructor.newStringFromCharsMixedBegin change before and after this patch? If we can see how much of an impact this has on CJK strings it would be appreciated.

You may have better insights from doing your own runs on your own systems.
Here's a sample from a recent run. Mostly small improvements and a few small negatives.

Benchmark Linux aarch64 | Linux x64 | MacOSX aarch64 | MacOSX x64 newStringFromCharsMixedBegin-size:64 | 1.22% | 0.91% | -0.81% | 1.08% newStringFromCharsMixedBegin-size:7 | 4.40% | 0.26% | -1.88% | 4.60%

src/java.base/share/classes/java/lang/StringUTF16.java

jaikiran · 2023-11-05T13:29:40Z

Hello Roger, it looks like there are some whitespace related issues in the changes which jcheck has caught https://github.com/openjdk/jdk/pull/16425/checks?check_run_id=18357062638 and thus hasn't created a RFR for this yet.

src/java.base/share/classes/java/lang/String.java

mlbridge · 2023-11-06T16:28:21Z

Webrevs

dafedafe · 2023-11-07T16:40:00Z

This PR includes changes to the C2_MacroAssembler::char_array_compress_v intrinsic for RISCV that we couldn't test. @RealFYang could you please test and review it? Thanks a lot!

RogerRiggs · 2023-11-07T17:52:46Z

/author set dfenacci

Damon Fenacci authored the updates to the hotspot intrinsics.

openjdk · 2023-11-07T17:54:58Z

@RogerRiggs
Setting overriding author to Damon Fenacci <dfenacci@openjdk.org>. When this pull request is integrated, the overriding author will be used in the commit.

RogerRiggs · 2023-11-07T17:57:53Z

/author set rriggs
/contributor add dfenacci

openjdk · 2023-11-07T17:58:53Z

@RogerRiggs
Setting overriding author to Roger Riggs <rriggs@openjdk.org>. When this pull request is integrated, the overriding author will be used in the commit.

openjdk · 2023-11-07T17:58:56Z

@RogerRiggs
Contributor Damon Fenacci <dfenacci@openjdk.org> successfully added.

RogerRiggs · 2023-11-07T18:07:55Z

/author remove

Remove the override.

openjdk · 2023-11-07T18:10:17Z

@RogerRiggs
Overriding author Roger Riggs <rriggs@openjdk.org> was successfully removed. When this pull request is integrated, the pull request author will be used as the author of the commit.

dafedafe · 2023-11-08T16:33:45Z

It would also be good to have the aarch64 and x64 intrinsics properly reviewed. @theRealAph could I ask you to have a look at aarch64 and @TobiHartmann at x64 please? (you seem to be the last ones that made major changes in the intrinsics)

rgiulietti

First take ;-)
More will follow.

src/java.base/share/classes/java/lang/StringLatin1.java

src/java.base/share/classes/java/lang/StringUTF16.java

src/java.base/share/classes/java/lang/String.java

src/java.base/share/classes/java/lang/StringUTF16.java

Extra checking on maximum string lengths when calling toBytes().

src/java.base/share/classes/java/lang/String.java

…6.compress and normalize calling sequence

…ingBuilder, and Appendable methods in the case where the input arguments are modified during construction or a StringBuilder or Appendable method call.

rgiulietti

Looks good.
Maybe use StringUTF16.coderFromArrayLen() where suggested in the comments before integrating.

src/java.base/share/classes/java/lang/String.java

Simplified and slightly optimized x86 char_array_compress

src/hotspot/cpu/x86/macroAssembler_x86.cpp

Verified by manual tests with "-XX:AVX3Threshold=0" And test in the PR test/hotspot/jtreg/compiler/intrinsics/string/TestStringConstructionIntrinsics.java

rgiulietti

The Java code, both src and test, looks good to me.

src/hotspot/cpu/x86/macroAssembler_x86.cpp

dafedafe

Intrinsics look OK to me.

RogerRiggs · 2023-12-04T18:27:16Z

/integrate

openjdk · 2023-12-04T18:29:03Z

Going to push as commit 155abc5.
Since your change was applied there have been 456 commits pushed to the master branch:

316b783: 8321276: runtime/cds/appcds/dynamicArchive/DynamicSharedSymbols.java failed with "'17 2: jdk/test/lib/apps ' missing from stdout/stderr"
65be5e0: 8305931: jdk/jfr/jcmd/TestJcmdDumpPathToGCRoots.java failed with "Expected chains but found none"
f6be922: 8316193: jdk/jfr/event/oldobject/TestListenerLeak.java java.lang.Exception: Could not find leak
a9de5c7: 8315128: jdk/jfr/event/runtime/TestResidentSetSizeEvent.java fails with "The size should be less than or equal to peak"
d2c529c: 8319072: JFR: Turn off events for JFR.view
d5f59cf: 8321220: JFR: RecordedClass reports incorrect modifiers
9769dfe: 8321214: Parallel: Remove unused SpaceInfo::_min_dense_prefix
d23f4f1: 8315559: Delay TempSymbol cleanup to avoid symbol table churn
c17b8cf: 8320655: awt screencast robot spin and sync issues with native libpipewire api
ed5b8c3: 8225220: When the Tab Policy is checked,the scroll button direction displayed incorrectly.
... and 446 more: https://git.openjdk.org/jdk/compare/4679e9aa00c098cff715fb4deeb4d736e1063571...master

Your commit was automatically rebased without conflicts.

openjdk · 2023-12-04T18:29:10Z

@RogerRiggs Pushed as commit 155abc5.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

openjdk bot added the csr Pull request needs approved CSR before integration label Oct 30, 2023

openjdk bot added hotspot hotspot-dev@openjdk.org core-libs core-libs-dev@openjdk.org labels Oct 30, 2023

RogerRiggs added 2 commits October 31, 2023 09:32

Javadoc formatting

43f937d

RogerRiggs changed the title ~~Draft: 8311906: Improve robustness of String constructors with mutable array inputs~~ 8311906: Improve robustness of String constructors with mutable array inputs Nov 1, 2023

RogerRiggs marked this pull request as ready for review November 1, 2023 14:25

liach mentioned this pull request Nov 4, 2023

8311906: Race condition in String constructor #15902

Closed

4 tasks

liach reviewed Nov 4, 2023

View reviewed changes

ExE-Boss reviewed Nov 5, 2023

View reviewed changes

src/java.base/share/classes/java/lang/String.java Show resolved Hide resolved

Cleanup javadoc, whitespace, and formatting in the JMH benchmark

4662dec

openjdk bot added the rfr Pull request is ready for review label Nov 6, 2023

rgiulietti reviewed Nov 8, 2023

View reviewed changes

Misc updates for review comments, javadoc cleanup

00a5d06

Extra checking on maximum string lengths when calling toBytes().

openjdk bot added the rfr Pull request is ready for review label Nov 16, 2023

rgiulietti reviewed Nov 16, 2023

View reviewed changes

src/java.base/share/classes/java/lang/String.java Outdated Show resolved Hide resolved

openjdk bot added csr Pull request needs approved CSR before integration and removed ready Pull request is ready to be integrated labels Nov 17, 2023

Speed up getting the coder from a byte array returned from StringUTF1…

6419fd5

…6.compress and normalize calling sequence

openjdk bot mentioned this pull request Nov 20, 2023

Simplify x86 char_array_compress intrinsic #16735

Closed

3 tasks

RogerRiggs added 2 commits November 20, 2023 12:09

Normalize the spec for undefined behavior of String constructors, Str…

7924118

…ingBuilder, and Appendable methods in the case where the input arguments are modified during construction or a StringBuilder or Appendable method call.

undo noise chars

04d5877

rgiulietti approved these changes Nov 20, 2023

View reviewed changes

src/java.base/share/classes/java/lang/String.java Outdated Show resolved Hide resolved

src/java.base/share/classes/java/lang/String.java Outdated Show resolved Hide resolved

cl4es and others added 3 commits November 21, 2023 00:41

Simplified and slightly optimized x86 char_array_compress

d4bd5c9

Merge pull request #4 from cl4es/8311906_x64_intr_opt

0256b9e

Simplified and slightly optimized x86 char_array_compress

Apply StringUTF16.coderFromArrayLen

d201344

dafedafe reviewed Nov 27, 2023

View reviewed changes

src/hotspot/cpu/x86/macroAssembler_x86.cpp Outdated Show resolved Hide resolved

src/hotspot/cpu/x86/macroAssembler_x86.cpp Outdated Show resolved Hide resolved

src/hotspot/cpu/x86/macroAssembler_x86.cpp Outdated Show resolved Hide resolved

Use byte off branches in char_array_compress

5299c43

Verified by manual tests with "-XX:AVX3Threshold=0" And test in the PR test/hotspot/jtreg/compiler/intrinsics/string/TestStringConstructionIntrinsics.java

openjdk bot added ready Pull request is ready to be integrated and removed csr Pull request needs approved CSR before integration labels Nov 27, 2023

rgiulietti approved these changes Nov 28, 2023

View reviewed changes

dafedafe reviewed Nov 30, 2023

View reviewed changes

src/hotspot/cpu/x86/macroAssembler_x86.cpp Outdated Show resolved Hide resolved

src/hotspot/cpu/x86/macroAssembler_x86.cpp Outdated Show resolved Hide resolved

Correct jcc/jccb branches

b2fc385

cl4es approved these changes Dec 1, 2023

View reviewed changes

dafedafe approved these changes Dec 4, 2023

View reviewed changes

openjdk bot added the integrated Pull request has been integrated label Dec 4, 2023

openjdk bot closed this Dec 4, 2023

openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Dec 4, 2023

RogerRiggs deleted the 8311906-racy-string-constructor branch June 24, 2024 15:51

wenshao mentioned this pull request Apr 22, 2025

8355240: Remove unused Import in StringUTF16 #24788

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

8311906: Improve robustness of String constructors with mutable array inputs #16425

8311906: Improve robustness of String constructors with mutable array inputs #16425

RogerRiggs commented Oct 30, 2023 •

edited by openjdk bot

Loading

RogerRiggs commented Oct 30, 2023

bridgekeeper bot commented Oct 30, 2023

openjdk bot commented Oct 30, 2023

openjdk bot commented Oct 30, 2023

liach Nov 4, 2023

RogerRiggs Nov 6, 2023 •

edited

Loading

liach Nov 9, 2023

RogerRiggs Nov 9, 2023

jaikiran commented Nov 5, 2023

mlbridge bot commented Nov 6, 2023 •

edited

Loading

dafedafe commented Nov 7, 2023

RogerRiggs commented Nov 7, 2023

openjdk bot commented Nov 7, 2023

RogerRiggs commented Nov 7, 2023

openjdk bot commented Nov 7, 2023

openjdk bot commented Nov 7, 2023

RogerRiggs commented Nov 7, 2023

openjdk bot commented Nov 7, 2023

dafedafe commented Nov 8, 2023

rgiulietti left a comment

rgiulietti left a comment

rgiulietti left a comment

dafedafe left a comment

RogerRiggs commented Dec 4, 2023

openjdk bot commented Dec 4, 2023

openjdk bot commented Dec 4, 2023

8311906: Improve robustness of String constructors with mutable array inputs #16425

8311906: Improve robustness of String constructors with mutable array inputs #16425

Conversation

RogerRiggs commented Oct 30, 2023 • edited by openjdk bot Loading

Progress

Issues

Reviewers

Contributors

Reviewing

Webrev

RogerRiggs commented Oct 30, 2023

bridgekeeper bot commented Oct 30, 2023

openjdk bot commented Oct 30, 2023

openjdk bot commented Oct 30, 2023

liach Nov 4, 2023

Choose a reason for hiding this comment

RogerRiggs Nov 6, 2023 • edited Loading

Choose a reason for hiding this comment

liach Nov 9, 2023

Choose a reason for hiding this comment

RogerRiggs Nov 9, 2023

Choose a reason for hiding this comment

jaikiran commented Nov 5, 2023

mlbridge bot commented Nov 6, 2023 • edited Loading

Webrevs

dafedafe commented Nov 7, 2023

RogerRiggs commented Nov 7, 2023

openjdk bot commented Nov 7, 2023

RogerRiggs commented Nov 7, 2023

openjdk bot commented Nov 7, 2023

openjdk bot commented Nov 7, 2023

RogerRiggs commented Nov 7, 2023

openjdk bot commented Nov 7, 2023

dafedafe commented Nov 8, 2023

rgiulietti left a comment

Choose a reason for hiding this comment

rgiulietti left a comment

Choose a reason for hiding this comment

rgiulietti left a comment

Choose a reason for hiding this comment

dafedafe left a comment

Choose a reason for hiding this comment

RogerRiggs commented Dec 4, 2023

openjdk bot commented Dec 4, 2023

openjdk bot commented Dec 4, 2023

RogerRiggs commented Oct 30, 2023 •

edited by openjdk bot

Loading

RogerRiggs Nov 6, 2023 •

edited

Loading

mlbridge bot commented Nov 6, 2023 •

edited

Loading