Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8315968: Move java.util.Digits to jdk.internal.util and refactor to reduce duplication #15651

Closed
wants to merge 3 commits into from

Conversation

wenshao
Copy link
Contributor

@wenshao wenshao commented Sep 10, 2023

java.util.DecimalDigits::DIGITS and java.lang.StringLatin1.PACKED_DIGITS are duplicates, We need to move java.util.Digits/OctalDigits/DecimalDigits/HexDigits to the jdk.internal.util package, and modify these classes to public class, so that classes in other packages can also access them.

DecimalDigits::DIGITS provides a new digitPair static method, used to replace StringLatin1.PACKED_DIGITS access.

In order to be consistent with the original StringLatin1.PACKED_DIGITS, OctalDigits::DIGITS and DecimalDigits::DIGITS are modified to little-endian. HexDigits::DIGITS will also be modified after PR #14745 is merged.

These changes will be used by PR #15658 #15555


Progress

  • Change must be properly reviewed (1 review required, with at least 1 Reviewer)
  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue

Issue

  • JDK-8315968: Move java.util.Digits to jdk.internal.util and refactor to reduce duplication (Enhancement - P4)

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/15651/head:pull/15651
$ git checkout pull/15651

Update a local copy of the PR:
$ git checkout pull/15651
$ git pull https://git.openjdk.org/jdk.git pull/15651/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 15651

View PR using the GUI difftool:
$ git pr show -t 15651

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/15651.diff

Webrev

Link to Webrev Comment

@bridgekeeper
Copy link

bridgekeeper bot commented Sep 10, 2023

👋 Welcome back wenshao! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk openjdk bot added the rfr Pull request is ready for review label Sep 10, 2023
@openjdk
Copy link

openjdk bot commented Sep 10, 2023

@wenshao The following labels will be automatically applied to this pull request:

  • core-libs
  • security

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added security security-dev@openjdk.org core-libs core-libs-dev@openjdk.org labels Sep 10, 2023
@mlbridge
Copy link

mlbridge bot commented Sep 10, 2023

@RogerRiggs
Copy link
Contributor

RogerRiggs commented Sep 10, 2023

Instead of packing more stuff into SharedSecrets, how about moving some common utilities into package jdk.internal.util or a new package jdk.internal.string.
For example, Digits, HexDigits, DecimalDigits, OctalDigits.
That would make access better without the overhead shared secrets.

@cl4es
Copy link
Member

cl4es commented Sep 10, 2023

Yes, moving java.util.Digits to jdk.internal.util was what I had in mind, too. SharedSecrets are only necessary when we have things that need to stay package-private, such as member functions on String. Static utilities like this are generally easier to maintain when they are public but in a non-exported package like jdk.internal.util.

Copy link
Contributor

@RogerRiggs RogerRiggs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would cleaner and easier to review, if you moved the classes together and did not refactor the functions. Only include the dependencies. Update the functions in a separate PR. (We can retitle the issue to match).

src/java.base/share/classes/java/lang/Long.java Outdated Show resolved Hide resolved
src/java.base/share/classes/java/util/Digits.java Outdated Show resolved Hide resolved
Copy link
Member

@liach liach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In short: I suggest splitting the change of moving getChars stringSize into another patch; they are sufficiently distinct and can shrink this PR's size by a half at least.

总结:推荐这个PR专门处理移除StringLatin1.PACKED_DIGITS字段和Digits移包。
迁移getCharsstringSize分另外一个补丁。这两个内容唯一共通点是StringLatin1.getChars有些冲突,但是除此之外没必要两个补丁合并,大幅增加代码改动,难review,回滚和git blame,比如这个补丁里面Long AbstractStringBuilder等一堆改动实际上是这两个method,和PACKED_DIGITS无关。

* wins. The iteration results are also routinely inlined in the generated
* code after loop unrolling.
*/
public static int stringSize(int x) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest splitting the moves of stringSize getChars into a new PR dependent on this one; your future date and time optimizations can depend on that one, which exposes stringSize.

Having the DecimalDigits package move and stringSize getChars moves together complicates the file changes, and it's hard to detect if there's any accidental typo/malicious code in the new additions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this PR is split into two PRs, the other two PRs I submitted #15658 #15555 cannot be based on this PR.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, do the package move separate from the refactoring.
The other PRs can wait a bit or be committed as is and take part in the refactoring later.
One step at a time please.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refactoring of stringSize has been restored

src/java.base/share/classes/java/lang/StringUTF16.java Outdated Show resolved Hide resolved
src/java.base/share/classes/java/lang/StringUTF16.java Outdated Show resolved Hide resolved
src/java.base/share/classes/java/lang/StringUTF16.java Outdated Show resolved Hide resolved
src/java.base/share/classes/java/lang/StringUTF16.java Outdated Show resolved Hide resolved
src/java.base/share/classes/java/lang/StringUTF16.java Outdated Show resolved Hide resolved
src/java.base/share/classes/java/lang/StringLatin1.java Outdated Show resolved Hide resolved
@openjdk
Copy link

openjdk bot commented Sep 11, 2023

@wenshao this pull request can not be integrated into master due to one or more merge conflicts. To resolve these merge conflicts and update this pull request you can run the following commands in the local repository for your personal fork:

git checkout reduce_duplicate
git fetch https://git.openjdk.org/jdk.git master
git merge FETCH_HEAD
# resolve conflicts and follow the instructions given by git merge
git commit -m "Merge master"
git push

@openjdk openjdk bot added merge-conflict Pull request has merge conflict with target branch and removed merge-conflict Pull request has merge conflict with target branch labels Sep 11, 2023
wenshao added a commit to wenshao/jdk that referenced this pull request Sep 11, 2023
* wins. The iteration results are also routinely inlined in the generated
* code after loop unrolling.
*/
public static int stringSize(int x) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, do the package move separate from the refactoring.
The other PRs can wait a bit or be committed as is and take part in the refactoring later.
One step at a time please.

@@ -0,0 +1,294 @@
/*
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can git be convinced to show this as a rename instead of a delete and add?
The history will be cleaner.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just looked it up and git actually doesn't have commands for renames like hg, but rather decides if something is a delete+add or a rename based on the amount of changes. Too many changes and it'll automatically look like a remove+add.

Which is yet another argument for splitting up this in multiple PRs. One to move, one to refactor.

Copy link
Contributor Author

@wenshao wenshao Sep 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refactoring of stringSize has been restored, the problem of 'delete & add' has been resolved

@openjdk
Copy link

openjdk bot commented Sep 11, 2023

@wenshao Please do not rebase or force-push to an active PR as it invalidates existing review comments. Note for future reference, the bots always squash all changes into a single commit automatically as part of the integration. See OpenJDK Developers’ Guide for more information.

@openjdk
Copy link

openjdk bot commented Sep 11, 2023

@wenshao Please do not rebase or force-push to an active PR as it invalidates existing review comments. Note for future reference, the bots always squash all changes into a single commit automatically as part of the integration. See OpenJDK Developers’ Guide for more information.

@openjdk
Copy link

openjdk bot commented Sep 11, 2023

@wenshao Please do not rebase or force-push to an active PR as it invalidates existing review comments. Note for future reference, the bots always squash all changes into a single commit automatically as part of the integration. See OpenJDK Developers’ Guide for more information.

@openjdk
Copy link

openjdk bot commented Sep 12, 2023

@wenshao Please do not rebase or force-push to an active PR as it invalidates existing review comments. Note for future reference, the bots always squash all changes into a single commit automatically as part of the integration. See OpenJDK Developers’ Guide for more information.

wenshao added a commit to wenshao/jdk that referenced this pull request Sep 12, 2023
@cl4es
Copy link
Member

cl4es commented Sep 12, 2023

Running some additional testing. This mostly looks fine.

One issue is that you're swapping the byte-order in DecimalDigits::DIGITS but not in OctalDigits and HexDigits. I think we need to keep these internally consistent to avoid surprises.

I also would like to see performance numbers of the byte order swap evaluated in isolation. I suspect the real effect is small and might be due to JIT noise rather than a real effect, and that this swap got rushed in without solid evidence that it helps.

If there's no significant performance difference I would prefer if we kept DecimalDigits::DIGITS big-endian encoded - which is more intuitive to most - and adjust code depending on DecimalDigits::digitPair to use ByteArray rather than ByteArrayLittleEndian.

@cl4es
Copy link
Member

cl4es commented Sep 12, 2023

If there's no significant performance difference I would prefer if we kept DecimalDigits::DIGITS big-endian encoded - which is more intuitive to most - and adjust code depending on DecimalDigits::digitPair to use ByteArray rather than ByteArrayLittleEndian.

Just a datapoint but when I test implementing HexFormat::toHexDigits using either ByteArray or ByteArrayLittleEndian then the difference is in the noise:

Name                           Cnt  Base   Error   Test   Error  Unit   Diff%
HexFormatBench.toHexDigitsLong  15 1,950 ± 0,012  1,941 ± 0,016 us/op    0,5% (p = 0,081 )

@wenshao
Copy link
Contributor Author

wenshao commented Sep 12, 2023

Running some additional testing. This mostly looks fine.

One issue is that you're swapping the byte-order in DecimalDigits::DIGITS but not in OctalDigits and HexDigits. I think we need to keep these internally consistent to avoid surprises.

I also would like to see performance numbers of the byte order swap evaluated in isolation. I suspect the real effect is small and might be due to JIT noise rather than a real effect, and that this swap got rushed in without solid evidence that it helps.

If there's no significant performance difference I would prefer if we kept DecimalDigits::DIGITS big-endian encoded - which is more intuitive to most - and adjust code depending on DecimalDigits::digitPair to use ByteArray rather than ByteArrayLittleEndian.

I prefer to use little endian, most environments are little endian. Changes to HexDigit will have to wait until #PR 14745 is merged.

@RogerRiggs
Copy link
Contributor

Please update this PR title and description to indicate this refactoring to move to jdk.internal.util.
I can update the Jira title and description to match after that.

@wenshao wenshao changed the title 8315968: Consolidate java.util.Digits and StringLatin1::PACKED_DIGITS 8315968: Move java.util.Digits to jdk.internal.util and refactor Sep 12, 2023
@openjdk openjdk bot changed the title 8315968: Move java.util.Digits to jdk.internal.util and refactor 8315968: Move java.util.Digits to jdk.internal.util and refactor Sep 12, 2023
@wenshao wenshao changed the title 8315968: Move java.util.Digits to jdk.internal.util and refactor 8315968: Move java.util.Digits to jdk.internal.util and refactor to reduce duplication Sep 12, 2023
@wenshao
Copy link
Contributor Author

wenshao commented Sep 12, 2023

Please update this PR title and description to indicate this refactoring to move to jdk.internal.util. I can update the Jira title and description to match after that.

The title has been updated, please help update the title of JIRA

@RogerRiggs
Copy link
Contributor

The title has been updated, please help update the title of JIRA

The description needs an update too.

@liach
Copy link
Member

liach commented Sep 12, 2023

I have updated the JBS bug title and explicitly stated the actions in this patch:

  1. Move Digits to jdk.internal.util
  2. Keep the tables in DecimalDigits and access via a new digitPair static method
  3. Change the DecimalDigits' table to be LE (instead of BE) and update implementation

@wenshao
Copy link
Contributor Author

wenshao commented Sep 12, 2023

The title has been updated, please help update the title of JIRA

The description needs an update too.

The description has also been updated

@cl4es
Copy link
Member

cl4es commented Sep 12, 2023

I prefer to use little endian, most environments are little endian. Changes to HexDigit will have to wait until #PR 14745 is merged.

I don't have a very strong opinion except that we should be consistent across these related implementations. If you could either include OctalDigits changes in #14745 or prepare a separate PR for it I'm OK with this.

@openjdk
Copy link

openjdk bot commented Sep 12, 2023

⚠️ @wenshao the full name on your profile does not match the author name in this pull requests' HEAD commit. If this pull request gets integrated then the author name from this pull requests' HEAD commit will be used for the resulting commit. If you wish to push a new commit with a different author name, then please run the following commands in a local repository of your personal fork:

$ git checkout reduce_duplicate
$ git commit --author='Preferred Full Name <you@example.com>' --allow-empty -m 'Update full name'
$ git push

@openjdk
Copy link

openjdk bot commented Sep 12, 2023

@wenshao This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8315968: Move java.util.Digits to jdk.internal.util and refactor to reduce duplication

Reviewed-by: rriggs, liach, redestad

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 12 new commits pushed to the master branch:

  • d75d977: 8285447: StackWalker minimal batch size should be optimized for getCallerClass
  • fc3e826: 8314832: Few runtime/os tests ignore vm flags
  • 347beb2: 8315998: Remove dead ClassLoaderDataGraphKlassIteratorStatic
  • 50158f3: 8316002: Remove unnecessary seen_dead_loader in ClassLoaderDataGraph::do_unloading
  • 6f2684b: 8315948: JDK-8315818 broke Xcomp on libgraal
  • 8b4f9a8: 8315990: Amend problemlisted tests to proper position
  • 455c471: 8313277: Resolve multiple definition of 'normalize' when statically linking JDK native libraries with user code
  • 3b422d0: 8316094: Problemlist compiler/rangechecks/TestRangeCheckHoistingScaledIV.java
  • f55e799: 8316038: Fix doc typos in java.io.Console and java.util.Scanner
  • 9480078: 8315550: G1: Fix -Wconversion warnings in g1NUMA
  • ... and 2 more: https://git.openjdk.org/jdk/compare/d0be73a78038faf9509623bc4ba71eb4385cd645...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

As you do not have Committer status in this project an existing Committer must agree to sponsor your change. Possible candidates are the reviewers of this PR (@RogerRiggs, @cl4es) but any other Committer may sponsor as well.

➡️ To flag this PR as ready for integration with the above commit message, type /integrate in a new comment. (Afterwards, your sponsor types /sponsor in a new comment to perform the integration).

@openjdk openjdk bot added the ready Pull request is ready to be integrated label Sep 12, 2023
Copy link
Contributor

@RogerRiggs RogerRiggs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates.

@wenshao
Copy link
Contributor Author

wenshao commented Sep 12, 2023

/integrate

@openjdk openjdk bot added the sponsor Pull request is ready to be sponsored label Sep 12, 2023
@openjdk
Copy link

openjdk bot commented Sep 12, 2023

@wenshao
Your change (at version c68c4e8) is now ready to be sponsored by a Committer.

@cl4es
Copy link
Member

cl4es commented Sep 12, 2023

/sponsor

@openjdk
Copy link

openjdk bot commented Sep 12, 2023

Going to push as commit e084516.
Since your change was applied there have been 12 commits pushed to the master branch:

  • d75d977: 8285447: StackWalker minimal batch size should be optimized for getCallerClass
  • fc3e826: 8314832: Few runtime/os tests ignore vm flags
  • 347beb2: 8315998: Remove dead ClassLoaderDataGraphKlassIteratorStatic
  • 50158f3: 8316002: Remove unnecessary seen_dead_loader in ClassLoaderDataGraph::do_unloading
  • 6f2684b: 8315948: JDK-8315818 broke Xcomp on libgraal
  • 8b4f9a8: 8315990: Amend problemlisted tests to proper position
  • 455c471: 8313277: Resolve multiple definition of 'normalize' when statically linking JDK native libraries with user code
  • 3b422d0: 8316094: Problemlist compiler/rangechecks/TestRangeCheckHoistingScaledIV.java
  • f55e799: 8316038: Fix doc typos in java.io.Console and java.util.Scanner
  • 9480078: 8315550: G1: Fix -Wconversion warnings in g1NUMA
  • ... and 2 more: https://git.openjdk.org/jdk/compare/d0be73a78038faf9509623bc4ba71eb4385cd645...master

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot added the integrated Pull request has been integrated label Sep 12, 2023
@openjdk openjdk bot closed this Sep 12, 2023
@openjdk openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review sponsor Pull request is ready to be sponsored labels Sep 12, 2023
@openjdk
Copy link

openjdk bot commented Sep 12, 2023

@cl4es @wenshao Pushed as commit e084516.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core-libs core-libs-dev@openjdk.org integrated Pull request has been integrated security security-dev@openjdk.org
6 participants