Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8319197: Exclude hb-subset and hb-style from compilation #16440

Closed
wants to merge 2 commits into from

Conversation

djelinski
Copy link
Member

@djelinski djelinski commented Oct 31, 2023

hb-subset and hb-style APIs are not used and not exported by libfontmanger. We can cut the compilation time by not compiling the unused files.

The added exclusions reduce the build time by ~1 minute (~8%) on my machine. This is with harfbuzz 8, with harfbuzz 7 the improvement was ~30 seconds.

Client libs tests continue to pass.


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed (2 reviews required, with at least 1 Reviewer, 1 Author)

Issue

  • JDK-8319197: Exclude hb-subset and hb-style from compilation (Enhancement - P4)

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/16440/head:pull/16440
$ git checkout pull/16440

Update a local copy of the PR:
$ git checkout pull/16440
$ git pull https://git.openjdk.org/jdk.git pull/16440/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 16440

View PR using the GUI difftool:
$ git pr show -t 16440

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/16440.diff

Webrev

Link to Webrev Comment

@bridgekeeper
Copy link

bridgekeeper bot commented Oct 31, 2023

👋 Welcome back djelinski! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk openjdk bot changed the title 8319197 8319197: Exclude hb-subset and hb-style from compilation Oct 31, 2023
@openjdk
Copy link

openjdk bot commented Oct 31, 2023

@djelinski The following labels will be automatically applied to this pull request:

  • build
  • client

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added build build-dev@openjdk.org client client-libs-dev@openjdk.org labels Oct 31, 2023
@djelinski djelinski marked this pull request as ready for review November 1, 2023 09:10
@openjdk openjdk bot added the rfr Pull request is ready for review label Nov 1, 2023
@mlbridge
Copy link

mlbridge bot commented Nov 1, 2023

Webrevs

Copy link
Member

@erikj79 erikj79 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me from a build point of view, but needs someone from client to approve as well.

/reviewers 2

@openjdk
Copy link

openjdk bot commented Nov 1, 2023

@djelinski This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8319197: Exclude hb-subset and hb-style from compilation

Reviewed-by: erikj, aivanov, prr

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 60 new commits pushed to the master branch:

  • e4803e0: 8318580: "javax/swing/MultiMonitor/MultimonVImage.java failing with Error. Can't find library: /open/test/jdk/java/awt/regtesthelpers" after JDK-8316053
  • cdf3373: 8319316: Clarify text around which layouts a linker supports
  • 1696603: 8308453: Convert JKS test keystores in test/jdk/javax/net/ssl/etc to PKCS12
  • b3126b6: 8319455: Test compiler/print/CompileCommandMemLimit.java times out
  • 1c2ea1d: 8319153: Fix: Class is a raw type in ProcessTools
  • 96e6e67: 4365952: Cannot disable JFileChooser
  • 2d4bbf4: 8319465: Typos in javadoc of com.sun.management.OperatingSystemMXBean methods
  • 8fb94fd: 8319379: G1: gc/logging/TestUnifiedLoggingSwitchStress.java crashes after JDK-8318894
  • b5c863b: 8316533: C2 compilation fails with assert(verify(phase)) failed: missing Value() optimization
  • 377138c: 8318959: C2: define MachNode::fill_new_machnode() statically
  • ... and 50 more: https://git.openjdk.org/jdk/compare/2182c93689f35c809f4d79a8002c838f8c4f56f3...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready Pull request is ready to be integrated label Nov 1, 2023
@openjdk
Copy link

openjdk bot commented Nov 1, 2023

@erikj79
The total number of required reviews for this PR (including the jcheck configuration and the last /reviewers command) is now set to 2 (with at least 1 Reviewer, 1 Author).

@openjdk openjdk bot removed the ready Pull request is ready to be integrated label Nov 1, 2023
Copy link
Member

@aivanov-jdk aivanov-jdk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

You may want to wait for Phil's approval though.

@openjdk openjdk bot added the ready Pull request is ready to be integrated label Nov 1, 2023
# hb-subset and hb-style APIs are not needed, excluded to cut on compilation time.
LIBFONTMANAGER_EXCLUDE_FILES += hb-ft.cc hb-subset-cff-common.cc \
hb-subset-cff1.cc hb-subset-cff2.cc hb-subset-input.cc hb-subset-plan.cc \
hb-subset.cc hb-subset-instancer-solver.cc gsubgpos-context.cc hb-style.cc
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm quite confused from where you got gsubgpos-context.cc. I don't see any file of that name.
Also I just realised that since the Nov 2020 import of freetype it has a bunch of handly ifdefs that
you can see in hb-config.h
And so probably a better way of doing this, even if it means one of these files (the instance-solver one) doesn't seem to get the ifdef treatment is to use the specific defines passed to the build.
Then I'd expect harfbuzz to apply the same ifdef even if these files got refactored.
I further noticed that the same 2020 update adds HAVE_FREETYPE and hb-ft.cc isn't compiled unless you actually set that. Which we don't
So this whole exclude can go away and be replaced by a few -D defines.
You could probably go overboard and experiment with all sorts of combinations from hb-config.h but I recommend you just apply the minimal set that makes your build time happier.
hb-style.cc isn't one I'd have realised we don't currently need .. and for many of them it isn't clear to me without research what the consequences would be of excluding it. Without that support would something non-optimal happen in some cases - you can't trust the tests we have to come near proving it.
I know we don't subset, and that's the only one I consider "safe" to use here without actual research.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These two
#define HB_NO_SUBSET_LAYOUT
#define HB_NO_SUBSET_CFF

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can find the file here:
https://github.com/openjdk/jdk/blob/f262f06c97b9ea94cd6119b3a8beb16bf804d083/src/java.desktop/share/native/libharfbuzz/graph/gsubgpos-context.cc

Hb-subset files are not guarded by ifdefs; HB_NO_SUBSET_LAYOUT only excludes some functions, but not others. I'll check the impact of that define when I'm back in office.

I checked the list of HB APIs here ; hb-subset and hb-style belong to non-core API sets, so I decided to check if hb-style was needed. It was not.

We can safely stop compiling the files listed because:

  • none of the harfbuzz functions are exported, and they are not accessible outside of libfontmanager.
  • if we excluded any file that was actually used, linker would refuse to link libfontmanager

Given that we only copy an arbitrary subset of harfbuzz files anyway (see UPDATING.txt), would it make sense to remove these files instead?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW, I compiled the code without this PR, but with HARFBUZZ_CFLAGS += -DHB_NO_SUBSET_LAYOUT -DHB_NO_SUBSET_CFF instead, and checked make LOG=profile output. Results:

  • without this change, compiling hb-subset.cc took 56 seconds, and hb-subset-plan.cc took 33 seconds
  • with this change, compiling hb-subset.cc took 33 seconds, and hb-subset-plan.cc took 22 seconds

It's a nice improvement, but not compiling these files at all is still much better.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@djelinski Just curious; what would the effect be to both include this change and setting the NO_* defines?

If all the references to these defines are made in the excluded files then the only reason for doing that would be some kind of information to subsequent readers of the code, but they might also be checked elsewhere, and thus give an additional speedup.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HB_NO_SUBSET_CFF is indeed referenced in 2 header files, which are in turn included somewhere else.
I added the defines on top of the excludes, but this resulted in no measurable build time difference.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity, I tried building with HB_LEAN, which is a shortcut for "no optional features"; I got a couple of new warnings, and one error (hb_font_funcs_set_glyph_v_kerning_func is deprecated since 2.0 and not compiled with HB_LEAN; fortunately its use in hb-jdk-font.cc can be simply commented out). After resolving these problems, fontmanager compiled successfully - but again, no measurable time difference.

@prrace
Copy link
Contributor

prrace commented Nov 6, 2023

So if I have this right,
(1) excluding several unused files from the build saves 60 seconds.
(2) Using the two specific #defines instead saves about 34 seconds.
(3) The two specific #defines on top of the excludes from (1) made no difference
(4) HB_LEAN had no impact either .. I presume you mean when used on top of (1) as well, which suggests that most of what's in there is cheap to compile.

The build exclusion does however include another couple of files, at least one of which is unrelated, not sure if you ever said
how much that saved, but that presumably accounts for some part of 60 sec vs 34 secs

@djelinski
Copy link
Member Author

(1) - (4) - that's correct.

My tests were hardly scientific; most of the time I only deleted and recompiled libfontmanager using 8 parallel jobs; the time taken by the recompilation was in the 50-75 seconds range both with and without HB_LEAN.
Without the exclusions, the rebuild of libfontmanager takes ~2 minutes.

File build times from a sample run, taken from build-profile.log:

file time without any changes time with -DHB_NO_SUBSET_LAYOUT -DHB_NO_SUBSET_CFF
hb-subset-cff-common.cc 7.46s 2.68s
hb-subset-cff1.cc 14.06s 2.47s
hb-subset-cff2.cc 13.14s 2.82s
hb-subset-input.cc 5.70 s 4.36s
hb-subset-plan.cc 33.62s 22.27s
hb-subset.cc 56.15s 33.33s
hb-subset-instancer-solver.cc 4.81s 2.81s
gsubgpos-context.cc 11.66s 7.48s
hb-style.cc 9.49s 6.67s

Copy link
Contributor

@prrace prrace left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, let's just go with the exclusion at least for now.
If it becomes a problem on a subsequent upgrade it may need to be changed.
We can also explore other options later.

@djelinski
Copy link
Member Author

Thanks!

/integrate

@openjdk
Copy link

openjdk bot commented Nov 7, 2023

Going to push as commit e1cae72.
Since your change was applied there have been 62 commits pushed to the master branch:

  • 419ed90: 8319450: New methods java.net.InetXAddress.ofLiteral() miss @SInCE tag
  • 439ed04: 8319233: AArch64: Build failure with clang due to -Wformat-nonliteral warning
  • e4803e0: 8318580: "javax/swing/MultiMonitor/MultimonVImage.java failing with Error. Can't find library: /open/test/jdk/java/awt/regtesthelpers" after JDK-8316053
  • cdf3373: 8319316: Clarify text around which layouts a linker supports
  • 1696603: 8308453: Convert JKS test keystores in test/jdk/javax/net/ssl/etc to PKCS12
  • b3126b6: 8319455: Test compiler/print/CompileCommandMemLimit.java times out
  • 1c2ea1d: 8319153: Fix: Class is a raw type in ProcessTools
  • 96e6e67: 4365952: Cannot disable JFileChooser
  • 2d4bbf4: 8319465: Typos in javadoc of com.sun.management.OperatingSystemMXBean methods
  • 8fb94fd: 8319379: G1: gc/logging/TestUnifiedLoggingSwitchStress.java crashes after JDK-8318894
  • ... and 52 more: https://git.openjdk.org/jdk/compare/2182c93689f35c809f4d79a8002c838f8c4f56f3...master

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot added the integrated Pull request has been integrated label Nov 7, 2023
@openjdk openjdk bot closed this Nov 7, 2023
@openjdk openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Nov 7, 2023
@openjdk
Copy link

openjdk bot commented Nov 7, 2023

@djelinski Pushed as commit e1cae72.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@djelinski djelinski deleted the hb-subset branch November 7, 2023 09:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build build-dev@openjdk.org client client-libs-dev@openjdk.org integrated Pull request has been integrated
Development

Successfully merging this pull request may close these issues.

5 participants