8313697: [XWayland][Screencast] consequent getPixelColor calls are slow #15250

azvegint · 2023-08-11T17:19:40Z

Right now, each call to getPixelColor/createScreenCapture opens and closes a ScreenCast session.
When there are many such calls in a row, it becomes a rather time-consuming operation.

For example,

for (int i = 0; i < 50; i++) {
    for (int j = 0; j < 50; j++) {
        robot.getPixelColor(i, j);
    }
}

50x50 took 375219ms

This fix offers optimization by keeping the session open and closing it only 2 seconds after the last activity.

50x50 took 28113ms which is ~ 13 times faster.

Testing looks good.

Progress

Change must be properly reviewed (1 review required, with at least 1 Reviewer)
Change must not contain extraneous whitespace
Commit message must refer to an issue

Issues

JDK-8313697: [XWayland][Screencast] consequent getPixelColor calls are slow (Bug - P3)
JDK-8310334: [XWayland][Screencast] screen capture error message in debug (Bug - P4)

Reviewers

Sergey Bylokhov (@mrserb - Reviewer)
Phil Race (@prrace - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/15250/head:pull/15250
$ git checkout pull/15250

Update a local copy of the PR:
$ git checkout pull/15250
$ git pull https://git.openjdk.org/jdk.git pull/15250/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 15250

View PR using the GUI difftool:
$ git pr show -t 15250

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/15250.diff

Webrev

Link to Webrev Comment

… from the one used by the current session.

bridgekeeper · 2023-08-11T17:21:15Z

👋 Welcome back azvegint! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

azvegint · 2023-08-11T17:21:21Z

src/java.desktop/unix/native/libawt_xawt/awt/screencast_pipewire.c

                fp_pw_stream_disconnect(screenProps->data->stream);
+                fp_pw_thread_loop_lock(pw.loop);


Moving pw_stream_disconnect call out of this lock fixes JDK-8310334

openjdk · 2023-08-11T17:23:31Z

@azvegint The following label will be automatically applied to this pull request:

client

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

mlbridge · 2023-08-11T17:41:30Z

Webrevs

prrace · 2023-08-11T21:57:41Z

src/java.desktop/unix/classes/sun/awt/screencast/ScreencastHelper.java

@@ -28,6 +28,7 @@
 import sun.awt.UNIXToolkit;
 import sun.security.action.GetPropertyAction;

+import javax.swing.Timer;


Is there a reason you can't use java.util.Timer ?
Ideally [*] AWT classes should not depend on Swing classes since already Swing classes depend on AWT classes ..

[*] yes, I know we already have such dependencies but only for very good reasons.

Of course we can use java.util.Timer. Swing timer is just more convenient to use. Updated.

mrserb · 2023-08-14T20:25:08Z

src/java.desktop/unix/classes/sun/awt/screencast/ScreencastHelper.java

+        timerTask = new TimerTask() {
+            @Override
+            public void run() {
+                closeSession();


What happen if the "closeSession" is executed why the new Robot#getRGBPixels is called? In the robot code we call timerCloseSessionRestart to restart/cancel timer, but I think it could be possible that closeSession() is already executed and may close session right before it will be used by the robot but after we "open" it.

Or this case is not possible?

Both ScreencastHelper#getRGBPixels(Robot#getRGBPixels calls it internally) and ScreencastHelper#closeSession are synchronized.
So ScreencastHelper#closeSession will wait until the execution of ScreencastHelper#getRGBPixels is finished and vice versa.

Many hours of testing with random delays close to DELAY_BEFORE_SESSION_CLOSE worked fine.

Java part looks good to me. Why 2s and not more or less ?
The delay could be set through a system property ?

Why 2s and not more or less ?

This is the average value chosen so as not to slow down tests execution, and not to annoy the user with the "Screen is being shared" icon while the session is still open.

The delay could be set through a system property ?

This functionality could be added, but I don't see much reason to do so.
If a screenshot is taken from time to time, the benefit of holding the session for a long time will be quite small compared to frequent requests.
Plus it requires some CSR work.

azvegint · 2023-08-15T18:44:03Z

/issue add JDK-8310334

openjdk · 2023-08-15T18:45:45Z

@azvegint
Adding additional issue to issue list: 8310334: [XWayland][Screencast] screen capture error message in debug.

mrserb

I think it is good as is, an additional option could be added if this functionality will work well and no other solution will not be found.

But we also can try to close the session if the robot object is collected by GC.

openjdk · 2023-08-16T05:29:35Z

@azvegint This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8313697: [XWayland][Screencast] consequent getPixelColor calls are slow
8310334: [XWayland][Screencast] screen capture error message in debug

Reviewed-by: serb, prr

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 135 new commits pushed to the master branch:

808bb1f: 8314246: javax/swing/JToolBar/4529206/bug4529206.java fails intermittently on Linux
6445314: 8314477: Improve definition of "prototypical type"
d27daf0: 8314129: Make fields final in java.util.Scanner
a8ab3be: 8314261: Make fields final in sun.net.www
3bb8afb: 8314489: Add javadoc index entries for java.lang.Math terms
2505ceb: 8314533: ProblemList runtime/cds/appcds/customLoader/HelloCustom_JFR.java on linux-all with ZGC
b33ff30: 8313661: [REDO] Relax prerequisites for java.base-jmod target
62ca001: 8313357: Revisit requiring SA tests on OSX to either run as root or use sudo
388dcff: 8282712: VMConnection.open() does not detect if VM failed to be created, resulting in NPE
e8f6b3e: 8314268: Missing include in assembler_riscv.hpp
... and 125 more: https://git.openjdk.org/jdk/compare/1da82a34b14189814e45a93c68620ccb51427111...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

azvegint · 2023-08-18T10:43:34Z

/integrate

openjdk · 2023-08-18T10:44:22Z

Going to push as commit 2f04bc5.
Since your change was applied there have been 140 commits pushed to the master branch:

33d5dfd: 8314543: gitattributes: make diffs easier to read
5058854: 8314389: AttachListener::pd_set_flag obsolete
891c3f4: 8307356: Metaspace: simplify BinList handling
0299364: 8314249: Refactor handling of invokedynamic in JVMCI ConstantPool
96778dd: 8314169: Combine related RoundingMode logic in j.text.DigitList
808bb1f: 8314246: javax/swing/JToolBar/4529206/bug4529206.java fails intermittently on Linux
6445314: 8314477: Improve definition of "prototypical type"
d27daf0: 8314129: Make fields final in java.util.Scanner
a8ab3be: 8314261: Make fields final in sun.net.www
3bb8afb: 8314489: Add javadoc index entries for java.lang.Math terms
... and 130 more: https://git.openjdk.org/jdk/compare/1da82a34b14189814e45a93c68620ccb51427111...master

Your commit was automatically rebased without conflicts.

openjdk · 2023-08-18T10:44:29Z

@azvegint Pushed as commit 2f04bc5.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

azvegint added 2 commits August 7, 2023 19:39

initial

8f5b128

Handle the case where a new screen capture requires a token different…

60ea277

… from the one used by the current session.

azvegint commented Aug 11, 2023

View reviewed changes

openjdk bot added the rfr Pull request is ready for review label Aug 11, 2023

openjdk bot added the client client-libs-dev@openjdk.org label Aug 11, 2023

prrace reviewed Aug 11, 2023

View reviewed changes

azvegint added 3 commits August 14, 2023 07:10

Make multiple calls to doCleanup() safe

81ff41d

switch to java.util.Timer

0b6e8f4

fix app exit issue

bb92788

mrserb reviewed Aug 14, 2023

View reviewed changes

mrserb approved these changes Aug 16, 2023

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Aug 16, 2023

prrace approved these changes Aug 17, 2023

View reviewed changes

openjdk bot added the integrated Pull request has been integrated label Aug 18, 2023

openjdk bot closed this Aug 18, 2023

openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Aug 18, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

8313697: [XWayland][Screencast] consequent getPixelColor calls are slow #15250

8313697: [XWayland][Screencast] consequent getPixelColor calls are slow #15250

azvegint commented Aug 11, 2023 •

edited by openjdk bot

bridgekeeper bot commented Aug 11, 2023

azvegint Aug 11, 2023

openjdk bot commented Aug 11, 2023

mlbridge bot commented Aug 11, 2023 •

edited

prrace Aug 11, 2023

azvegint Aug 14, 2023

mrserb Aug 14, 2023

azvegint Aug 14, 2023

bourgesl Aug 15, 2023

azvegint Aug 15, 2023 •

edited

azvegint commented Aug 15, 2023

openjdk bot commented Aug 15, 2023

mrserb left a comment

openjdk bot commented Aug 16, 2023 •

edited

azvegint commented Aug 18, 2023

openjdk bot commented Aug 18, 2023

openjdk bot commented Aug 18, 2023

		fp_pw_stream_disconnect(screenProps->data->stream);
		fp_pw_thread_loop_lock(pw.loop);

8313697: [XWayland][Screencast] consequent getPixelColor calls are slow #15250

8313697: [XWayland][Screencast] consequent getPixelColor calls are slow #15250

Conversation

azvegint commented Aug 11, 2023 • edited by openjdk bot

Progress

Issues

Reviewers

Reviewing

Webrev

bridgekeeper bot commented Aug 11, 2023

azvegint Aug 11, 2023

Choose a reason for hiding this comment

openjdk bot commented Aug 11, 2023

mlbridge bot commented Aug 11, 2023 • edited

Webrevs

prrace Aug 11, 2023

Choose a reason for hiding this comment

azvegint Aug 14, 2023

Choose a reason for hiding this comment

mrserb Aug 14, 2023

Choose a reason for hiding this comment

azvegint Aug 14, 2023

Choose a reason for hiding this comment

bourgesl Aug 15, 2023

Choose a reason for hiding this comment

azvegint Aug 15, 2023 • edited

Choose a reason for hiding this comment

azvegint commented Aug 15, 2023

openjdk bot commented Aug 15, 2023

mrserb left a comment

Choose a reason for hiding this comment

openjdk bot commented Aug 16, 2023 • edited

azvegint commented Aug 18, 2023

openjdk bot commented Aug 18, 2023

openjdk bot commented Aug 18, 2023

azvegint commented Aug 11, 2023 •

edited by openjdk bot

mlbridge bot commented Aug 11, 2023 •

edited

azvegint Aug 15, 2023 •

edited

openjdk bot commented Aug 16, 2023 •

edited