8211294: ScrollPane content is blurry with 125% scaling

[fthevenet]

This PR aims to fix the blurriness to sometimes affects some controls (such as TextArea) in a scene when rendered with a scaling factor that is not an integer (typically when viewed on a HiDPI screen with a 125%, 150% or 175% output scaling).

Please note that regardless of what the JBS issue (and therefore the title of this PR) states, this does not appear to be a Windows only issue as I have observed the same issue on Linux (Ubuntu 20.04).

The following conditions are necessary for the blurriness to appear, but do not guarantee that it will:

• The node, or one of its parents, must have set Node::cacheProperty to true

• The scaling factor (as detected automatically or explicitly set via glass.win/gtk.uiScale) must be a non integer number (e.g. 1.25, 1.5, 175)

• The effective layout X or Y coordinates for the cached node must be != 0;

Under these conditions, the translate coordinates for the transform used when the cached image for a node is rendered to the screen may be non integer numbers, which is the cause for the blurriness.

Based on these observations, this PR fixes the issue by simply rounding the translate coordinates (using Math.round) before the transform is applied in renderCacheToScreen() and as far as I can tell, it fixes the blurriness in all the previously affected applications (both trivial test cases or with complex scenegraphs) and does not appear to introduce other noticeable visual artifacts.

Still, there might be a better place somewhere else higher up in the call chain where this should be addressed as it could maybe be the root cause for other rendering glitches, though I'm not yet familiar enough with the code to see if it is really the case.

---

Progress

• Change must not contain extraneous whitespace
• Commit message must refer to an issue
• Change must be properly reviewed

Issue

• JDK-8211294: ScrollPane content is blurry with 125% scaling

Reviewers

• Kevin Rushforth (@kevinrushforth - Reviewer)
• Ambarish Rapte (@arapte - Reviewer)

Download

$ git fetch https://git.openjdk.java.net/jfx pull/308/head:pull/308
$ git checkout pull/308

[bridgekeeper]

👋 Welcome back fthevenet! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[mlbridge]

Webrevs

• 05: Full - Incremental (12db928)
• 04: Full - Incremental (969f23f)
• 03: Full - Incremental (c448691)
• 02: Full - Incremental (5720f03)
• 01: Full - Incremental (4087c06)
• 00: Full (cce931d)

[fthevenet]

Also, I don't really have an idea on how this could be tested other than visually, so I'm open to suggestions.

[johanvos]

The visual representation corresponds with digits, so there can be tests that check if the numbers are what we expect them to be.
It's good that this is not windows-only, so that it can be tackled on linux as well. But what is not clear to me: does this require a physical HiDPI screen, or is setting the scale factor manually good enough to reproduce the bug?

[danielpeintner]

FYI: there is another bug JDK-8199592 that reads somewhat similar even though the issues are very different. It is also marked as a "Windows" bug.
Maybe the root of both issues is the same...

[fthevenet]

The visual representation corresponds with digits, so there can be tests that check if the numbers are what we expect them to be.
It's good that this is not windows-only, so that it can be tackled on linux as well. But what is not clear to me: does this require a physical HiDPI screen, or is setting the scale factor manually good enough to reproduce the bug?

The issue will appear consistently as long as the conditions I listed are met, regardless of the actual number of pixels the physical screen can display.
For instance you'll see the problem, if you apply a 125% scaling on 1080p screen (a common configuration on 13'' laptops).
Also, it will occur regardless of whether the scaling is applied at the OS level and picked up by javafx or if it is only set for a single application using the glass.xxx.uiScale property.

[kevinrushforth]

/reviewers 2

[openjdk]

@kevinrushforth
The number of required reviews for this PR is now set to 2 (with at least 1 of role reviewers).

[fthevenet]

Hello,
Did anyone get a chance to look into this?
Thanks!

[kevinrushforth]

Not yet. I took a quick look, and this is helpful in pointing out where the problem is, but I don't know whether it is the right solution to simply round the transform values when rendering the cache to the screen.

[fthevenet]

Hi,

Is there anything I can do to help get this moving forward again?

As mentioned before, I agree the proposed fix might be simply be addressing the symptoms rather than the root cause and I'm willing to keep working on it but I'd really appreciate a second opinion and some insights as to where to direct my attention if this fix proves to be not enough.

For what it's worth, I've been living with this fix on my dev machine for a while now, and it does provide the expected relief to the underlying issue, without me noticing any obvious regressions, neither visually nor performance-wise (I haven't run any targeted benchmarks, though).

With JDK-8199592 fixed, I think it'd be great if we could tackle all of the scaling issues in 16 (might be getting a bit late now, though)

[kevinrushforth]

I hope to take a look at this, along with other pending reviews, early next week. There is still some time to get this into 16 if we can find a robust fix.

[fthevenet]

I hope to take a look at this, along with other pending reviews, early next week. There is still some time to get this into 16 if we can find a robust fix.

That's great news, thanks!

[kevinrushforth]

I spent a bit of time looking at this. I think the root cause of the problem is in ScrollPane itself. It is attempting to layout its children by doing a snap to pixel (meaning that the final scaled translation should be an integer value), but it is failing to do so. This is mostly not a problem when caching is disabled, since our text rendering does sub-pixel antialiasing that looks crisp even at non-integer boundaries. However, translating an already-rendered image by a non-integer boundary will cause the blurriness we are seeing. There is another issue with the Y translation which isn't 0 even when not using a ScrollPane.

I'll continue looking at this in the coming week.

[kevinrushforth]

One more comment: given the quality problems that necessarily arise when the translation of a cached image is not on an integer boundary, part of the solution might be to snap the cached image to a pixel boundary as is done in this PR, but we would need to ensure that this doesn't impact smooth scrolling of a TextArea.

[fthevenet]

Further investigations on my part raised one more question, which hopefully you can answer:
To which extend should setSnapToPixel ensure children of a region are indeed snap to whole pixel coordinates?

To make it clearer, please consider the following sample:

public class Blur extends Application {
    @Override
    public void start(final Stage stage) throws Exception {
        var root = new Pane();
        root.setSnapToPixel(true);
        var ctrl = new CheckBox("Cached");

        ctrl.setLayoutX(0.5);
        ctrl.setLayoutY(0.5);
        ctrl.cacheProperty().bind(ctrl.selectedProperty());
        ctrl.setSelected(true);

        var ctrl2 = new Button("Foo");
        ctrl2.setLayoutX(0.5);
        ctrl2.setLayoutY(30.5);
        ctrl2.cacheProperty().bind(ctrl.selectedProperty());

        var ctrl3 = new Button("Bar");
        ctrl3.setLayoutY(60);
        ctrl3.cacheProperty().bind(ctrl.selectedProperty());

        root.getChildren().addAll(ctrl, ctrl2, ctrl3);
        Scene scene;
        scene = new Scene(root, 200, 200);

        stage.setTitle("Blur test");
        stage.setScene(scene);
        stage.show();
    }

    public static void main(String[] args) {

        launch(args);
    }
}

In this sample, LayoutX and Y properties are deliberately set to non integer values for the first two controls (the last one serves as a visual baseline, but snapToPixel is set to true. Also clicking the check box toggles caching for all controls.

Here's what it looks looks at 100% scaling, (OpenJFX 15.0.1), with cache enabled:

and with cache disabled:

What is the legitimate result to expect here; should root.setSnapToPixel(true); override setLayoutX(0.5); and align everything for crisp rendering? Or am I misunderstanding the scope of setSnapToPixel and it has no effect when layout is set explicitly?

Also, for comparison here is what it looks like when render with the patch proposed in this PR:
With cache on

And off:

Note that now text is crisp in all circumstances, but graphics (as seen in the border of button 'foo') are only crisp when cache is on (and the code in the patch kicks in).

[fthevenet]

One more comment: given the quality problems that necessarily arise when the translation of a cached image is not on an integer boundary, part of the solution might be to snap the cached image to a pixel boundary as is done in this PR, but we would need to ensure that this doesn't impact smooth scrolling of a TextArea.

I've done some quick tests with the minimal sample below, and could not notice any performance impact with this patch compared to 15.0.1.

public class TextScroll extends Application {

    @Override
    public void start(Stage stage) throws Exception {
        var textArea = new TextArea();
        var txt = new File(getClass().getResource("/Scrollpane.java").getFile());
        textArea.setText(Files.readString(txt.toPath(), StandardCharsets.UTF_8));
        var root = new ScrollPane(textArea);
        root.setFitToHeight(true);
        root.setFitToWidth(true);
        Scene scene;
        scene = new Scene(root, 800, 600);
        stage.setTitle("Scroll test");
        stage.setScene(scene);
        stage.show();
    }

    public static void main(String[] args) {
        launch(args);
    }
}

[kevinrushforth]

Actually, when I mentioned smooth scrolling, it wasn't performance I was thinking of, it was the ability to scroll by fractional pixel amounts, but with the default snap-to-pixel enabled, that won't happen anyway.

I'll look into the other questions you raised tomorrow, specifically what we might want to do about rendering the cache, and also your question about snapToPixel.

As for the specific problem with ScrollPane, I found the cause of that one. There is a (long-standing from the look of it) bug in Region::updateSnappedInsets. It attempts to account for snap-to-pixel, but does so incorrectly, using Math.ceil without taking screen scaling into account (that is, without calling the snapXxxx methods). The correct code should be something like:

            final boolean snap = isSnapToPixel();
            snappedTopInset = snapSizeY(insets.getTop(), snap);
            snappedRightInset = snapSizeX(insets.getRight(), snap);
            snappedBottomInset = snapSizeY(insets.getBottom(), snap);
            snappedLeftInset = snapSizeX(insets.getLeft(), snap);

This fixes the ScrollPane problem without needing to modify the cached rendering code.

[fthevenet]

Thanks for that.

I am wondering; is it ok to potentially address both sub-issues discussed here (Scrollpane snaping vs cache rendering) under the same JBS bug? Or would it be better to address them under separate issues?

Speaking of which, the JBS issue title is in fact misleading, as the issue appears to be not specific to Windows; what's the best practice in such cases; rename the issue? Open a new one? Leave it as is?

[kevinrushforth]

I am wondering; is it ok to potentially address both sub-issues discussed here (Scrollpane snaping vs cache rendering) under the same JBS bug? Or would it be better to address them under separate issues?

Since both issues -- the snap-to-pixel bug and the rendering of the cached image -- are causing blurriness, it could be OK in principle to address both of them as part of the same bug fix.

However, in this instance the snap-to-pixel bug has a simple, well-understood, and safe solution, while the cached rendering bug isn't nearly as simple; there are a couple ways it could be fixed, each with their own implications. Given that, I would prefer to address the snap-to-pixel bug here (which can easily make jfx16), and file a follow-up bug for the cached rendering.

the JBS issue title is in fact misleading, as the issue appears to be not specific to Windows; what's the best practice in such cases; rename the issue? Open a new one? Leave it as is?

This is easily fixed by renaming the JBS issue, and then updating the PR title to match. I'll update the JBS issue now, after which you can update the PR title.

Here are some thoughts about the cached rendering, which should find their way into the new issue:

Whatever we do to fix this, the end result should be that rendering a shape or control into a cache and then rendering that cached image should match rendering that shape or control directly. This should be true the first time it is rendered, and should remain true as long as the transform is unchanged (or differs only by a translation delta of whole pixel values) from when the cache was rendered into. This is clearly broken for rendering text if the translation is not on a pixel boundary.

Which leads into a question you asked.

What is the legitimate result to expect here; should root.setSnapToPixel(true); override setLayoutX(0.5); and align everything for crisp rendering? Or am I misunderstanding the scope of setSnapToPixel and it has no effect when layout is set explicitly?

A Pane will not force layoutX and layoutY to be on an integer boundary, since it is documented to not alter the position of any of its children. The subclasses of Pane do layout their children, so snap-to-pixel will take effect. So the fact that the controls in your most recent example are being rendered on a non-pixel boundary is not a bug. The fact that the text is so blurry when rendered from a cached image is a bug (as mentioned above).

[kevinrushforth]

For completeness, here is the patch for the snap-to-pixel issue:

diff --git a/modules/javafx.graphics/src/main/java/javafx/scene/layout/Region.java b/modules/javafx.graphics/src/main/java/javafx/scene/layout/Region.java
index 565d52b516..00c0f6da61 100644
--- a/modules/javafx.graphics/src/main/java/javafx/scene/layout/Region.java
+++ b/modules/javafx.graphics/src/main/java/javafx/scene/layout/Region.java
@@ -989,17 +989,11 @@ public class Region extends Parent {
     /** Called to update the cached snapped insets */
     private void updateSnappedInsets() {
         final Insets insets = getInsets();
-        if (_snapToPixel) {
-            snappedTopInset = Math.ceil(insets.getTop());
-            snappedRightInset = Math.ceil(insets.getRight());
-            snappedBottomInset = Math.ceil(insets.getBottom());
-            snappedLeftInset = Math.ceil(insets.getLeft());
-        } else {
-            snappedTopInset = insets.getTop();
-            snappedRightInset = insets.getRight();
-            snappedBottomInset = insets.getBottom();
-            snappedLeftInset = insets.getLeft();
-        }
+        final boolean snap = isSnapToPixel();
+        snappedTopInset = snapSizeY(insets.getTop(), snap);
+        snappedRightInset = snapSizeX(insets.getRight(), snap);
+        snappedBottomInset = snapSizeY(insets.getBottom(), snap);
+        snappedLeftInset = snapSizeX(insets.getLeft(), snap);
     }
 
     /**

We will need a test case for this. You can see UIRenderSceneTest.java for an example of a test that forces Hi-DPI scaling.

[kevinrushforth]

In looking at the other instances where snap-to-pixel is done correctly for Insets, the above should use snapSpace{X,Y} rather than snapSize{X,Y}

[kevinrushforth]

I did some testing with your latest patch, and I don't see any problem when dragging a window between screens with different scales. It seems to be recalculating the cached insets and using them in layout as I would expect. Do you have a test case that shows the problem?

[fthevenet]

Unfortunately I've only seen it within an application with a fairly complex scene graph, which make isolating the issue tricky.

I'm still trying to reproduce it a minimal sample, but in the mean time, please have a look at the a screen capture below, which should at least help clarify the issue I'm observing:

This snippet is captured after the window had been moved from a screen scaled to 100% to a second one scaled to 150%; notice how the controls in the "Chart Properties" pane only snap to pixel when the mouse pointer enters the pane.

[fthevenet]

Digging deeper into my own code, it appears that there are several areas where I need to account for screen scale within the application's code itself (i.e. I use coordinates from mouseEvent to set the layout of some elements, and these are not snapped), so it probably safe to ignore these issues in the context of the PR.

[kevinrushforth]

So that leaves two follow-on bugs then:

1. Rendering a cached node doesn't match rendering it directly even when the transform is unchanged.
2. TextFlow: methods copied from Region have not picked up subsequent changes in those methods

[kevinrushforth]

I filed the following follow-on bugs:

1. JDK-8258694 : Rendering cached nodes with sub-pixel translation is blurry
2. JDK-8258697 : TextFlow: methods copied from Region have diverged

[fthevenet]

I've added a test that verifies that the inset values for a scrollpane in a scene are snapped to whole pixels, given the actual screen scale.
The test passes without this fix for screen scale of 100% or 200% but fails at 125%, 150% and 175%
With this fix, it passes at all screen scale.

[kevinrushforth]

The test and the fix look good. I left a couple inline comments.

[kevinrushforth]

Looks good.

[arapte]

Apologies for delay on this review, Change looks good but I did observe an issue with layout of TabPane header area when the header is set to be shown on Left or Right side. The last tab-header's border seems to be clipped.
Please check the code below. This is a very specific scenario, If tab's label is changed from "Tab" to "tab" then the issue does not occur. I did not observe any other such scenarios yet.

import javafx.application.Application;
import javafx.scene.Scene;
import javafx.scene.control.TabPane;
import javafx.scene.control.Tab;
import javafx.stage.Stage;
import javafx.geometry.Side;

public class TabPaneExample extends Application {
    public static void main(String[] args) {
        launch(args);
    }

    public void start(Stage primaryStage) {
        Tab tab0 = new Tab("Tab 0");
        Tab tab1 = new Tab("Tab 1");
	Tab tab2 = new Tab("Tab 2");
	TabPane tabPane = new TabPane();
        tabPane.getTabs().addAll(tab0, tab1, tab2);
	tabPane.setSide(Side.LEFT);

        Scene scene = new Scene(tabPane);
        primaryStage.setScene(scene);
        primaryStage.show();
    }
}

|
Output with this fix. | Output without this fix.

Edit: Also, please note the difference of space between letter a and b in Tab's label.

[fthevenet]

Apologies for delay on this review, Change looks good but I did observe an issue with layout of TabPane header area when the header is set to be shown on Left or Right side. The last tab-header's border seems to be clipped.
Please check the code below. This is a very specific scenario, If tab's label is changed from "Tab" to "tab" then the issue does not occur. I did not observe any other such scenarios yet.

import javafx.application.Application;
import javafx.scene.Scene;
import javafx.scene.control.TabPane;
import javafx.scene.control.Tab;
import javafx.stage.Stage;
import javafx.geometry.Side;

public class TabPaneExample extends Application {
    public static void main(String[] args) {
        launch(args);
    }

    public void start(Stage primaryStage) {
        Tab tab0 = new Tab("Tab 0");
        Tab tab1 = new Tab("Tab 1");
	Tab tab2 = new Tab("Tab 2");
	TabPane tabPane = new TabPane();
        tabPane.getTabs().addAll(tab0, tab1, tab2);
	tabPane.setSide(Side.LEFT);

        Scene scene = new Scene(tabPane);
        primaryStage.setScene(scene);
        primaryStage.show();
    }
}

|
Output with this fix. | Output without this fix.

Edit: Also, please note the difference of space between letter a and b in Tab's label.

At which screen scale did you observe this?

[kevinrushforth]

@arapte I couldn't reproduce the problem you are seeing with the border, but I do see the text spacing issue. Did you merge in the latest changes from master into your local branch when you ran your tests? This is the best way to ensure you are running what will eventually be integrated (especially when the source branch is fairly out of date, which I see that it is in this case).

Update: If I run your test program using a build directly from the target branch, I see what you are seeing with the clipped tab pane border. That is likely an unrelated bug that has since been fixed.

The text spacing does seem to be changed as a result of this fix. It will need a closer look to know whether this fix has caused the problem, or exposed a bug elsewhere.

[arapte]

At which screen scale did you observe this?

Both border clip and font spacing issues are observed only at 125% scale.

Update: If I run your test program using a build directly from the target branch, I see what you are seeing with the clipped tab pane border. That is likely an unrelated bug that has since been fixed.

I also see same behavior. Border clip issue does not occur with latest code but the difference in font spacing is observed(only at 125%)

[fthevenet]

I'm suspecting that this fix might be revealing an issue related to JDK-8199592.
Keeping in mind that the revision on which this fix is based does not include the fix for JDK-8199592, a way to confirm that would be to rebase that fix on top of the latest rev and see if the test program still exhibits the same behaviour.
I'll try to do that later on today.

[kevinrushforth]

@fthevenet The truncated line is very likely related to the fix for JDK-8199592. I always merge in the latest master when doing my testing, which is why I initially didn't see that particular problem.

The text spacing is a different problem, since the altered behavior after applying your fix happens with or without the fix for JDK-8199592. It seems likely that it is a separate bug exposed by the fix for this bug rather than caused by it.

[kevinrushforth]

I did some additional testing this morning, and there is definitely an existing problem with positioning labels and text nodes, which is more noticeable when drawing vertical text. The values I see for layoutX and layoutY of the text node (or label) are not correct either before or after this fix. So with this fix, they are just wrong in a slightly different way.

So I don't think this should block the integration of this PR. Rather, this is another follow-up bug that we should file.

[arapte]

Looks good to me.

[openjdk]

@fthevenet This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8211294: ScrollPane content is blurry with 125% scaling

Reviewed-by: kcr, arapte

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 64 new commits pushed to the master branch:

• c197b62: 8242621: TabPane: Memory leak when switching skin
• e74f679: 8254101: Update copyright header for files modified in 2020
• 53fe38b: 8258592: Control labels in Dialogs are truncated at certain DPI scaling levels
• e61b923: 8242361: JavaFX Web View crashes with Segmentation Fault, when HTML contains Data-URIs
• fb8e0cd: 8253356: JavaFX Terminology Refresh
• f2928d9: 8256983: GitHub actions: specify the version of each platform OS and compiler
• ebb59e9: 8201568: zForce touchscreen input device fails when closed and immediately reopened
• 97d655f: 8256012: Fix build of Monocle for Linux
• e1adfa9: 8257758: Allow building of JavaFX native libs for Apple Silicon
• 1a8652a: 8257719: JFXPanel scene fails to render correctly on HiDPI after fix for JDK-8199592
• ... and 54 more: https://git.openjdk.java.net/jfx/compare/d10f948ee7380ac73bc4e2d5bff1caba50fe00a8...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

As you do not have Committer status in this project an existing Committer must agree to sponsor your change. Possible candidates are the reviewers of this PR (@kevinrushforth, @arapte) but any other Committer may sponsor as well.

➡️ To flag this PR as ready for integration with the above commit message, type /integrate in a new comment. (Afterwards, your sponsor types /sponsor in a new comment to perform the integration).

[fthevenet]

/integrate

[openjdk]

@fthevenet
Your change (at version 12db928) is now ready to be sponsored by a Committer.

[kevinrushforth]

/sponsor

[openjdk]

@kevinrushforth @fthevenet Since your change was applied there have been 64 commits pushed to the master branch:

• c197b62: 8242621: TabPane: Memory leak when switching skin
• e74f679: 8254101: Update copyright header for files modified in 2020
• 53fe38b: 8258592: Control labels in Dialogs are truncated at certain DPI scaling levels
• e61b923: 8242361: JavaFX Web View crashes with Segmentation Fault, when HTML contains Data-URIs
• fb8e0cd: 8253356: JavaFX Terminology Refresh
• f2928d9: 8256983: GitHub actions: specify the version of each platform OS and compiler
• ebb59e9: 8201568: zForce touchscreen input device fails when closed and immediately reopened
• 97d655f: 8256012: Fix build of Monocle for Linux
• e1adfa9: 8257758: Allow building of JavaFX native libs for Apple Silicon
• 1a8652a: 8257719: JFXPanel scene fails to render correctly on HiDPI after fix for JDK-8199592
• ... and 54 more: https://git.openjdk.java.net/jfx/compare/d10f948ee7380ac73bc4e2d5bff1caba50fe00a8...master

Your commit was automatically rebased without conflicts.

Pushed as commit 9c84c77.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.