Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8239589: JavaFX UI will not repaint after reconnecting via Remote Desktop #315

Closed
wants to merge 4 commits into from

Conversation

Schmidor
Copy link
Contributor

@Schmidor Schmidor commented Oct 8, 2020

When connecting via Remote Desktop to a Windows 10 machine ans starting a JavaFX application, the D3D pipeline is used successfully.
After closing the the Remote Desktop session and reconnecting, the D3D requests fail with an D3DERR_DEVICEREMOVED HResult, and the application contents are not rendered anymore.
Reinitializing the whole pipeline fixes that.


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

  • JDK-8239589: JavaFX UI will not repaint after reconnecting via Remote Desktop

Download

$ git fetch https://git.openjdk.java.net/jfx pull/315/head:pull/315
$ git checkout pull/315

@bridgekeeper
Copy link

@bridgekeeper bridgekeeper bot commented Oct 8, 2020

👋 Welcome back Schmidor! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk openjdk bot added the rfr label Oct 8, 2020
@mlbridge
Copy link

@mlbridge mlbridge bot commented Oct 8, 2020

Webrevs

@kevinrushforth
Copy link
Member

@kevinrushforth kevinrushforth commented Oct 8, 2020

/reviewers 2

@openjdk
Copy link

@openjdk openjdk bot commented Oct 8, 2020

@kevinrushforth
The number of required reviews for this PR is now set to 2 (with at least 1 of role reviewers).

@kevinrushforth
Copy link
Member

@kevinrushforth kevinrushforth commented Oct 8, 2020

This will need a fair bit of testing. I'm not certain that we always want to recreate the factory when the device is lost, but we can evaluate that during the review.

@Schmidor
Copy link
Contributor Author

@Schmidor Schmidor commented Oct 8, 2020

The full recreation of the pipeline is only done when D3DERR_DEVICEREMOVED is occurring. On other causes of why the device is not ready, only the factory reference in the painter is cleared.

@kevinrushforth
Copy link
Member

@kevinrushforth kevinrushforth commented Oct 9, 2020

The full recreation of the pipeline is only done when D3DERR_DEVICEREMOVED is occurring

I see that now. That part is fine then.

Another concern is that this would be the first time we have recreated a GraphicsPipeline -- they are effectively singletons today. This could have unexpected side effects and would need to be very carefully reviewed and tested.

Have you considered an alternative where you retain the GraphicsPipeline, but recreate the native device?

@Schmidor
Copy link
Contributor Author

@Schmidor Schmidor commented Oct 13, 2020

I hope I understood your intend. I've tried to replace the recreation of the GraphicsPipeline-Instance by only calling the native parts. Unfortunately that doesn't seem to work at all. Probably I'm missing some updates.

    public boolean reinit() {
        if (d3dEnabled) {
            if (creator != Thread.currentThread()) {
                throw new IllegalStateException(
                        "This operation is not permitted on the current thread ["
                                + Thread.currentThread().getName() + "]");
            }
            notifyAllResourcesReleased();
            nDispose();
            for (int i = 0; i != factories.length; ++i) {
                factories[i] = null;
            }
            nInit(PrismSettings.class);
            factories = new D3DResourceFactory[nGetAdapterCount()];
        }
        return d3dEnabled;
    }

@Schmidor Schmidor changed the title 8239589: JavaFX UI will not repaint after reconnecting via Remote Desktop [WIP] 8239589: JavaFX UI will not repaint after reconnecting via Remote Desktop Oct 13, 2020
@Schmidor
Copy link
Contributor Author

@Schmidor Schmidor commented Oct 13, 2020

On a side note, not all resources seem to be freed / recreated on pipeline disposal. There are some error outputs, and some parts in a webview are only rendered after interactions. So it is probably better to mark this as WIP. I would be happy if you have some advice for me.

java.lang.IllegalStateException: unmanaged resource freed from pool D3D Vram Pool
	at com.sun.prism.impl.BaseResourcePool.resourceFreed(BaseResourcePool.java:463)
	at com.sun.prism.impl.ManagedResource.dispose(ManagedResource.java:127)
	at com.sun.prism.impl.BaseTexture.dispose(BaseTexture.java:297)
	at com.sun.scenario.effect.impl.prism.ps.PPSDrawable.flush(PPSDrawable.java:69)
	at com.sun.scenario.effect.impl.ImagePool.dispose(ImagePool.java:267)
	at com.sun.scenario.effect.impl.Renderer.getRenderer(Renderer.java:367)
	at com.sun.scenario.effect.Effect.getCompatibleImage(Effect.java:479)
	at com.sun.javafx.webkit.prism.WCGraphicsPrismContext$Layer.<init>(WCGraphicsPrismContext.java:1369)
	at com.sun.javafx.webkit.prism.WCGraphicsPrismContext$ClipLayer.<init>(WCGraphicsPrismContext.java:1440)
	at com.sun.javafx.webkit.prism.WCGraphicsPrismContext.setClip(WCGraphicsPrismContext.java:328)
	at com.sun.webkit.graphics.GraphicsDecoder.decode(GraphicsDecoder.java:225)
	at com.sun.webkit.graphics.WCRenderQueue.decode(WCRenderQueue.java:92)
	at com.sun.webkit.WebPage.paint2GC(WebPage.java:736)
	at com.sun.webkit.WebPage.paint(WebPage.java:703)
	at com.sun.javafx.sg.prism.web.NGWebView.renderContent(NGWebView.java:95)
	at com.sun.javafx.sg.prism.NGNode.doRender(NGNode.java:2072)
	at com.sun.javafx.sg.prism.NGNode.render(NGNode.java:1964)
	at com.sun.javafx.sg.prism.NGGroup.renderContent(NGGroup.java:270)
	at com.sun.javafx.sg.prism.NGRegion.renderContent(NGRegion.java:579)
	at com.sun.javafx.sg.prism.NGNode.doRender(NGNode.java:2072)
	at com.sun.javafx.sg.prism.NGNode.render(NGNode.java:1964)
	at com.sun.javafx.tk.quantum.ViewPainter.doPaint(ViewPainter.java:479)
	at com.sun.javafx.tk.quantum.ViewPainter.paintImpl(ViewPainter.java:328)
	at com.sun.javafx.tk.quantum.UploadingPainter.run(UploadingPainter.java:143)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
	at com.sun.javafx.tk.RenderJob.run(RenderJob.java:58)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
	at com.sun.javafx.tk.quantum.QuantumRenderer$PipelineRunnable.run(QuantumRenderer.java:125)
	at java.base/java.lang.Thread.run(Thread.java:832)

@openjdk openjdk bot removed the rfr label Oct 13, 2020
@Schmidor Schmidor changed the title [WIP] 8239589: JavaFX UI will not repaint after reconnecting via Remote Desktop 8239589: JavaFX UI will not repaint after reconnecting via Remote Desktop Oct 27, 2020
@openjdk openjdk bot added the rfr label Oct 27, 2020
@chewett
Copy link

@chewett chewett commented Nov 26, 2020

Now more of my company are remoting into machines this issue is causing problems for us.

@kevinrushforth kevinrushforth marked this pull request as draft Jan 11, 2021
@openjdk openjdk bot removed the rfr label Jan 11, 2021
@kevinrushforth kevinrushforth marked this pull request as ready for review Jan 11, 2021
@openjdk openjdk bot added the rfr label Jan 11, 2021
@kevinrushforth
Copy link
Member

@kevinrushforth kevinrushforth commented Jan 11, 2021

I moved it to Draft before noticing that there were additional commits after your last comment that I had missed seeing earlier, so I moved it back to RFR.

@kevinrushforth
Copy link
Member

@kevinrushforth kevinrushforth commented Feb 11, 2021

This is a good starting point, but it will need additional work, possibly in the native D3D code as well as on the Java side, to fully cleanup and recreate all of the resources after the device is recreated. As discussed offline, I'll take a stab at this using your PR as a starting point.

I ran some tests this afternoon. It does detect that the devide was removed, and the disposes and recreates the device, but then it has problem drawing anything with a texture; it throws an exception and there are rendering artifacts:

KCR: create resource factor for screen 0
D3DContext::testCooperativeLevel
D3DContext::testCooperativeLevel
D3DContext::testCooperativeLevel
D3DContext::testCooperativeLevel
D3DContext::testCooperativeLevel
KCR: D3DERR_DEVICEREMOVED
KCR: dispose graphics pipeline
KCR: instance is null: reinitialize D3DPipeline
Exception in thread "JavaFX Application Thread" java.lang.NullPointerException: Cannot invoke "com.sun.prism.GraphicsPipeline.is3DSupported()" because the return value of "com.sun.prism.GraphicsPipeline.getPipeline()" is null
	at javafx.graphics/com.sun.javafx.tk.quantum.QuantumToolkit.isSupported(QuantumToolkit.java:1209)
	at javafx.graphics/com.sun.javafx.application.PlatformImpl.isSupportedImpl(PlatformImpl.java:979)
	at javafx.graphics/com.sun.javafx.application.PlatformImpl.isSupported(PlatformImpl.java:646)
	at javafx.graphics/javafx.application.Platform.isSupported(Platform.java:262)
	at javafx.graphics/com.sun.javafx.scene.input.PickResultChooser.processOffer(PickResultChooser.java:182)
	at javafx.graphics/com.sun.javafx.scene.input.PickResultChooser.offer(PickResultChooser.java:143)
	at javafx.graphics/javafx.scene.Node.intersects(Node.java:5229)
	at javafx.graphics/javafx.scene.Node$1.intersects(Node.java:553)
	at javafx.graphics/com.sun.javafx.scene.NodeHelper.intersects(NodeHelper.java:258)
	at javafx.graphics/javafx.scene.layout.Region.doPickNodeLocal(Region.java:3227)
        ...
	at javafx.graphics/javafx.scene.Scene.pick(Scene.java:2031)
	at javafx.graphics/javafx.scene.Scene$MouseHandler.process(Scene.java:3810)
	at javafx.graphics/javafx.scene.Scene.processMouseEvent(Scene.java:1851)
	at javafx.graphics/javafx.scene.Scene$ScenePeerListener.mouseEvent(Scene.java:2584)
	at javafx.graphics/com.sun.javafx.tk.quantum.GlassViewEventHandler$MouseEventNotification.run(GlassViewEventHandler.java:409)
	at javafx.graphics/com.sun.javafx.tk.quantum.GlassViewEventHandler$MouseEventNotification.run(GlassViewEventHandler.java:299)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:391)
	at javafx.graphics/com.sun.javafx.tk.quantum.GlassViewEventHandler.lambda$handleMouseEvent$2(GlassViewEventHandler.java:447)
	at javafx.graphics/com.sun.javafx.tk.quantum.QuantumToolkit.runWithoutRenderLock(QuantumToolkit.java:413)
	at javafx.graphics/com.sun.javafx.tk.quantum.GlassViewEventHandler.handleMouseEvent(GlassViewEventHandler.java:446)
	at javafx.graphics/com.sun.glass.ui.View.handleMouseEvent(View.java:556)
	at javafx.graphics/com.sun.glass.ui.View.notifyMouse(View.java:942)
	at javafx.graphics/com.sun.glass.ui.win.WinApplication._runLoop(Native Method)
	at javafx.graphics/com.sun.glass.ui.win.WinApplication.lambda$runLoop$3(WinApplication.java:174)
	at java.base/java.lang.Thread.run(Thread.java:832)
...
KCR: create resource factor for screen 0
D3DContext::testCooperativeLevel
D3DContext::testCooperativeLevel
KCR: dispose graphics pipeline

HelloFontSize

@kevinrushforth
Copy link
Member

@kevinrushforth kevinrushforth commented Feb 16, 2021

I created PR #403 using this as a starting point, so I am moving this PR back to draft (and it probably can be closed).

@kevinrushforth kevinrushforth marked this pull request as draft Feb 16, 2021
@kevinrushforth
Copy link
Member

@kevinrushforth kevinrushforth commented Mar 17, 2021

I just created the final PR #430 to fix this bug. As a result, I'm closing this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
3 participants