Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LWJGL integration #9

Closed
Spasi opened this issue May 21, 2019 · 13 comments

Comments

@Spasi
Copy link

commented May 21, 2019

Opening this issue to discuss a simple way to integrate DriftFX and LWJGL.

After looking at the current example code, I don't think DriftFX needs to know anything about LWJGL (like #8 suggests). The opposite is a cleaner approach, i.e. the integration work should happen in LWJGL:

  • LWJGL creates new bindings for the DriftFX shared library.
  • The user adds both DriftFX and LWJGL as dependencies to their project.
  • Everything is done in Java, no need for the user to compile native code. The JavaFX setup is done using the DriftFX (Java) API, rendering is done using LWJGL's existing bindings, surface/context management is done using the proposed DriftFX bindings.

This way users could get started much more easily with DriftFX and efxclipse-drift maintainers don't have any extra work to do.

There is just one thing that will simplify the process: It would be nice if DriftFX exposed a C API. LWJGL will then be able to auto-generate the bindings and it would lessen maintenance work going forward. Note that you could still use C++ internally, only the public API would be C. You could also layer the C API on top of the C++ API, assuming there are users that prefer a C++ API.

@redrezo

This comment has been minimized.

Copy link
Member

commented May 27, 2019

i was also planning to use drift in pure java by using lwjgl, but since the first use cases required a native API lwjgl had to wait
i thought i just need a Java API to make the context current, acquire and present textures and use lwjgl for the GL calls
It would be great if the lwjgl side could offer an integration layer
It should be easy to add a C API on top of the existing one
I would appreciate it if you could offer an API proposal that suits your needs since I have almost no experience in designing C APIs

@Renanse

This comment has been minimized.

Copy link

commented Sep 5, 2019

Would really like to see a solution like that proposed by @Spasi

@Spasi

This comment has been minimized.

Copy link
Author

commented Sep 8, 2019

Hey @redrezo,

LWJGL now has DriftFX bindings in the driftfx branch1. See the HelloDriftFX sample and note that there aren't any native methods, the user can integrate JavaFX and OpenGL without writing/building native code. LWJGL exposes a C-like API and internally I was able to use the C++ DriftFX API directly. So this issue can be closed2.

Also, there's no extra dependency. The bindings include the DriftFX source3 (both Java and native) and everything is built together. Afaict your license allows this, but please let me know if that's not the case or if you're simply not OK with it.

Now, the problem is that performance is not good. It's actually worse than using PixelWriter/PixelReader. The demo is a port of glxgears, that normally renders in 0.37ms at 4K resolution. With DriftFX, I'm seeing 2.9ms at the default size (600x600 with 2x HiDPI) and 40ms at 4K (i.e. 25fps instead of ~2700fps). The SimpleColorRenderer in this repository also exhibits the same bad scaling with rendering area size.

I enabled verbose logging and it appears that there's lot of texture creation/recreation going on every frame. Is this expected behavior? Btw, I'm testing on Zulu JDK 8, Windows 10, Ryzen 1800X, GeForce GTX 970 with latest drivers. I also tried the 3 different TransferModes and MainMemory was actually a bit faster than NVDXInterop (colors were wrong though, possibly an RGBA/BGRA mix-up somewhere in the pipeline).

Thanks!


  1. It should build fine on Windows & Linux, haven't tried macOS yet. Build instructions:
    set/export JAVA_HOME=<path to JDK 8 with JavaFX 8>
    vcvarsall amd64 // Windows-only
    ant compile-templates tests // Will take 2-4 minutes, builds everything and runs tests
    ant demo -Dclass=org.lwjgl.demo.driftx.HelloDriftFX
    
  2. Converting DriftFX to a C API would still be a good idea, C++ doesn't offer anything interesting to such a simple API.
  3. Currently copied, will be changed to a git submodule if this effort goes forward.
@tomsontom

This comment has been minimized.

Copy link
Contributor

commented Sep 8, 2019

Great work! We‘ll come back to you on the points you raised and ways to fix them

@redrezo

This comment has been minimized.

Copy link
Member

commented Sep 9, 2019

Thats a great news @Spasi!
Thanks for your effort!

Regarding your performance observations:
At the moment drift creates for every frame a new texture. Its lifecycle is like creation (acquire) -> client rendering -> queuing it to fx (present) -> fx draws
ing -> deletion
We are planning to implement a reuse mechanism for those textures - so we only need to recreate them if a different size is acquired. This should speed up things a lot.

I'll will have a look at your demo once i find the time

@redrezo

This comment has been minimized.

Copy link
Member

commented Sep 10, 2019

@Spasi you can give the wip_performance branch a try - it should be way faster

@Spasi

This comment has been minimized.

Copy link
Author

commented Sep 12, 2019

Hey @redrezo, I tried the wip_performance branch and it's crashing almost immediately when launching the demo:

Stack: [0x000000002b250000,0x000000002b350000],  sp=0x000000002b34e458,  free space=1017k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [nvoglv64.dll+0x94e949]

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j  org.eclipse.fx.drift.internal.GPUSyncUtil.nCreateFence()J+0
j  org.eclipse.fx.drift.internal.GPUSyncUtil.access$000()J+0
j  org.eclipse.fx.drift.internal.GPUSyncUtil$GLSync.<init>()V+5
j  org.eclipse.fx.drift.internal.GPUSyncUtil$GLSync.CreateFence()Lorg/eclipse/fx/drift/internal/GPUSyncUtil$GLSync;+4
j  org.eclipse.fx.drift.internal.QuantumRendererHelper.syncExecuteWithFence(Ljava/util/function/Supplier;)Lorg/eclipse/fx/drift/internal/QuantumRendererHelper$WithFence;+5
j  org.eclipse.fx.drift.impl.NGDriftFXSurface.createTexture(Lcom/sun/prism/Graphics;Lorg/eclipse/fx/drift/internal/Frame;)Lcom/sun/prism/Texture;+155
j  org.eclipse.fx.drift.impl.NGDriftFXSurface.renderFrame(Lcom/sun/prism/Graphics;Lorg/eclipse/fx/drift/internal/Frame;)V+19
j  org.eclipse.fx.drift.impl.NGDriftFXSurface.renderContent(Lcom/sun/prism/Graphics;)V+27
j  com.sun.javafx.sg.prism.NGNode.doRender(Lcom/sun/prism/Graphics;)V+330
j  com.sun.javafx.sg.prism.NGNode.render(Lcom/sun/prism/Graphics;)V+34
j  com.sun.javafx.sg.prism.NGGroup.renderContent(Lcom/sun/prism/Graphics;)V+160
j  com.sun.javafx.sg.prism.NGRegion.renderContent(Lcom/sun/prism/Graphics;)V+111
j  com.sun.javafx.sg.prism.NGNode.doRender(Lcom/sun/prism/Graphics;)V+330
j  com.sun.javafx.sg.prism.NGNode.render(Lcom/sun/prism/Graphics;)V+34
j  com.sun.javafx.tk.quantum.ViewPainter.doPaint(Lcom/sun/prism/Graphics;Lcom/sun/javafx/sg/prism/NodePath;)V+201
j  com.sun.javafx.tk.quantum.ViewPainter.paintImpl(Lcom/sun/prism/Graphics;)V+982
j  com.sun.javafx.tk.quantum.PresentingPainter.run()V+326
j  java.util.concurrent.Executors$RunnableAdapter.call()Ljava/lang/Object;+4
j  java.util.concurrent.FutureTask.runAndReset()Z+47
j  com.sun.javafx.tk.RenderJob.run()V+1
j  java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V+95
j  java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5
j  com.sun.javafx.tk.quantum.QuantumRenderer$PipelineRunnable.run()V+8
j  java.lang.Thread.run()V+11

The crash happens in the QuantumRenderer-0 thread (not in QuantumRenderer-DriftFX-Helper). Afaict, there's no OpenGL context current in that thread when glFenceSync is called.

@redrezo

This comment has been minimized.

Copy link
Member

commented Sep 16, 2019

Sorry @Spasi , i forgot to push the conditional code for windows that prevents the gl fence calls..
The branch should work now for windows as well
also i added a javafx fps counter to the surface itself which can be enabled with -Ddriftfx.showfps=true

@Spasi

This comment has been minimized.

Copy link
Author

commented Sep 17, 2019

Hey @redrezo, the crash is fixed and performance is much better. Will report exact numbers tomorrow, currently testing over a remote desktop solution which skews performance.

The lwjgl3 driftfx branch has been updated if you'd like to test it yourself.

@Spasi

This comment has been minimized.

Copy link
Author

commented Sep 18, 2019

One problem with the current API is that there's no way to synchronize the rendering thread with surface presentation. If no throttling is applied, the rendering loop is able to run at 4000+ fps, with no "back-pressure" happening. This appears to be working fine mostly, but when the window is resized to be larger, something fails to keep up. There's stutterring and blank frames and sometimes rendering stops completely for a second or two.

I'm able to reproduce this both with and without -Djavafx.animation.fullspeed=true. When it works fine, with fullspeed enabled and -Ddriftfx.showfps=true, it's getting up to 700-800fps (I think 1000fps is the hard cap for JavaFX), which is not as good as LWJGL3FX but good enough for many use cases.

With the rendering loop throttled to 60fps, it's all nice and smooth at every window size/resolution (including 4K).

@redrezo

This comment has been minimized.

Copy link
Member

commented Sep 23, 2019

@Spasi
I'm happy to hear that the recent changes improved the performance!
I stumbled upon this back pressure issue while i tried to optimize performance.
I'm working on a swapchain approach. This will also gurantee a constant resource usage compared to pooling. I'm going to support a mailbox and a queue. If your swapchain only holds 2 images in mailbox mode acquire would block until the next image gets available.. this would kind of sync the render thread..
Do you have any other ideas/references how i could provide a sync mechanism to control back pressure?

@tomsontom

This comment has been minimized.

Copy link
Contributor

commented Oct 11, 2019

can we close this issue?

@Spasi

This comment has been minimized.

Copy link
Author

commented Oct 11, 2019

Yes, DriftFX/LWJGL integration is complete.

@Spasi Spasi closed this Oct 11, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.