upgrade to processing 4 #337

shrynx · 2020-10-19T08:07:44Z

here is a start for getting processing 4 working.

Currently default rendering mode works, also OPENGL. with P2D and P3D sometimes there is no output in the window but can save the correctly rendered image.
I'll look into it next week could be a processing issue

getting the gluegen and jogl jars is bit tricky as they are still release candidate
I have attached appropriately packaged fat jars processing-4-jars.zip

shrynx · 2020-10-19T08:46:45Z

src/clj/quil/applet.clj

@@ -66,7 +66,6 @@
  (let [m              (meta applet)
        keep-on-top?   (:keep-on-top m)
        surface        (.getSurface applet)
-        frame          (.frame applet)


frame wasn't being used and has also been removed

nbeloglazov · 2020-10-26T20:39:54Z

project.clj

@@ -1,4 +1,4 @@
-(defproject quil "3.1.1-SNAPSHOT"
+(defproject quil "4.0.0-SNAPSHOT-1"


I created processing4 branch so that master is still on 3 in case we need to fix some issues there. Can you change your pull request to merge into the new branch?

jackrusher · 2021-03-01T08:07:36Z

Hey everybody! Any word on Processing 4 support?

sritchie · 2021-03-03T04:50:02Z

Haha, I came to say the same thing and @jackrusher has beat me to it! We're extremely keen over at https://github.com/sicmutils/sicmutils to use Quil as a base for physics animations, and JDK11+ is a requirement here. I'm happy to help on the effort too.

nbeloglazov · 2021-03-03T05:32:17Z

Damn, I missed that @shrynx updated branch in this PR. Merged now.

sritchie · 2021-03-15T13:10:28Z

Hi @nbeloglazov! do you have any plans to publish that branch to Clojars? I'd love to depend on it in a project and would rather get people on your code than push my own jar.

Thank you!

nbeloglazov · 2021-03-15T17:06:02Z

Ok, I will. I guess I also need to push all the jars that @shrynx listed in the pull request

nbeloglazov · 2021-03-16T04:59:19Z

Deployed Quil 4.0.0-SNAPSHOT together with corresponding supporting libraries. Big thanks to @shrynx for preparing those libs, especially fat jars.

jackrusher · 2021-03-16T07:07:15Z

@nbeloglazov Thanks! I'll give it a test soon. :)

jackrusher · 2021-03-16T12:23:55Z

Sadly, it causes the JVM to dump core when I attempt to create an applet. It looks like the bundled JOGL is having trouble creating an OpenGL context. :/

--------------- S U M M A R Y ------------

Command Line: -Dclojure.basis=.cpcache/4194990013.basis clojure.main -m nrepl.cmdline --middleware ["cider.nrepl/cider-middleware"]

Host: MacBookPro13,3 x86_64 2700 MHz, 8 cores, 16G, Darwin 20.3.0
Time: Tue Mar 16 13:16:37 2021 CET elapsed time: 188 seconds (0d 0h 3m 8s)

--------------- T H R E A D ---------------

Current thread (0x00007fabc1c7d000): JavaThread "nREPL-session-17e07d15-67ec-49a8-a6ed-25d7fb0f4c0d" daemon [_thread_in_native, id=28675, stack(0x0000700003302000,0x0000700003402000)]

Stack: [0x0000700003302000,0x0000700003402000], sp=0x00007000033fdf20, free space=1007k
Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
C [AppKit+0x3af8b4] -[NSOpenGLContext setView:]+0xbd
C [libjogl_desktop.dylib+0x18124] createContext+0x184
C [libjogl_desktop.dylib+0x7f575] Java_jogamp_opengl_macosx_cgl_CGL_createContext0__JJZJZLjava_lang_Object_2I+0x95
j jogamp.opengl.macosx.cgl.CGL.createContext0(JJZJZLjava/lang/Object;I)J+0
j jogamp.opengl.macosx.cgl.CGL.createContext(JJZJZLjava/nio/IntBuffer;)J+33
j jogamp.opengl.macosx.cgl.MacOSXCGLContext$NSOpenGLImpl.create(JIII)J+836
j jogamp.opengl.macosx.cgl.MacOSXCGLContext.createContextARBImpl(JZIII)J+91
j jogamp.opengl.GLContextImpl.createContextARBVersions(JZIIIII[I[I)J+169
j jogamp.opengl.GLContextImpl.createContextARBMapVersionsAvailable(Lcom/jogamp/nativewindow/AbstractGraphicsDevice;IIZ)Z+204
j jogamp.opengl.GLContextImpl.mapGLVersions(Lcom/jogamp/nativewindow/AbstractGraphicsDevice;)Z+399
j jogamp.opengl.GLContextImpl.createContextARB(JZ)J+116
j jogamp.opengl.macosx.cgl.MacOSXCGLContext.createImpl(J)Z+287
j jogamp.opengl.GLContextImpl.makeCurrentWithinLock(I)I+224
j jogamp.opengl.GLContextImpl.makeCurrent(Z)I+488
j jogamp.opengl.GLContextImpl.makeCurrent()I+2
j jogamp.opengl.macosx.cgl.MacOSXCGLDrawableFactory.getOrCreateSharedResourceImpl(Lcom/jogamp/nativewindow/AbstractGraphicsDevice;)Ljogamp/opengl/macosx/cgl/MacOSXCGLDrawableFactory$SharedResource;+227
j jogamp.opengl.macosx.cgl.MacOSXCGLDrawableFactory.getOrCreateSharedResourceImpl(Lcom/jogamp/nativewindow/AbstractGraphicsDevice;)Ljogamp/opengl/SharedResourceRunner$Resource;+2
j jogamp.opengl.GLDrawableFactoryImpl.getOrCreateSharedResource(Lcom/jogamp/nativewindow/AbstractGraphicsDevice;)Ljogamp/opengl/SharedResourceRunner$Resource;+13
j jogamp.opengl.GLDrawableFactoryImpl.createSharedResourceImpl(Lcom/jogamp/nativewindow/AbstractGraphicsDevice;)Z+2
j com.jogamp.opengl.GLDrawableFactory.createSharedResource(Lcom/jogamp/nativewindow/AbstractGraphicsDevice;)Z+2
j com.jogamp.opengl.GLProfile.initProfilesForDeviceCritical(Lcom/jogamp/nativewindow/AbstractGraphicsDevice;)Z+231
j com.jogamp.opengl.GLProfile.initProfilesForDevice(Lcom/jogamp/nativewindow/AbstractGraphicsDevice;)Z+30
j com.jogamp.opengl.GLProfile.initProfilesForDefaultDevices()V+621
j com.jogamp.opengl.GLProfile.access$000()V+0
j com.jogamp.opengl.GLProfile$1.run()Ljava/lang/Object;+59
v ~StubRoutines::call_stub
V [libjvm.dylib+0x39a2c8] _ZN9JavaCalls11call_helperEP9JavaValueRK12methodHandleP17JavaCallArgumentsP6Thread+0x21a
V [libjvm.dylib+0x418d08] JVM_DoPrivileged+0x6e5
J 2103 java.security.AccessController.doPrivileged(Ljava/security/PrivilegedAction;)Ljava/lang/Object; java.base@11.0.8 (0 bytes) @ 0x0000000124c6f712 [0x0000000124c6f640+0x00000000000000d2]
j com.jogamp.opengl.GLProfile.initSingleton()V+78
j com.jogamp.opengl.GLProfile.getProfileMap(Lcom/jogamp/nativewindow/AbstractGraphicsDevice;Z)Ljava/util/HashMap;+0
j com.jogamp.opengl.GLProfile.get(Lcom/jogamp/nativewindow/AbstractGraphicsDevice;Ljava/lang/String;)Lcom/jogamp/opengl/GLProfile;+16
j com.jogamp.opengl.GLProfile.getGL2ES2()Lcom/jogamp/opengl/GLProfile;+5
j processing.opengl.PSurfaceJOGL.initGL()V+40
j processing.opengl.PSurfaceJOGL.initFrame(Lprocessing/core/PApplet;)V+14
j processing.core.PApplet.initSurface()Lprocessing/core/PSurface;+34
j quil.Applet.initSurface()Lprocessing/core/PSurface;+39
j processing.core.PApplet.runSketch([Ljava/lang/String;Lprocessing/core/PApplet;)V+776
j quil.applet$applet_run.invokeStatic(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;+522
j quil.applet$applet_run.invoke(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;+9
j quil.applet$applet.invokeStatic(Lclojure/lang/ISeq;)Ljava/lang/Object;+1435
j quil.applet$applet.doInvoke(Ljava/lang/Object;)Ljava/lang/Object;+6

nbeloglazov · 2021-03-18T17:01:30Z

That's not a very readable/google-able error :(
You used p2d or p3d renderer?

jackrusher · 2021-03-25T14:33:13Z

Argh, sorry, I should have included that in the report: p2d!

jackrusher · 2021-05-17T14:12:41Z

Also dumps core with P3D.

lspector · 2021-05-30T22:17:05Z

I'm getting:

Exception in thread "main" java.lang.RuntimeException: No such var: cstr/includes?, compiling:(quil/helpers/docs.clj:30:11)

when I try to run an old project of mine with the quil dependency changed to [quil "4.0.0-SNAPSHOT-1"].

lein deps seems to pull in the new library fine, but then when I run it (lein run pucks.worlds.dev.world1) I get that error which seems to be coming from within Quil.

lspector · 2021-06-18T20:41:48Z

Introducing some students to Quil and jumping through hoops to install and get our systems to use old Java versions. Seems like this was close to being resolved... I guess I'm just hoping to bump it back on to the to-do list of anyone who knows how to fix it. Thanks for anything anyone can do!

nbeloglazov · 2021-06-21T02:40:34Z

@lspector regarding your issue with cstr/includes? - includes? was added in clojure 1.8 and your project uses 1.7. I updated it to use 1.10.3 locally and it worked. Also I deployed version quil 4.0.0-SNAPSHOT which uses latest processing 4.0.0-alpha-4.

For the opengl issue. I tried running it on linux and it wasn't even able to load gluegen library. I spent couple of hours trying to get to the root of this but couldn't even find current version of gluegen (2.4.0) we are using, its source code. The latest widely available is 2.3.2. Pretty frustrating.

lspector · 2021-06-21T02:54:46Z

Beautiful! Thank you so much @nbeloglazov!

kvnsmth · 2021-07-10T16:56:19Z

Thanks for working on supporting processing 4! This snapshot version works fine so far in my testing with the java2d renderer. Both p2d and p3d complain about not finding natives/linux-amd64//libgluegen_rt.so on startup, but maybe this is what you are running into @nbeloglazov? I'm happy to help debug but unfortunately I'm not that familiar with the Clojure dev ecosystem.

nbeloglazov · 2021-07-12T00:28:42Z

@kvnsmth yes, that what I was running into. The issue is the following. Processing depends on gluegen/jogl libraries which are part of JogAmp project. These libraries depend on some native code for each platform there is a separate native library that is loaded by gluegen/jogl. For example if you are on linux 64 - it tries to load linux-amd64/libgluegen_rt.so library. There are multiple ways to package gluegen/jogl libraries. What we've been using is creating a fat jar: single gluegen JAR file that contains all native libraries inside in such a way when gluegen runs - it's able to find and load native libraries from that JAR file. The benefits of that approach is that we can upload fat gluegen jar to Maven/Clojars and it's just works.

But that no longer works with Processing 4 as it uses newer gluegen/jogl, 2.4.0. And for some reason it can no longer load native libraries from the fat jar. I can't figure out why. I would try to debug it (I was able to reproduce it locally by creating simple java file that initializes jogl library, without having any clojure) but the problem that I can't find source code of gluegen/jogl 2.4.0 so debugging is not working (I don't have good enough skills to debug compiled java code).

So the problem not with clojure/lein/clojars, but with gluegen/jogl itself. If you have experience with java - you can give it a shot, I can provide setup that I used locally to reproduce it.

kvnsmth · 2021-07-13T14:29:51Z

@nbeloglazov thanks for that explanation! That helps me understand more of the issue. I have not done Java development for a very long time but, time permitting, I can help investigate if you feel like writing up what you're doing.

I was able to track down a few things to help my own understanding of how all this fits together and perhaps it will help you since you said you couldn't find the source.

Discussion of 2.4 Release, it looks to be very slow going and is still currently a release candidate.
JogAmp Git repositories: https://jogamp.org/cgit/ (for source)
JogAmp CI: https://jogamp.org/chuck/ (for correlation of builds to source)
It appears that the Processing team has considered/is considering moving from JOGL to LWJGL. See Renderers section on their wiki. I don't know enough to know the significance but maybe this would just magically solve things. 😄

blueberry · 2021-08-24T10:05:51Z

#333 (comment)

pablinos · 2022-04-15T07:17:55Z

but the problem that I can't find source code of gluegen/jogl 2.4.0 so debugging is not working

Yeah that's not easy to find. I was thinking there might be 2.4.0 branch, but there is an rc branch that looks like it could be it. However, that hasn't had a commit since 2018.

Looking at the build logs for the latest RC, it seems to reference the latest master branch commits, but they all seem to be in 2020 and not 2021, as referred to in the build name. Is it worth trying to debug against that to see if we can find the problem?

I was able to reproduce it locally by creating simple java file that initializes jogl library, without having any clojure

Have you got this test case anywhere? Someone might be able to use it to move it along. I'll have a go, but I suspect I won't get very far!

jackrusher · 2022-12-26T09:46:33Z

It looks like Processing 4.x has gone general release. Should we try a build relying on their latest?

sritchie · 2022-12-26T12:30:09Z

Alpha8 worked well on my M1 with JDK17 for a processing-only application via the Java SDK. It would be awesome to get quil up to speed!

dkyy · 2023-08-30T17:06:59Z

Hello. I’m new to everything here (lein, clojure, quil) but have been having trouble getting things running, and think they relate to the discussions here. Basically - the default “lein new quil” doesn’t generate anything that works. If I make some changes then “lein run” works, but “lein repl” does not. I’m running on a MacOS MacBook Pro M1 Max. Here’s a summary of the steps I’ve been taking…

WIth the default lein quil project (created with %lein new quil hello-4) when I “lein run” I get this error
No :main namespace specified in project.clj.

So I added this line to project.cli
:main "hello-4.core"

Now when I “lein run” I get:
Execution error (ClassNotFoundException) at jdk.internal.loader.BuiltinClassLoader/loadClass (BuiltinClassLoader.java:641). com.apple.eawt.QuitHandler

If I change the quil dependency in project.cli from “3.1.0” to "4.0.0-SNAPSHOT” then… the java window opens, things animate, briefly. I also also get this error:

Syntax error compiling at (/private/var/folders/cn/ws27369s34z4503m3qxqbr780000gn/T/form-init3206095596361001001.clj:1:125).
Cannot find anything to run for: hello-4.core

If I add this to core.clj
(defn -main [& args])
and to a “lein run” then perfection — everything works as expected.

But …if I “lein repl” then I get this:

Exception in thread "Thread-1" java.lang.ClassCastException: class java.lang.String cannot be cast to class clojure.lang.Named (java.lang.String is in module java.base of loader 'bootstrap'; clojure.lang.Named is in unnamed module of loader 'bootstrap')
	at clojure.core$namespace.invokeStatic(core.clj:1612)
	at clojure.core$namespace.invoke(core.clj:1612)
	at leiningen.repl$init_ns.invokeStatic(repl.clj:171)
	at leiningen.repl$init_ns.invoke(repl.clj:170)
	at leiningen.repl$wrap_init_ns.invokeStatic(repl.clj:176)
	at leiningen.repl$wrap_init_ns.invoke(repl.clj:175)
	at leiningen.repl$handler_for.invokeStatic(repl.clj:201)
	at leiningen.repl$handler_for.invoke(repl.clj:197)
	at leiningen.repl$server_forms.invokeStatic(repl.clj:245)
	at leiningen.repl$server_forms.invoke(repl.clj:234)
	at leiningen.repl$server$fn__11554.invoke(repl.clj:330)
	at clojure.lang.AFn.applyToHelper(AFn.java:152)
	at clojure.lang.AFn.applyTo(AFn.java:144)
	at clojure.core$apply.invokeStatic(core.clj:667)
	at clojure.core$with_bindings_STAR_.invokeStatic(core.clj:1990)
	at clojure.core$with_bindings_STAR_.doInvoke(core.clj:1990)
	at clojure.lang.RestFn.invoke(RestFn.java:425)
	at clojure.lang.AFn.applyToHelper(AFn.java:156)
	at clojure.lang.RestFn.applyTo(RestFn.java:132)
	at clojure.core$apply.invokeStatic(core.clj:671)
	at clojure.core$bound_fn_STAR_$fn__5818.doInvoke(core.clj:2020)
	at clojure.lang.RestFn.invoke(RestFn.java:397)
	at clojure.lang.AFn.run(AFn.java:22)
	at java.base/java.lang.Thread.run(Thread.java:833)

and things hang.

I’ve been doing this from the terminal because when I try it from Emacs with Cider, or Sublime with Clojure-Sublimed I get other problems. And figure it’s best to start from the basics.

I read here https://www.reddit.com/r/Clojure/comments/10ho3om/is_quil_moving_forward/ that a fix is in the works. But not sure if that relates to my issues…

Thanks in advance,

david

jackrusher · 2023-08-30T18:05:56Z

We've got a build that works, including on M1/M2 hardware, but we haven't pushed it to Clojars yet. I'll update this ticket when it's available there for testing :)

dgtized · 2023-11-27T18:05:35Z

Quil v4.3.1323 has been released to Clojars. It tracks the Processing 4.3 and p5js 1.7.0 releases for aarch64, amd64, and MacOS Universal (intel/M1) architectures on Java 17+.

get it working

4dd11b5

shrynx mentioned this pull request Oct 19, 2020

Plans to support Processing 4 / Java 11+ #333

Closed

shrynx commented Oct 19, 2020

View reviewed changes

nbeloglazov reviewed Oct 26, 2020

View reviewed changes

shrynx changed the base branch from master to processing4 October 26, 2020 21:39

nbeloglazov merged commit 4244bf0 into quil:processing4 Mar 3, 2021

shrynx deleted the shrynx/processsing-v4 branch March 3, 2021 17:58

nbeloglazov mentioned this pull request Jul 19, 2021

Ensure compatibility with java 9 #228

Closed

This was referenced Jan 3, 2022

liblibgluegen_rt.so.so: cannot open shared object file: No such file or directory #357

Closed

Quil not working on Java 9 on Mac OS #225

Closed

lspector mentioned this pull request Apr 16, 2023

Merge processing4 and docs? #378

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

upgrade to processing 4 #337

upgrade to processing 4 #337

shrynx commented Oct 19, 2020 •

edited

Loading

shrynx Oct 19, 2020

nbeloglazov Oct 26, 2020

jackrusher commented Mar 1, 2021

sritchie commented Mar 3, 2021

nbeloglazov commented Mar 3, 2021

sritchie commented Mar 15, 2021

nbeloglazov commented Mar 15, 2021

nbeloglazov commented Mar 16, 2021

jackrusher commented Mar 16, 2021

jackrusher commented Mar 16, 2021

nbeloglazov commented Mar 18, 2021

jackrusher commented Mar 25, 2021

jackrusher commented May 17, 2021

lspector commented May 30, 2021

lspector commented Jun 18, 2021

nbeloglazov commented Jun 21, 2021

lspector commented Jun 21, 2021

kvnsmth commented Jul 10, 2021

nbeloglazov commented Jul 12, 2021

kvnsmth commented Jul 13, 2021

blueberry commented Aug 24, 2021

pablinos commented Apr 15, 2022

jackrusher commented Dec 26, 2022

sritchie commented Dec 26, 2022

dkyy commented Aug 30, 2023

jackrusher commented Aug 30, 2023

dgtized commented Nov 27, 2023

		@@ -1,4 +1,4 @@
		(defproject quil "3.1.1-SNAPSHOT"
		(defproject quil "4.0.0-SNAPSHOT-1"

upgrade to processing 4 #337

upgrade to processing 4 #337

Conversation

shrynx commented Oct 19, 2020 • edited Loading

shrynx Oct 19, 2020

Choose a reason for hiding this comment

nbeloglazov Oct 26, 2020

Choose a reason for hiding this comment

jackrusher commented Mar 1, 2021

sritchie commented Mar 3, 2021

nbeloglazov commented Mar 3, 2021

sritchie commented Mar 15, 2021

nbeloglazov commented Mar 15, 2021

nbeloglazov commented Mar 16, 2021

jackrusher commented Mar 16, 2021

jackrusher commented Mar 16, 2021

nbeloglazov commented Mar 18, 2021

jackrusher commented Mar 25, 2021

jackrusher commented May 17, 2021

lspector commented May 30, 2021

lspector commented Jun 18, 2021

nbeloglazov commented Jun 21, 2021

lspector commented Jun 21, 2021

kvnsmth commented Jul 10, 2021

nbeloglazov commented Jul 12, 2021

kvnsmth commented Jul 13, 2021

blueberry commented Aug 24, 2021

pablinos commented Apr 15, 2022

jackrusher commented Dec 26, 2022

sritchie commented Dec 26, 2022

dkyy commented Aug 30, 2023

jackrusher commented Aug 30, 2023

dgtized commented Nov 27, 2023

shrynx commented Oct 19, 2020 •

edited

Loading