OpenJDK 11 / OpenJFX support #5753
Conversation
… and later have a new java.awt.desktop package that replaces the custom handlers in the com.apple.eawt package. This commit uses java.lang.reflect.Proxy to generate handlers of the correct type.
Focusing first on getting "hello world" applications running under Java 11, this initial migration works for Mac OS X (10.14.1). There are a number of issues and warnings still to address so the overall PR remains WIP. Specific modifications to address processing#5750: - Modification in build system to address a unified JDK / JRE. - Modification in build system and application code to deal with version naming convention migration. - Modification of OS X specific system paths changed by Java 11. - Migration to OpenJDK (processing#4415) via AdoptOpenJDK (selected due to long term build support and some vendor independence). - Migration to OpenJFX (processing#5286). - Removal of deprecated `java.ext.dirs` (replacement funcationality is still being explored). - Update to the appbundler fork. - Update to the JDT jars. Documentation is still incomplete as is reaching feature parity on mac as well as testing on other OS'. Please don't merge yet.
|
Currently only tested on Mac. Documentation is still incomplete. Please don't merge yet. |
|
Note that this required use of #5747. |
|
Great; also helpful that you're testing on a Mac, since this is where we're likely to see the most significant gaps/problems in compatibility. |
|
@benfry - Totally! I am doing some work in Linux right now. General community - I just mentioned this in #5750 but I don't actually have access to windows currently (maybe I'll buy a copy if I am doing a bit of work here...). That said, is there anyone readily available with windows floating around willing to do a bit of a deeper dive into the JDK? |
|
For anyone interested in fiddling with Windows but on another platform, Microsoft makes available virtual machine images for use in development/evaluation with VMware, Parallels, Virtual Box… |
|
Oh that's nifty. I did not know Microsoft did that. Thanks! |
|
Yeah, I've paid out of pocket for several Windows licenses over the years for testing Processing, but nice that we have other options nowadays. |
As part of processing#5750, added support for AdoptOpenJDK (also required for - OracleDownloadUrlGenerator - AdoptOpenJdkDownloadUrlGenerator which support Downloader. Tests and documentation included (required refactor of build.xml).
|
(Seeing the 32-bit warnings/workarounds…) If we make the move to Java 11 we also might want to drop 32-bit support. If you run into things where 32-bit support is making a mess (or not), just keep them in mind to help us make that decision. |
As part of processing#5753 and processing#5750, working AdoptOpenJDK released java on Elementary Linux 5.0 (Juno, 64bit). Linux 4.15.0-36-generic. Tested with Java 11. JDK downloads working and PDE runs but sketches wont execute due to broken java path.
As part of processing#5750 / processing#5753, example sketch and PDE running in Linux with Elementary 5.0 (Juno, 64bit). Linux 4.15.0-36-generic. Specifically, resolved issue with java path on linux given new JDK structure.
|
Thanks @benfry! I appreciate it. I'll see how Windows goes but, at the least, I would recommend dropping Linux 32-bit support given what build are available from AdoptOpenJDK. |
|
Quick status update... I got PDE and an example sketch running under Linux (Elementary 5.0). I'll try to catalog / tackle a few issues that I noticed there but I want move to Windows next just to see if there is anything major. |
As part of processing#5750 and processing#5753, have the development environment and example sketch working in Windows 64 bit with some display issues in the IDE in particular. This is using AdoptOpenJDK 11 with Windows 10.
|
The IDE and simple example sketches are working in Linux64, Mac, and Windows64. There are some display issues that I noticed in the IDE and there's some warnings to address before some code clean up. I also have not been solving I am going to take another pass at Mac, Linux, and Windows each to harden some of this. I think some of the UI issues I have seen so far look similar to what I have seen in the past when taking other projects from proprietary JDK to OpenJDK pre-Java 11. Mostly sizing is strange. Anyway... Question for the community... I know there's some automated test in Processing but is there a list of things to try out after a bit of a larger change like this? I was going to check the examples and then (if they are not covered in the examples) the different rendering modes, fullscreen, and export. Anything else of particular interest? |
As part of processing#5753 / processing#5750, remove deprecated com.apple.eawt calls made in the core module. Specifically, respond to (JEP 272)[http://openjdk.java.net/jeps/272] which is generating warnings on sketch run and may cause failure in future JDKs.
Sync up to official master
Sync to master processing while working on processing#5753 / processing#5750
|
What's the |
|
Those lambdas with the #5747 will work when classes are compiled to run on a different JDK with different Lambdas aren't true closures here. They're compiled to specific instance of a class. |
|
Hey friends I am so sorry for the delay. @ssilverman - Thank you for your comments and insight. I was going to send a note on Friday asking your opinion actually on this but I ran out of time before I could get that out. It seems like JEP 272 and JEP 260 have left us in a bit of an inconvenient spot with regards to the runtime warnings re illegal reflective access and plans of making @benfry - Thanks for writing back and for that helpful context. With that background in mind, at one point I was hoping to allow for compilation of Processing under either 8 or 11, adding I am sure the folks here already have it but, just for posterity's sake, this is covered in JEP 220 with the JVM refusing to start when |
|
If we make the move to Java 11, we'd drop Java 8 immediately and not worry about backwards compatibility for compiling or running. Most likely, we'd be calling it Processing 4 because of the number of things that would be changing underneath (JVM, app launcher, how it interacts with the OS, etc). And people who wanted to use Java 8 would have a clear choice to use 3.x. FWIW, the more I'm thinking about it, I'm fairly certain we'd drop 32-bit support altogether. (The one exception would be if @gohai has a problem here with ARM, though those boards seem to be headed all 64-bit too). And if we can't get OpenJDK 11 to behave as well as JDK 8, we might just leave it as a branch/fork for people who are intent on using Java 11 (and who can incorporate changes from the main branch as needed). |
|
Hey there! A quick favor to the community, if you are on linux and you are testing these changes out... can you drop a note here with the distro and the rendering engine (P2D, FX2D, etc)? Just trying to feel out where we have folks trying this and where there might be gaps. Thanks, |
|
Hi @sampottinger ! What about sharing a list of names of Examples so we call all test the same and post our OS? |
|
When will this be migrated in the main processing branch? |
|
Sorry dropped the ball on the examples @hamoid. 😅 I’ve been using something on my end but would probably want to clean it up just a smidge. I’ll try to do that. |
|
Hey @clankill3r! I have been in some discussion with Ben but he’s been busy. There’s some discussion of this becoming a beta either in the 3 train or starting the 4 but there’s more he wants to tackle (close out more issues currently waiting for work in the processing issue tracker). I’m not sure if that would happen before or after this merges in. Sorry I don’t have much more concrete info than that. That said, in the meantime, I am in the process of investigating the JOGL 2.4 / LWJGL situation and looking at the language server. Some of that would be a pretty big lift though... Still sorting through what I’ll have time to implement. |
|
@sampottinger reminds me I think I forgot to reply to you re. LWJGL? Use the email on my GitHub profile if you want to discuss (although there's also a Gitter chat that @codeanticode set up on that topic) |
|
What is the purpose of looking into LWJGL?
Op zo 29 sep. 2019 om 19:52 schreef Neil C Smith <notifications@github.com>:
… @sampottinger <https://github.com/sampottinger> reminds me I think I
forgot to reply to you re. LWJGL? Use the email on my GitHub profile if you
want to discuss (although there's also a Gitter chat that @codeanticode
<https://github.com/codeanticode> set up on that topic)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#5753?email_source=notifications&email_token=AAFUKWQCXWQVSXFXECVXXT3QMDTMXA5CNFSM4GP2FC5KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD732VJA#issuecomment-536324772>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAFUKWV7HKXYHK4EAVOH6UTQMDTMXANCNFSM4GP2FC5A>
.
|
|
Hey @neilcsmith-net! Happy to hear from you and would love to discuss. I'll reach out on email. @clankill3r - JOGL is in a weird spot and Ben articulated that LWJGL may be a better path forward long term. It seems JOGL has been a reliable partner for Processing for some time and they are making progress towards 2.4 which solves the problems we need solved but, while it looks close to finished, it's not clear when it will land. For context: while they are active in development now, they haven't released a new version since 2015. LWJGL may be a bit more active in its development. That said, it comes with some issues for us and, in particular, will challenge some parts of core because it is doesn't use AWT (see GLFW). So, there's pros and cons. |
|
@clankill3r as Sam alludes to, why would you not really? JOGL has been a dead project for the last few years, and while it's picked up again, Processing has had to ship a patched version for the last few releases. There is still no stable JOGL that supports Java 11 properly. There is already a mostly functional LWJGL renderer by @codeanticode at https://github.com/codeanticode/processing-lwjgl However some changes to core are required to support running with headless AWT (most of the AWT usage can stay!), and also to support running events on the main thread ( |
|
Ciao Sam few word about PVector, I read there an idea to change PVector class for Processing 4 https://github.com/processing/processing/wiki/Processing-4#other-nuts-and-bolts . Few time ago i propose do @benfry to work it, if he want on my Vector class to design it specificly to Processing https://github.com/StanLepunK/Rope/tree/master/src/rope/vector |
|
@StanLepunK Interesting! Surely a time to be considering providing double variants, and maybe moving most of Processing's maths API to double? It's 2019 after all! 😄 Aside - that wiki page is somewhat depressing, particularly the Goodbye Java? section - spent 15+ years reading semi-idiotic rants from Processing people about Sun, Oracle and Java. Java and OpenJDK is in the healthiest place it's been in in many years. It's far from a mess, unlike Processing right now! |
|
@neilcsmith-net Go to the double, it's something what I thing to my vector class too, so it's a good idea... now just wait what @sampottinger and @benfry think about that... I've a feeling the "Goob bye Java" is not for today :) |
|
You can't just go to double. It has been float for years, libraries expect
it to be float.
Also the times when people actually need double is almost never. I had it a
few times making a phyllotaxis for example but in general it's overkill.
Also it does not come for free. The L1 and L2 cache size are rather small.
Going to double will increase cache misses which are slow. There is a
reason why game engines like source, unity and unreal have vectors using
float instead of double.
They also provide no alternative to the float version.
The only thing I would do is get rid of:
`transient protected float[] array;`
It makes PVector exceed from 108 bits to 140 bits so it needs more padding.
Op wo 2 okt. 2019 om 14:33 schreef Stan Le Punk <notifications@github.com>:
… @neilcsmith-net <https://github.com/neilcsmith-net> Go to the double,
it's something what I thing to my vector class too, so it's a good idea...
now just what @sampottinger <https://github.com/sampottinger> and @benfry
<https://github.com/benfry> think about that... I've a feeling the "Goob
bye Java" is not for today :)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#5753?email_source=notifications&email_token=AAFUKWVS6IPFA2M4HW43OYDQMSILDA5CNFSM4GP2FC5KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAESRSQ#issuecomment-537471178>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAFUKWSR5TTYE75TSW6Y7W3QMSILDANCNFSM4GP2FC5A>
.
|
|
@clankill3r ok, I understand the problem for the double and the cache memory, in my library that can I propose to Processing, the idea is have a part for the ivec(int), vec (float), bvec (boolean) and why not a dvec(double). For this time there is only bvec, ivec and vec available. I don't understand and what you mean about the |
|
@StanLepunK clankill3r was saying that this line is taking up precious RAM/cache for every PVector object, possibly slowing down memory-heavy things. Every vector type in your library stores 13 32-bit numbers, coming in at 52 bytes, compared to only 8 bytes if I only needed a |
|
@clankill3r if you were at all bothered about cache misses you wouldn't be using PVector in the first place, you'd be using float[] or FloatBuffer. A PVector that easily provides a window into a contiguous buffer would be useful in such cases. But using float over double in general is a PITA, and causes odd deviations from upstream Java math code for little reason. If v4 is a breaking change, and Java 11 probably is, then it's worth at least considering?! |
|
@neilcsmith-net I would normally also use doubles for pretty much everything. One of the few exceptions is when working with a GPU. Simply because the GPU typically takes floats. It's not a major issue, one could even allow both, but would probably end up casting to float before passing it to the GPU. (I typically don't use PVectors, but that doesn't mean they can't end up being used by Processing itself. But if I am to make something that really requires more performance, I would go more low level than Processing anyway.) From the OpenGL documentation on vertex specification (https://www.khronos.org/opengl/wiki/Vertex_Specification):
Casting doubles to floats everywhere does add some cost, up to 5 additional clock cycles for the conversion instruction alone on modern Intel CPUs. There will be additional instructions involved as well, but they are likely less costly. Of course, most Processing applications don't require optimizing away a few microseconds each frame. |
|
@Sardtok maybe check how many times PApplet is already casting from float to double and back again in its use of And we've got well off-topic - sorry @sampottinger - just suggesting that this may also be the time (given breaking changes) to look at the implementations of some of those methods in PApplet. Maybe something to chit-chat more on the forum? |
|
Hey there @StanLepunK, @neilcsmith-net, @clankill3r, @dzaima, @Sardtok - Thanks for weighing in! Let's redirect the PVector conversation to sampottinger#111. |
Introduces ANTLR4 and Java 8 language feature support within IDE while also adding additional hooks for localization of syntax error messages, addressing processing/processing#3054 and processing/processing#3055. The PR is broadly a continuation of processing/processing#3055, bringing it up to speed with the latest Processing master plus the changes introduced at processing/processing#5753. **Requires processing/processing#5753 as pre-requisite.** This introduces a number of edits beyond processing/processing#3055 beyond compatibility with current Processing master and processing/processing#5753 including: - Update to the grammar itself - Change ANTLR listeners to emit `TextTransform.Edit` to unify JDT-based `PreprocessingService` and `JavaBuild`, removing code with duplicate purpose. - Introduction of syntax error rewriting with support for localization. - Addition of complete localized strings set for English and Spanish. - Addition of partial localized strings set for other languages. - Refactor of ANTLR-related code for testability and readability - Expansion of tests including full parse tests for new Java features (type inference, lambdas).
* Move to ANTLR 4 with Java 11 lang features and localization. Introduces ANTLR4 and Java 8 language feature support within IDE while also adding additional hooks for localization of syntax error messages, addressing processing/processing#3054 and processing/processing#3055. The PR is broadly a continuation of processing/processing#3055, bringing it up to speed with the latest Processing master plus the changes introduced at processing/processing#5753. **Requires processing/processing#5753 as pre-requisite.** This introduces a number of edits beyond processing/processing#3055 beyond compatibility with current Processing master and processing/processing#5753 including: - Update to the grammar itself - Change ANTLR listeners to emit `TextTransform.Edit` to unify JDT-based `PreprocessingService` and `JavaBuild`, removing code with duplicate purpose. - Introduction of syntax error rewriting with support for localization. - Addition of complete localized strings set for English and Spanish. - Addition of partial localized strings set for other languages. - Refactor of ANTLR-related code for testability and readability - Expansion of tests including full parse tests for new Java features (type inference, lambdas). * Ask travis for ant upgrade prior to run. * Ask Travis for java11 update. * Add openjdk ppa * Travis no confirmation on add ppa. * Force newer ant on travis. * Swtich ant download to www-us mirror. * Switch ant to 1.10.7 * Start x for unit tests in travis. * More complete start x in travis. * Revert x in travis. * Try x in services.
|
Hey friends! Just a heads up that benfry/processing4#1 just posted. |
|
Nice, good luck with the merge battle. |
|
Closing now that development has moved here: https://github.com/processing/processing4 |
Moves to OpenJDK11 as part of #5750. There's lots in here so a quick rundown of the bigger stuff...
Primary required changes:
Some changes directly support OpenJFX / OpenJDK 11:
Secondary required changes:
There were some secondary required changes that impacted the usability of Processing after having moved to OpenJFX / OpenJDK 11:
java.ext.dirs. Would be forced by JEP 220.Incidental changes:
This was (ahem) a bit of a larger PR with the above modifications. So, I wanted to introduce automated tests when possible and convenient along with a few changes for platform sustainability in order to support development:
build/build.xmljre/build.xml.