Investigate what it would take for a future version of vlcj to use Java Foreign Linker API instead of JNA

[caprica]

Incubating with JDK 16, so this is far from usable for vlcj to be honest, but it might offer a nice path for the future.

https://openjdk.java.net/jeps/389

Pre-requisite for development: Java 16, using incubator flags.

Pre-requisite for release: Java 19 (?!), September 2022 (it will still be incubating in JDK 18).

Will almost certainly require use of the Java Module System. :-( 👎🏻

[caprica]

https://github.com/openjdk/panama-foreign/blob/foreign-jextract/doc/panama_ffi.md

[caprica]

Seems quite feasible now. But very verbose. Will need a LOT of hand-crated bindings.

jextract works, but it generates way too much useless stuff, and it's pretty difficult to read the generated stuff (I know the idea is just to use the generated classes, but I like to see/understand what's actually going on).

[caprica]

A lot of work here writing new hand-crafted bindings, but jextract just not giving me the results I want.

It does all seem to be feasible though - basic media player created with event callbacks fully working.

There are going to be issues around callbacks in that they require a static Java method to call back into - so when dealing e.g. with multiple media players generating events there is going to need to be a way to discriminate which events are for which media player (the native API provides a user data value that should help here).

[caprica]

Also need a replacement for:

• JNA's Native.getComponentId()
• native fullscreen strategies

For these, I am tempted to move the native stuff to a separate project that could be used in a transition period, and then work on gradually removing them.

The goal is to remove all JNA from vlcj-core to be replaced with Panama FFI, even if that means keeping some JNA code around (for a while) in a separate project/module.

[caprica]

So much work here...

[caprica]

Dropping Native.getComponentId() is going to be quite difficult, it may be necessary to keep JNA around for it:

https://github.com/java-native-access/jna/blob/a0641dd928776a8e81478015a1f14a39326887f3/native/dispatch.c#L3061

[caprica]

Need a new solution for native library discovery.

[caprica]

Also probably need to move to instance-based bindings rather than global static methods.

[caprica]

An instance-based approach leads to MASSIVE amounts of change unfortunately.

However, it should provide the following benefits:

• More isolation between difference native library instances
• Possibility to have multiple media player factory instances with different configurations (i.e. different native VLC configuration switches per factory instance)

[caprica]

Remember to check back with the master branch to bring recent changes over where appropriate - this FFI branch was taken some time ago.

[caprica]

The instance-based approach is not feasible with the changes to FFI included in JDK17 onwards.

Prior to JDK17, with FFI it was possible to invoke a symbol lookup on the native library by explicitly specifying the absolute native library path - now that is gone, you lookup the symbol based on the "standard" native library loading mechanism. That means, once libvlc is loaded, it's loaded. You can't load it again from a different place.

Removing the instance-based approach makes everything simpler in fact, although there is a lot of rework required to get rid of it.

[caprica]

Reworking the callbacks is proving to be extremely tedious, but almost everything else is fine now.

[caprica]

FFI with JDK 17 is in the incubation phase.

Unfortunately, even with JDK 18 due March 2022 it appears that FFI will remain in incubation phase.

There are changes coming with JDK 18 pertaining to things like boolean types (that will require some rework /sad-face), although it is also mooted to come with new methods to help with copying arrays from memory handles - this has only been used in a very small number of places in vlcj.

[caprica]

For reference: https://openjdk.java.net/jeps/419

[caprica]

Current status of the vlcj-ffi branch as of Feb 7th 2022...

Pending some cleanup, at least as far as FFI in JDK 17 goes, all functionality has been migrated to FFI for the libvlc bindings.

There is a corresponding vlcj branch which has almost entirely been migrated to FFI for libvlc.

There are still some holdouts, the use of X11, libc, kernel32 and msvrt in a small number of places for various things.

The native full-screen handling has not yet been migrated, and may well become a separate set of projects.

The most annoying problem remaining is not having a good substitute for JNA's Native.getComponentId, see #1107.

In terms of testing, normal media players work, as do callback media players, and video engine (OpenGL) video players although there is a LOT of functionality to test and this will take considerable time.

The vlcj-examples project needs to be migrated (at least in an ffi branch) to test vlcj with FFI.

The plan is for vlcj-5.x to remain with JDK 8 and track VLC 4.x (which still appears to be a LONG time away).

This means probably we will use vlcj-6.x for the FFI release of vlcj.

The end goal is NOT to maintain both the JNA and FFI versions of vlcj, eventually the FFI version will be the baseline.

[caprica]

Native discovery...

Since we will no longer be using JNA's library loading, some changes to native library discovery may be needed.

FFI symbol lookup works by searching java.library.path (which can of course be set as a -D system property).

But also, we can use System.load("/absolute/path/to/libvlc.so") instead of just System.loadLibrary("vlc").

Note that even with FFI, we must explicitly load libvlc by one of those calls.

In principle therefore, the existing native discovery mechanism could be kept mostly in tact, with a small change to use System.load, and if discovery initially fails then fall back to System.loadLibrary perhaps.

Unfortunately it seems we must still set VLC_PLUGIN_PATH, although testing without this when VLC 4.x final release is made would be worth a try.

[caprica]

There is another wrinkle.

Native.loadLibrary from JNA can find e.g. libX11 and load it (we need XInitThreads on Linux).

But System.loadLibrary does not find it. Indeed the default java.library.path does not reference a directory that contains libX11.so.

On my Linux it is here: /usr/lib/x86_64-linux-gnu/libX11.so

So we would have to use some sort of discovery (e.g. well-known directory locations to find this library as well).

Not ideal.

[caprica]

Native discovery has been completely refactored to support the previously described issue.

Discovery moves to the vlcj-ffi project, and is somewhat simpler.

[caprica]

JDK 18 has just been released and unfortunately the new incubator release of FFI has caused significant breakage.

The code has now been migrated and all existing tests are still passing.

There are over 250 unit tests to check the bindings are correct (well, in some cases just that they don't crash).

The basic examples have also been tested successfully (although not all examples as there are MANY).

Further, the javafx demo project (that exercises native callbacks) has also been confirmed working.

Now we wait to see what changes will come with JDK 19 (six months from now), and if in indeed FFI moves out of incubator status.

[caprica]

There are now around 400 unit tests for the new FFI bindings, all working with JDK 18.

[chengenzhao]

Panama should be start preview in 19 and 2nd preview in 20 and eventually released in 21
After that would you mind providing jmod to be downloaded just like JavaFX project does?
e.g. vlcj_osx-x64.jmod(including vlc.dylib in it), vlcj_osx-aarch64.jmod(including vlc-m1.dylib in it),
vlcj_win-x64.jmod, vlcj_linux-aarch64.jmod etc.

[caprica]

You are massively oversimplifying what it would take to ship VLC with a Java application.

There is not just vlc.dylib (or .so, or .dll). There are a LOT of individual shared objects for all the plugins, and also those plugins dynamically link against other things like ffmpeg, fontconfig, qt, etc... etc... etc...., and you want that for ALL possible CPU architectures?!

It is much more sensible IMO to use an OS installer and depend on LibVLC instead.

[caprica]

Quite a few changes required to move from incubator in JDK 18 to preview in JDK 19...

Blockers for JDK 19:

• using early access builds
• early access builds not supported by IntelliJ IDEA
• JDK 19 not supported by Lombok

[caprica]

Latest version of IntelliJ IDEA does now support JDK 19, but Lombok still does not.

[caprica]

Dropped Lombok.

jdk19 branch now compiles with latest jdk19 EA build and all tests (450+) pass on Linux and Windows.

[caprica]

Some issues around using restricted *unsafe" API's that would require a JVM command-line switch that I would like to avoid. There may be alternatives here, it needs investigation.

[caprica]

This is now working with the first public release of JDK 19. It still requires "--enable-preview" settings.

It is unlikely that any release of the FFI version of vlcj will be made until VLC 4.0.0 is released.

[caprica]

JDK 20 is now released (today), FFI is in "second preview", so still not generally available.

Need to check for any problems with an upgrade to JDK 20.

[caprica]

Changes....

https://bugs.openjdk.org/browse/JDK-8295045

[caprica]

Latest JEP for jdk20 https://openjdk.org/jeps/434

[caprica]

Unfortunately a LOT of changes from jdk-19 to jdk-20 for this.

Having to use asUnbounded() everywhere seems sketchy but necessary.

Almost all unit tests confirmed working, at least on Linux.

[caprica]

Need to rename the project/artefact to vlcj-ffm instead of vlcj-ffi to match the JDK API name.

[caprica]

All unit-tests, on Linux, passing with *racle's JDK 20 and the Temurin JDK 20.

[caprica]

All unit-tests passing on Windows.