Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Linux x64 build failing #18

Closed
JohnRDOrazio opened this issue Nov 4, 2020 · 9 comments
Closed

Linux x64 build failing #18

JohnRDOrazio opened this issue Nov 4, 2020 · 9 comments

Comments

@JohnRDOrazio
Copy link
Member

so here's what we've got so far for a Linux build:

  • the JCEF linux x64 build was built with JDK 11, so trying to use JDK 8 (1.8) will not work, the build will not succeed. Using JDK 11 works but introduces new problems: in order to dynamically load the JCEF native libraries at runtime we can use a reflection hack to reset the System user path and make sure that the path to the JCEF native libraries are in the user path and reset the java.library.path with this new path. However this hack only works with JDK 8, not with JDK 11. For JDK >= 9 there is however another approach using Lookup and MethodHandles (see https://stackoverflow.com/questions/15409223/adding-new-paths-for-native-libraries-at-runtime-in-java). This seems to be working, however the JCEF component is not being initialized...

  • studying the sample application provided by the JCEF build for Linux x64 I can see that the LD_LIBRARY_PATH environment variable also needs to be set with the path to the JCEF native libraries. The plugin code is dynamically downloading the JCEF build from a github release and saving it to this folder: System user path + '/.BibleGetPluginOpenOffice/JCEF' . So this path is needed in the LD_LIBRARY_PATH environment variable. Perhaps this is all that is needed to get the JCEF component to work? So how do we set this environment variable? Can we reset it at runtime?

  • For testing purposes I tried exporting the LD_LIBRARY_PATH variable in my Ubuntu 20.04 instance (I'm using WSL2 on Windows 10, and using X11 forwarding in order to be able to launch graphical interfaces = Netbeans, Apache OpenOffice...), and I added a System.out.println() in the code to see the value of the LD_LIBRARY_PATH variable. This shows me that the variable now contains paths that weren't in the variable before on the terminal, but it does not contain the path to the JCEF native libraries. So I'm guessing the Ant script in Netbeans is maybe overwriting this variable during the build? So next step to test a solution is perhaps to figure out how to set this variable in the Ant script? Or find a way to overwrite it again at runtime? I tried using the same method used to reset the java.library.path, to try and reset the LD_LIBRARY_PATH variable, but didn't succeed, it seems to generate an exception. And I don't believe it's a recommended approach, environment variables really should be set in the environment before launching a Java runtime and the JDK releases after JDK 8 have been discouraging overwriting these variables at runtime, they started showing warnings to discourage people from using this approach...

@JohnRDOrazio
Copy link
Member Author

JohnRDOrazio commented Nov 7, 2020

From my latest tests:

  1. I exported LD_LIBRARY_PATH with the path to the JCEF native libraries, inside my .bashrc profile. Probably why it wasn't working last time was that I needed to log out and log back in again, so the change could be picked up in the environment, because now it's actually being picked up when I debug->run the add-on. LD_LIBRARY_PATH contains both the OpenOffice library path, the JDK library path, and the JCEF library path. So now we just have to figure out how to set this environment variable before launching the add-on. Same goes for the LD_PRELOAD environment variable which should have value "libcef.so".

  2. Since Netbeans is now picking up on these environment variables, it started complaining that a series of dependencies are not met. These are dependencies needed by Chrome / Chromium in general, so if Chrome isn't already installed on the system, these dependencies might not be met (as was the case in my WSL2 Ubuntu instance). This took care of that:

    sudo apt-get install gconf-service libasound2 libatk1.0-0 libatk-bridge2.0-0 libc6 libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libgbm-dev libgtk-3-0 libnspr4 libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxcursor-dev libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils 
  3. However I was still getting a build error complaining that libnss3.so was not at version 3.22. On my system, libnss3.so was actually at version 3.49, the issue was that OpenOffice also uses libnss3.so and this was the first one getting picked up from the library path, and this one is at version 3.14. So I backed up the libnss3.so and the libnssutil3.so in the /opt/openoffice4/program/ path, and I copied the system version to this folder. That took care of that error and didn't produce any further errors in OpenOffice, so I will ask the OpenOffice team if these can be updated.

  4. I was still getting some exceptions and the JCEF component was not correctly initialized, but this was because we were still missing a needed step at this point: on Linux and MacOS systems the jcef libraries need to be bootstrapped by calling CefApp.startup(new String[0]); before cefApp = CefApp.getInstance(settings);. CefApp.startup(args) returns a boolean, so we can use a conditional, if startup returns true then we can CefApp.getInstance.

  5. I was still getting an exception at this point, which pointed me to this solution : jcef_helper needs to be executable! This is simple enough: sudo chmod +x jcef_helper.

  6. And now everything is starting to fall into place. I'm still getting an exception which I believe is due to my using X11 graphics forwarding between WSL2 and Windows 10 (via VcXsrv ), so this might be a corner case issue and probably won't be a problem in most Linux environments:

Caught handled GLException: EGLGLXDrawableFactory - Could not initialize shared resources for EGLGraphicsDevice[type .egl, v0.0.0, connection 172.21.144.1:0, unitID 0, handle 0x0, owner true, ResourceToolkitLock[obj 0x1ce845d4, isOwner true, <1c38ddb8, 4da07790>[count 1, qsz 0, owner <Thread-9-SharedResourceRunner>]]] on thread Thread-9-SharedResourceRunner
    [0]: jogamp.opengl.egl.EGLDrawableFactory$SharedResourceImplementation.createSharedResource(EGLDrawableFactory.java:561)
    [1]: jogamp.opengl.SharedResourceRunner.run(SharedResourceRunner.java:353)
    [2]: java.base/java.lang.Thread.run(Thread.java:834)
Caused[0] by GLException: Failed to created/initialize EGL display incl. fallback default: native 0x0, error 0x3008/0x3005 on thread Thread-9-SharedResourceRunner
    [0]: jogamp.opengl.egl.EGLDisplayUtil.eglGetDisplayAndInitialize(EGLDisplayUtil.java:381)
    [1]: jogamp.opengl.egl.EGLDisplayUtil.access$300(EGLDisplayUtil.java:61)
    [2]: jogamp.opengl.egl.EGLDisplayUtil$1.eglGetAndInitDisplay(EGLDisplayUtil.java:404)
    [3]: com.jogamp.nativewindow.egl.EGLGraphicsDevice.open(EGLGraphicsDevice.java:125)
    [4]: jogamp.opengl.egl.EGLDrawableFactory$SharedResourceImplementation.createEGLSharedResourceImpl(EGLDrawableFactory.java:575)
    [5]: jogamp.opengl.egl.EGLDrawableFactory$SharedResourceImplementation.createSharedResource(EGLDrawableFactory.java:559)
    [6]: jogamp.opengl.SharedResourceRunner.run(SharedResourceRunner.java:353)
    [7]: java.base/java.lang.Thread.run(Thread.java:834)
Caught handled GLException: EGLGLXDrawableFactory - Could not initialize shared resources for X11GraphicsDevice[type .x11, connection 172.21.144.1:0, unitID 0, handle 0x0, owner false, ResourceToolkitLock[obj 0x56f3c465, isOwner true, <3001a75f, fa43dc0>[count 1, qsz 0, owner <Thread-9-SharedResourceRunner>]]] on thread Thread-9-SharedResourceRunner
    [0]: jogamp.opengl.egl.EGLDrawableFactory$SharedResourceImplementation.createSharedResource(EGLDrawableFactory.java:561)
    [1]: jogamp.opengl.SharedResourceRunner.run(SharedResourceRunner.java:353)
    [2]: java.base/java.lang.Thread.run(Thread.java:834)
Caused[0] by GLException: Failed to created/initialize EGL display incl. fallback default: native 0x0, error 0x3008/0x3005 on thread Thread-9-SharedResourceRunner
    [0]: jogamp.opengl.egl.EGLDisplayUtil.eglGetDisplayAndInitialize(EGLDisplayUtil.java:381)
    [1]: jogamp.opengl.egl.EGLDisplayUtil.access$300(EGLDisplayUtil.java:61)
    [2]: jogamp.opengl.egl.EGLDisplayUtil$1.eglGetAndInitDisplay(EGLDisplayUtil.java:404)
    [3]: com.jogamp.nativewindow.egl.EGLGraphicsDevice.open(EGLGraphicsDevice.java:125)
    [4]: jogamp.opengl.egl.EGLDrawableFactory$SharedResourceImplementation.createEGLSharedResourceImpl(EGLDrawableFactory.java:575)
    [5]: jogamp.opengl.egl.EGLDrawableFactory$SharedResourceImplementation.createSharedResource(EGLDrawableFactory.java:559)
    [6]: jogamp.opengl.SharedResourceRunner.run(SharedResourceRunner.java:353)
    [7]: java.base/java.lang.Thread.run(Thread.java:834)

@JohnRDOrazio
Copy link
Member Author

Another update: running sudo apt install nvidia-driver-390 to install the nvidia drivers in my Ubuntu WSL2 instance seems to have taken care of the last error above. Now I'm getting this:

CefAppState has changed : NEW
we seem to have an instance of CefApp: version = JCEF Version = 84.3.8.265
CEF Version = 84.3.8
Chromium Version = 84.0.4147.105
CefAppState has changed : INITIALIZING
initialize on Thread[AWT-EventQueue-0,6,main] with library path /usr/lib/jvm/jdk-11.0.9/lib/amd64/client;/usr/lib/jvm/jdk-11.0.9/lib/amd64/native_threads;/usr/lib/jvm/jdk-11.0.9/lib/amd64;/usr/lib/jvm/jdk-11.0.9/lib;/usr/java/packages/lib;/usr/lib64;/lib64;/lib;/usr/lib;/home/johnrdorazio/.BibleGetOpenOfficePlugin/JCEF
[1107/161033.408492:ERROR:main_delegate.cc(938)] Could not load cef.pak
[1107/161033.408622:ERROR:main_delegate.cc(955)] Could not load cef_100_percent.pak
[1107/161033.408630:ERROR:main_delegate.cc(964)] Could not load cef_200_percent.pak
[1107/161033.408637:ERROR:main_delegate.cc(974)] Could not load cef_extensions.pak
LaunchProcess: failed to execvp:
/usr/lib/jvm/jdk-11.0.9/lib/amd64/client;/usr/lib/jvm/jdk-11.0.9/lib/amd64/native_threads;/usr/lib/jvm/jdk-11.0.9/lib/amd64;/usr/lib/jvm/jdk-11.0.9/lib;/usr/java/packages/lib;/usr/lib64;/lib64;/lib;/usr/lib;/home/johnrdorazio/.BibleGetOpenOfficePlugin/JCEF/jcef_helper
LaunchProcess: failed to execvp:
/usr/lib/jvm/jdk-11.0.9/lib/amd64/client;/usr/lib/jvm/jdk-11.0.9/lib/amd64/native_threads;/usr/lib/jvm/jdk-11.0.9/lib/amd64;/usr/lib/jvm/jdk-11.0.9/lib;/usr/java/packages/lib;/usr/lib64;/lib64;/lib;/usr/lib;/home/johnrdorazio/.BibleGetOpenOfficePlugin/JCEF/jcef_helper
[1107/161033.488000:WARNING:gpu_process_host.cc(1233)] The GPU process has crashed 1 time(s)
[1107/161033.488535:WARNING:gpu_process_host.cc(1233)] The GPU process has crashed 2 time(s)
[1107/161033.489069:WARNING:gpu_process_host.cc(1233)] The GPU process has crashed 3 time(s)
[1107/161033.489656:WARNING:gpu_process_host.cc(1233)] The GPU process has crashed 4 time(s)
[1107/161033.490243:WARNING:gpu_process_host.cc(1233)] The GPU process has crashed 5 time(s)
[1107/161033.490916:WARNING:gpu_process_host.cc(1233)] The GPU process has crashed 6 time(s)
[1107/161033.491416:WARNING:gpu_process_host.cc(1233)] The GPU process has crashed 7 time(s)
[1107/161033.493966:ERROR:extension_system.cc(71)] Failed to parse extension manifest.
/home/johnrdorazio/NetBeansProjects/bibleget-openoffice/nbproject/build-uno-impl.xml:454: exec returned: 133
BUILD FAILED (total time: 21 seconds)

@JohnRDOrazio
Copy link
Member Author

JohnRDOrazio commented Nov 7, 2020

When testing the sample application from the command line, a simple ./run.sh gives this error:

[1107/180149.482477:ERROR:viz_main_impl.cc(150)] Exiting GPU process due to errors during initialization

Running ./run.sh detailed --disable-gpu prevents that error.

Therefore trying CefApp.startup(new String[]{"--disable-gpu"}) and cefApp = CefApp.getInstance(new String[]{"--disable-gpu"}, settings); in the add-on code.

However this does not get rid of the previous error.

@JohnRDOrazio
Copy link
Member Author

JohnRDOrazio commented Nov 7, 2020

This is the strangest thing, not even this gets rid of the previous error:

settings.resources_dir_path = Paths.get("/home/johnrdorazio/.BibleGetPluginOpenOffice/JCEF/").toAbsolutePath().normalize().toString();
settings.locales_dir_path = Paths.get("/home/johnrdorazio/.BibleGetPluginOpenOffice/JCEF/locales/").toAbsolutePath().normalize().toString();

I can avoid the cef.pak not loading error with this:

settings.pack_loading_disabled = true;

However it still complains about not finding the locale pak files.
The pak files are definitely in the right place, why is not finding them?

we seem to have an instance of CefApp:
JCEF Version = 84.3.8.265
CEF Version = 84.3.8
Chromium Version = 84.0.4147.105
CefAppState has changed : INITIALIZING
initialize on Thread[AWT-EventQueue-0,6,main] with library path /usr/lib/jvm/jdk-11.0.9/lib/amd64/client;/usr/lib/jvm/jdk-11.0.9/lib/amd64/native_threads;/usr/lib/jvm/jdk-11.0.9/lib/amd64;/usr/lib/jvm/jdk-11.0.9/lib;/usr/java/packages/lib;/usr/lib64;/lib64;/lib;/usr/lib;/home/johnrdorazio/.BibleGetOpenOfficePlugin/JCEF
[1107/223143.762119:WARNING:resource_bundle.cc(435)] locale_file_path.empty() for locale 
LaunchProcess: failed to execvp:
/usr/lib/jvm/jdk-11.0.9/lib/amd64/client;/usr/lib/jvm/jdk-11.0.9/lib/amd64/native_threads;/usr/lib/jvm/jdk-11.0.9/lib/amd64;/usr/lib/jvm/jdk-11.0.9/lib;/usr/java/packages/lib;/usr/lib64;/lib64;/lib;/usr/lib;/home/johnrdorazio/.BibleGetOpenOfficePlugin/JCEF/jcef_helper
LaunchProcess: failed to execvp:
/usr/lib/jvm/jdk-11.0.9/lib/amd64/client;/usr/lib/jvm/jdk-11.0.9/lib/amd64/native_threads;/usr/lib/jvm/jdk-11.0.9/lib/amd64;/usr/lib/jvm/jdk-11.0.9/lib;/usr/java/packages/lib;/usr/lib64;/lib64;/lib;/usr/lib;/home/johnrdorazio/.BibleGetOpenOfficePlugin/JCEF/jcef_helper
[1107/223143.787787:VERBOSE1:pref_proxy_config_tracker_impl.cc(185)] 0x7fcea40b1210: set chrome proxy config service to 0x7fcea4076c60
[1107/223143.790468:VERBOSE1:webrtc_internals.cc(119)] Could not get the download directory.
[1107/223143.792659:WARNING:gpu_process_host.cc(1233)] The GPU process has crashed 1 time(s)
[1107/223143.793312:WARNING:gpu_process_host.cc(1233)] The GPU process has crashed 2 time(s)
[1107/223143.793964:WARNING:gpu_process_host.cc(1233)] The GPU process has crashed 3 time(s)
[1107/223143.794429:WARNING:gpu_process_host.cc(1233)] The GPU process has crashed 4 time(s)
[1107/223143.795033:WARNING:gpu_process_host.cc(1233)] The GPU process has crashed 5 time(s)
[1107/223143.796022:WARNING:gpu_process_host.cc(1233)] The GPU process has crashed 6 time(s)
[1107/223143.796796:WARNING:gpu_process_host.cc(1233)] The GPU process has crashed 7 time(s)
[1107/223143.798003:ERROR:extension_system.cc(71)] Failed to parse extension manifest.
/home/johnrdorazio/NetBeansProjects/bibleget-openoffice/nbproject/build-uno-impl.xml:454: exec returned: 139
BUILD FAILED (total time: 11 seconds)

@JohnRDOrazio
Copy link
Member Author

Interesting, setting settings.windowless_rendering_enabled = false; instead of settings.windowless_rendering_enabled = OS.isLinux(); seems to get rid of the pak missing errors for the locale files?

@JohnRDOrazio
Copy link
Member Author

ooookkkk I fixed the error for the pak files not loading, it was due to the export LD_LIBRARY_PATH having disappeared from my .bashrc, so I put that in again along with export LD_PRELOAD and now thoses errors are gone, pak files are found and loaded. However browser UI is completely black, it's not rendering anything. So this is the current situation:

we seem to have an instance of CefApp:
JCEF Version = 84.3.8.265
CEF Version = 84.3.8
Chromium Version = 84.0.4147.105
CefAppState has changed : INITIALIZING

[1108/150948.696673:INFO:cpu_info.cc(53)] Available number of cores: 8

[1108/150948.696734:VERBOSE1:zygote_main_linux.cc(217)] ZygoteMain: initializing 0 fork delegates
[1108/150948.699041:INFO:cpu_info.cc(53)] Available number of cores: 8
initialize on Thread[AWT-EventQueue-0,6,main] with library path /home/johnrdorazio/.BibleGetOpenOfficePlugin/JCEF

[1108/150948.699142:VERBOSE1:zygote_main_linux.cc(217)] ZygoteMain: initializing 0 fork delegates
CefAppState has changed : INITIALIZED
we seem to have a client instance of cefApp
[1108/150948.739444:VERBOSE1:pref_proxy_config_tracker_impl.cc(185)] 0x7f2a5007c740: set chrome proxy config service to 0x7f2a5002a370

[1108/150948.741137:VERBOSE1:webrtc_internals.cc(119)] Could not get the download directory.
libGL error: No matching fbConfigs or visuals found
libGL error: failed to load driver: swrast
[1108/150948.856659:VERBOSE1:gpu_init.cc(377)] Quit GPU process launch to fallback to SwiftShader cleanly on Linux
[1108/150948.858652:ERROR:viz_main_impl.cc(150)] Exiting GPU process due to errors during initialization

libGL error: No matching fbConfigs or visuals found
libGL error: failed to load driver: swrast
libGL error: No matching fbConfigs or visuals found
libGL error: failed to load driver: swrast
libGL error: No matching fbConfigs or visuals found
libGL error: failed to load driver: swrast
[1108/150959.032376:ERROR:browser_host_impl.cc(276)] Creating a windowless browser without setting CefSettings.windowless_rendering_enabled may result in reduced performance or runtime errors.
[1108/150959.034186:VERBOSE1:pref_proxy_config_tracker_impl.cc(185)] 0x7f2a5017e260: set chrome proxy config service to 0x7f2a5016a3d0
Exception in thread "Thread-140" Exception in thread "Thread-140" malloc_consolidate(): invalid chunk size

@JohnRDOrazio
Copy link
Member Author

to fix the message from above Creating a windowless browser without setting CefSettings.windowless_rendering_enabled may result in reduced performance or runtime errors., I put the JCEF settings back to:

settings.windowless_rendering_enabled = OS.isLinux();

and that error message is gone. However it seeems that the --disable-gpu switch is not working, still getting the libGL errors... But this could very well just be the WSL2 Ubuntu instance which isn't exactly meant for the GPU stuff, perhaps in a regular Ubuntu instance we won't have these errors. Needs testing

@JohnRDOrazio
Copy link
Member Author

JohnRDOrazio commented Nov 9, 2020

just a note to self, this seems to fix the graphics error on WSL2:

export LIBGL_ALWAYS_INDIRECT=1

Add this to bash profile (before export DISPLAY=$(awk '/nameserver / {print $2; exit}' /etc/resolv.conf 2>/dev/null):0? ), exit and launch WSL2 Ubuntu instance again. Fixed on Ubuntu 18.03.

The error was:

libGL error: No matching fbConfigs or visuals found
libGL error: failed to load driver: swrast

Test by running:

 glxinfo | grep OpenGL

(The package mesa-utils must be installed already: sudo apt-get install mesa-utils)

Solution was here: https://askubuntu.com/a/1285593/433276

@JohnRDOrazio
Copy link
Member Author

build is now fixed with the new Installer process. Last commit to fix this process is 0657f21 .
Still a couple things to perfect, but at least we have a working setup process now. Closing.

@JohnRDOrazio JohnRDOrazio moved this from In progress to Done in BibleGet add-on for OpenOffice Nov 16, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

1 participant