Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

StackOverflowError when loading .so files #331

Closed
gertjanal opened this issue Dec 7, 2016 · 4 comments
Closed

StackOverflowError when loading .so files #331

gertjanal opened this issue Dec 7, 2016 · 4 comments
Labels
question

Comments

@gertjanal
Copy link
Contributor

gertjanal commented Dec 7, 2016 •
edited
Loading

Given:

static {
    // Register all formats and codecs
    avformat.av_register_all();
}
// ffmpeg code usage

This works on my machine (Ubuntu 14.04.5 LTS, x86_64) and on the machines of my colleagues (Linux Mint 18 Sarah, x86_64).

However on our deployment environment (Ubuntu 15.10, x86_64) the Loader.load() fails:

Caused by: java.lang.StackOverflowError
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
at java.lang.Runtime.load0(Runtime.java:809)
at java.lang.System.load(System.java:1086)
at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:637)
at org.bytedeco.javacpp.Loader.load(Loader.java:492)
at org.bytedeco.javacpp.Loader.load(Loader.java:409)
at org.bytedeco.javacpp.avcodec.<clinit>(avcodec.java:13)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.bytedeco.javacpp.Loader.load(Loader.java:464)
at org.bytedeco.javacpp.Loader.load(Loader.java:409)
at org.bytedeco.javacpp.avformat.<clinit>(avformat.java:14)

I don't think this is a real StackOverflow as the length of the stack is not that large. The issue seems related to bytedeco/javacv#113 .

What I've done so far:
Enabled debug:

Loading class org.bytedeco.javacpp.avutil
Loading class org.bytedeco.javacpp.avutil
Extracting jar:file:/opt/myapplication/lib/ffmpeg-3.1.2-1.2-linux-x86_64.jar!/org/bytedeco/javacpp/linux-x86_64/libavutil.so.55
Loading /tmp/javacpp28873975186486211/libavutil.so.55
Extracting jar:file:/opt/myapplication/lib/ffmpeg-3.1.2-1.2-linux-x86_64.jar!/org/bytedeco/javacpp/linux-x86_64/libjniavutil.so
Loading /tmp/javacpp28873975186486211/libjniavutil.so
Loading class org.bytedeco.javacpp.swresample
Loading class org.bytedeco.javacpp.avutil
Loading class org.bytedeco.javacpp.swresample
Extracting jar:file:/opt/myapplication/lib/ffmpeg-3.1.2-1.2-linux-x86_64.jar!/org/bytedeco/javacpp/linux-x86_64/libswresample.so.2
Loading /tmp/javacpp28873975186486211/libswresample.so.2
Extracting jar:file:/opt/myapplication/lib/ffmpeg-3.1.2-1.2-linux-x86_64.jar!/org/bytedeco/javacpp/linux-x86_64/libjniswresample.so
Loading /tmp/javacpp28873975186486211/libjniswresample.so
Loading class org.bytedeco.javacpp.avcodec
Loading class org.bytedeco.javacpp.avutil
Loading class org.bytedeco.javacpp.swresample
Loading class org.bytedeco.javacpp.avcodec
Extracting jar:file:/opt/myapplication/lib/ffmpeg-3.1.2-1.2-linux-x86_64.jar!/org/bytedeco/javacpp/linux-x86_64/libavcodec.so.57
Loading /tmp/javacpp28873975186486211/libavcodec.so.57
Extracting jar:file:/opt/myapplication/lib/ffmpeg-3.1.2-1.2-linux-x86_64.jar!/org/bytedeco/javacpp/linux-x86_64/libjniavcodec.so
Loading /tmp/javacpp28873975186486211/libjniavcodec.so
(exception occurs here)

This indicates that loading libjniavcodec.so fails.
ldd libjniavcodec.so however shows that all dependencies can be found (same output on my machine as on the deployment server)

$ ldd -v libjniavcodec.so
	linux-vdso.so.1 =>  (0x00007fff90bfe000)
	libavcodec.so.57 => /tmp/javacpp90472456501071/./libavcodec.so.57 (0x00007fc31350e000)
	libswresample.so.2 => /tmp/javacpp90472456501071/./libswresample.so.2 (0x00007fc3132f2000)
	libavutil.so.55 => /tmp/javacpp90472456501071/./libavutil.so.55 (0x00007fc31307b000)
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fc312d52000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fc312a4b000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fc312835000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fc312470000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fc312251000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fc31204d000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fc315475000)

	Version information:
	./libjniavcodec.so:
		libgcc_s.so.1 (GCC_3.0) => /lib/x86_64-linux-gnu/libgcc_s.so.1
		libavutil.so.55 (LIBAVUTIL_55) => /tmp/javacpp90472456501071/./libavutil.so.55
		libstdc++.so.6 (CXXABI_1.3) => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
		libstdc++.so.6 (GLIBCXX_3.4) => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
		libc.so.6 (GLIBC_2.14) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libc.so.6
		libavcodec.so.57 (LIBAVCODEC_57) => /tmp/javacpp90472456501071/./libavcodec.so.57
	/tmp/javacpp90472456501071/./libavcodec.so.57:
		libgcc_s.so.1 (GCC_3.0) => /lib/x86_64-linux-gnu/libgcc_s.so.1
		libdl.so.2 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libdl.so.2
		libswresample.so.2 (LIBSWRESAMPLE_2) => /tmp/javacpp90472456501071/./libswresample.so.2
		libm.so.6 (GLIBC_2.15) => /lib/x86_64-linux-gnu/libm.so.6
		libm.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libm.so.6
		libavutil.so.55 (LIBAVUTIL_55) => /tmp/javacpp90472456501071/./libavutil.so.55
		libstdc++.so.6 (CXXABI_1.3) => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
		libstdc++.so.6 (GLIBCXX_3.4) => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
		libc.so.6 (GLIBC_2.3.4) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.3) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.7) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.14) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.6) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libc.so.6
		libpthread.so.0 (GLIBC_2.3.2) => /lib/x86_64-linux-gnu/libpthread.so.0
		libpthread.so.0 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libpthread.so.0
	/tmp/javacpp90472456501071/./libswresample.so.2:
		libm.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libm.so.6
		libavutil.so.55 (LIBAVUTIL_55) => /tmp/javacpp90472456501071/./libavutil.so.55
		libc.so.6 (GLIBC_2.14) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libc.so.6
	/tmp/javacpp90472456501071/./libavutil.so.55:
		libm.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libm.so.6
		libpthread.so.0 (GLIBC_2.3.2) => /lib/x86_64-linux-gnu/libpthread.so.0
		libpthread.so.0 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libpthread.so.0
		libc.so.6 (GLIBC_2.7) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.14) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.17) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.3.4) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libc.so.6
	/usr/lib/x86_64-linux-gnu/libstdc++.so.6:
		ld-linux-x86-64.so.2 (GLIBC_2.3) => /lib64/ld-linux-x86-64.so.2
		libm.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libm.so.6
		libgcc_s.so.1 (GCC_4.2.0) => /lib/x86_64-linux-gnu/libgcc_s.so.1
		libgcc_s.so.1 (GCC_3.3) => /lib/x86_64-linux-gnu/libgcc_s.so.1
		libgcc_s.so.1 (GCC_3.0) => /lib/x86_64-linux-gnu/libgcc_s.so.1
		libc.so.6 (GLIBC_2.14) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.4) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.18) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.3.4) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.3) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.17) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.3.2) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libc.so.6
	/lib/x86_64-linux-gnu/libm.so.6:
		libc.so.6 (GLIBC_PRIVATE) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libc.so.6
	/lib/x86_64-linux-gnu/libgcc_s.so.1:
		libc.so.6 (GLIBC_2.14) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libc.so.6
	/lib/x86_64-linux-gnu/libc.so.6:
		ld-linux-x86-64.so.2 (GLIBC_2.3) => /lib64/ld-linux-x86-64.so.2
		ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2
	/lib/x86_64-linux-gnu/libpthread.so.0:
		ld-linux-x86-64.so.2 (GLIBC_2.2.5) => /lib64/ld-linux-x86-64.so.2
		ld-linux-x86-64.so.2 (GLIBC_2.3) => /lib64/ld-linux-x86-64.so.2
		ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2
		libc.so.6 (GLIBC_2.14) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.3.2) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_PRIVATE) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libc.so.6
	/lib/x86_64-linux-gnu/libdl.so.2:
		ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2
		libc.so.6 (GLIBC_PRIVATE) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libc.so.6

Also objdump shows the same information on both machines:

objdump -p libjniavcodec.so

libjniavcodec.so:     file format elf64-x86-64

Program Header:
    LOAD off    0x0000000000000000 vaddr 0x0000000000000000 paddr 0x0000000000000000 align 2**21
         filesz 0x000000000008b92d memsz 0x000000000008b92d flags r-x
    LOAD off    0x000000000008ba50 vaddr 0x000000000028ba50 paddr 0x000000000028ba50 align 2**21
         filesz 0x0000000000000bf8 memsz 0x0000000000001480 flags rw-
 DYNAMIC off    0x000000000008bd88 vaddr 0x000000000028bd88 paddr 0x000000000028bd88 align 2**3
         filesz 0x0000000000000240 memsz 0x0000000000000240 flags rw-
    NOTE off    0x00000000000001c8 vaddr 0x00000000000001c8 paddr 0x00000000000001c8 align 2**2
         filesz 0x0000000000000024 memsz 0x0000000000000024 flags r--
EH_FRAME off    0x0000000000072794 vaddr 0x0000000000072794 paddr 0x0000000000072794 align 2**2
         filesz 0x0000000000002c5c memsz 0x0000000000002c5c flags r--
   STACK off    0x0000000000000000 vaddr 0x0000000000000000 paddr 0x0000000000000000 align 2**4
         filesz 0x0000000000000000 memsz 0x0000000000000000 flags rw-
   RELRO off    0x000000000008ba50 vaddr 0x000000000028ba50 paddr 0x000000000028ba50 align 2**0
         filesz 0x00000000000005b0 memsz 0x00000000000005b0 flags r--

Dynamic Section:
  NEEDED               libavcodec.so.57
  NEEDED               libswresample.so.2
  NEEDED               libavutil.so.55
  NEEDED               libstdc++.so.6
  NEEDED               libm.so.6
  NEEDED               libgcc_s.so.1
  NEEDED               libc.so.6
  SYMBOLIC             0x0000000000000000
  RPATH                $ORIGIN/:/bytedeco/javacpp-presets/ffmpeg/cppbuild/linux-x86_64/lib/
  INIT                 0x0000000000028b20
  FINI                 0x000000000006a6dc
  INIT_ARRAY           0x000000000028ba50
  INIT_ARRAYSZ         0x0000000000000010
  FINI_ARRAY           0x000000000028ba60
  FINI_ARRAYSZ         0x0000000000000008
  GNU_HASH             0x00000000000001f0
  STRTAB               0x000000000000b478
  SYMTAB               0x0000000000002a00
  STRSZ                0x000000000001ae0d
  SYMENT               0x0000000000000018
  PLTGOT               0x000000000028c000
  PLTRELSZ             0x0000000000001248
  PLTREL               0x0000000000000007
  JMPREL               0x00000000000278d8
  RELA                 0x0000000000026ed0
  RELASZ               0x0000000000000a08
  RELAENT              0x0000000000000018
  VERNEED              0x0000000000026e10
  VERNEEDNUM           0x0000000000000005
  VERSYM               0x0000000000026286
  RELACOUNT            0x0000000000000060

Version References:
  required from libgcc_s.so.1:
    0x0b792650 0x00 07 GCC_3.0
  required from libavutil.so.55:
    0x0f818435 0x00 05 LIBAVUTIL_55
  required from libstdc++.so.6:
    0x056bafd3 0x00 06 CXXABI_1.3
    0x08922974 0x00 04 GLIBCXX_3.4
  required from libc.so.6:
    0x06969194 0x00 08 GLIBC_2.14
    0x09691a75 0x00 03 GLIBC_2.2.5
  required from libavcodec.so.57:
    0x035ff8b7 0x00 02 LIBAVCODEC_57

This list of needed libs is shown below, but all are available as they are loaded as indicated by the logs.

  NEEDED               libavcodec.so.57
  NEEDED               libswresample.so.2
  NEEDED               libavutil.so.55
  NEEDED               libstdc++.so.6
  NEEDED               libm.so.6
  NEEDED               libgcc_s.so.1
  NEEDED               libc.so.6

local:
-libstdc++.so.6
b6ef59815ddb8ecc738163e0791da622 libm.so.6
e9e5822012f0f0d32985685bf01ee8da libgcc_s.so.1
424b12d9c632b4b9f4350b3ac84caab3 libc.so.6

remote:
-libstdc++.so.6
ff1654f57d45d5350f681ce2472d3070 libm.so.6
d22bdd82e97c670c253e8c35dbfc7b72 libgcc_s.so.1
27a004526e446592fc221b2c565af16f: libc.so.6

I have no idea how to continue the investigation, does anyone have an idea on how to proceed?
@gertjanal
Copy link
Contributor Author

gertjanal commented Dec 7, 2016 •
edited
Loading

This might interesting:
http://www.oracle.com/technetwork/java/javase/crashes-137240.html#gbyzz

4.1.3 Crash due to Stack Overflow

A stack overflow in Java language code will normally result in the offending thread throwing java.lang.StackOverflowError. On the other hand, C and C++ write past the end of the stack and provoke a stack overflow. This is a fatal error which causes the process to terminate.

In the HotSpot implementation, Java methods share stack frames with C/C++ native code, namely user native code and the virtual machine itself. Java methods generate code that checks that stack space is available a fixed distance towards the end of the stack so that the native code can be called without exceeding the stack space. This distance towards the end of the stack is called “Shadow Pages.” The size of the shadow pages is between 3 and 20 pages, depending on the platform. This distance is tunable, so that applications with native code needing more than the default distance can increase the shadow page size. The option to increase shadow pages is -XX:StackShadowPages= n, where n is greater than the default stack shadow pages for the platform.

@gertjanal
Copy link
Contributor Author

gertjanal commented Dec 7, 2016 •
edited
Loading

I think I got it! While searching for -XX:StackShadowPages I came across the remote service startup script, which contains -Xss256k. When changed to -Xss512k, the error is no longer thrown 😃

@saudet saudet added the question label Dec 8, 2016
@saudet
Copy link
Member

saudet commented Dec 8, 2016

Awesome, thanks for the feedback! Might be a good idea to make a wiki page where to collect those weird errors and their solution. Everyone has write access, so feel free to create a new page.

@saudet saudet closed this as completed Dec 8, 2016
@gertjanal gertjanal changed the title [help wanted] StackOverflowError when loading .so files on Ubuntu 15.10 StackOverflowError when loading .so files Dec 8, 2016
@gertjanal
Copy link
Contributor Author

Added: https://github.com/bytedeco/javacpp-presets/wiki/Debugging-UnsatisfiedLinkError-and-StackOverflowError-on-Linux

saudet added a commit to bytedeco/javacpp that referenced this issue Nov 27, 2017
@saudet
* Move sizeof() and offsetof() data to global variables to preve…
7968093 
…nt `StackOverflowError` in `JNI_OnLoad()` (issue bytedeco/javacpp-presets#331)

 * Propagate within `Parser` type information from macros to other macros referencing them

Also fix a few more issues with `Parser`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@saudet @gertjanal