segmentation fault from HandBrakeCLI #236

Closed
mhl opened this Issue Jun 26, 2016 · 30 comments

Projects

None yet

7 participants

@mhl
mhl commented Jun 26, 2016

I've been getting segmentation faults from HandbrakeCLI when ripping DVDs. I've built HandBrake with ./configure --debug=std from the master branch of this repository, in the hope of getting a useful stack trace from the core dump. The example output and logs below are for HandBrakeCLI built from 700b4f8. It was invoked from a script with:

/home/mark/hb/build/HandBrakeCLI -v2 -N eng -a 1 -i /dev/dvd -t 1 -o '/home/mark/Videos/ALL_THE_PRESIDENTS_MEN_PAL1_2016-06-26T10:29:07/title-01.mkv' --preset-import-gui --preset='Android 720p30 MP3' -s scan --subtitle-forced --subtitle-default

The output on the terminal was: https://gist.github.com/mhl/53d763e9dbc6c10ba1a27434722518cb

Examining the core file with gdb gives the following:

$ file core 
core: ELF 32-bit LSB  core file Intel 80386, version 1 (SYSV), too many program header sections (218)
$ gdb /home/mark/hb/build/HandBrakeCLI core 
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/mark/hb/build/HandBrakeCLI...done.
[New LWP 2520]
[New LWP 2504]
[New LWP 2435]
[New LWP 2418]
[New LWP 2501]
[New LWP 2425]
[New LWP 2508]
[New LWP 2500]
[New LWP 2509]
[New LWP 2514]
[New LWP 2510]
[New LWP 2511]
[New LWP 2516]
[New LWP 2512]
[New LWP 2517]
[New LWP 2513]
[New LWP 2526]
[New LWP 2522]
[New LWP 2525]
[New LWP 2515]
[New LWP 2521]
[New LWP 2519]
[New LWP 2524]
[New LWP 2505]
[New LWP 2499]
[New LWP 2523]
[New LWP 2503]
[New LWP 2498]
[New LWP 2507]
[New LWP 2502]
[New LWP 2518]
[New LWP 2506]
[New LWP 2528]
[New LWP 2527]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".
Core was generated by `/home/mark/hb/build/HandBrakeCLI -v2 -N eng -a 1 -i /dev/dvd -t 1 -o /home/mark'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  ff_vector_fmul_scalar_sse.loop () at libavutil/x86/float_dsp.asm:124
124 VECTOR_FMUL_SCALAR
(gdb) bt
#0  ff_vector_fmul_scalar_sse.loop () at libavutil/x86/float_dsp.asm:124
#1  0xb560f5e0 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
(gdb) quit

You can see my config.info.txt here: https://gist.github.com/mhl/50146fd5d470bb4ab8831449e42fe676

And config.verbose.txt here: https://gist.github.com/mhl/660099700d22dd8ff9c33d7aa5fce470

Please let me know if there's any more information I should have provided, or investigation I can do. Thank-you for all your work on HandBrake!

@cehoyos
cehoyos commented Jun 26, 2016

Is the crash only reproducible with the dvd as input or does it also happen if you first dump the dvd into a file and then try to read the file? The file can for example be created with mplayer -dumpstream dvd:// or mplayer -dumpstream dvdnav://

@mhl
mhl commented Jun 27, 2016

@cehoyos If I do:

$ mplayer -dumpstream dvd://

... and then encode with:

$ /home/mark/hb/build/HandBrakeCLI -v2 -N eng -a 1 -i stream.dump -t 1 -o '/home/mark/Videos/ALL_THE_PRESIDENTS_MEN_PAL1_2016-06-26T10:29:07/title-01-from-stream-dump.mkv' --preset-import-gui --preset='Android 720p30 MP3' -s scan --subtitle-forced --subtitle-default

It works fine, there's no segfault. However, if I dd from /dev/dvd to a file and then give that as input, I get the same segfault. i.e. I get a segfault from HandbrakeCLI if I do:

$ dd if=/dev/dvd of=atpm.iso bs=2048
$ /home/mark/hb/build/HandBrakeCLI -v2 -N eng -a 1 -i atpm.iso -t 1 -o '/home/mark/Videos/ALL_THE_PRESIDENTS_MEN_PAL1_2016-06-26T10:29:07/title-01-from-iso.mkv' --preset-import-gui --preset='Android 720p30 MP3' -s scan --subtitle-forced --subtitle-default
...
[output elided]
...
[16:50:24] Writing Metadata to output file...
Encoding: task 2 of 2, 0.00 %[16:50:24] thread aac80b40 started ("Muxer")
[16:50:24] thread aa47fb40 started ("Reader")
[16:50:24] thread a9c7eb40 started ("Audio decoder (libavcodec)")
[16:50:24] thread a947db40 started ("Video decoder (libavcodec)")
[16:50:24] thread a8b81b40 started ("Video Synchronization")
[16:50:24] thread a8380b40 started ("AVCodec Audio encoder (libavcodec)")
[16:50:24] thread a7b7fb40 started ("H.264/AVC encoder (libx264)")
[16:50:24] thread a737eb40 started ("Muxer")
[16:50:24] thread a6b7db40 started ("Comb Detect")
[16:50:24] thread a637cb40 started ("Decomb")
[16:50:24] thread a5b7bb40 started ("Framerate Shaper")
[16:50:24] thread a537ab40 started ("Crop and Scale")
[mpeg2video @ 0xb04c7e00] warning: first frame is no keyframe
[16:50:24] sync: first pts audio 0x80bd is 0
Segmentation fault (core dumped)
@jstebbins
Contributor

If you reconfigure with --debug=max, it will add debug symbols in libav. Then a backtrace may give us a better view of where the crash is.

@mhl
mhl commented Jun 27, 2016

@jstebbins I tried to do that, but the results from gdb look the same to me. Am I doing something wrong?

I did:

$ cd hb
$ rm -rf build
$ ./configure --debug=max
[...]
$ cd build
$ make -j2
[...]
$ cd
$ /home/mark/hb/build/HandBrakeCLI -v2 -N eng -a 1 -i atpm.iso -t 1 -o '/home/mark/Videos/ALL_THE_PRESIDENTS_MEN_PAL1_2016-06-26T10:29:07/title-01-from-iso.mkv' --preset-import-gui --preset='Android 720p30 MP3' -s scan --subtitle-forced --subtitle-default
[...]
Segmentation fault (core dumped)
$ gdb /home/mark/hb/build/HandBrakeCLI core
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/mark/hb/build/HandBrakeCLI...done.
[New LWP 1507]
[New LWP 1508]
[New LWP 1509]
[New LWP 1510]
[New LWP 1503]
[New LWP 1511]
[New LWP 1512]
[New LWP 1460]
[New LWP 1513]
[New LWP 1486]
[New LWP 1487]
[New LWP 1456]
[New LWP 1494]
[New LWP 1457]
[New LWP 1496]
[New LWP 1484]
[New LWP 1502]
[New LWP 1485]
[New LWP 1488]
[New LWP 1489]
[New LWP 1490]
[New LWP 1491]
[New LWP 1492]
[New LWP 1493]
[New LWP 1495]
[New LWP 1497]
[New LWP 1498]
[New LWP 1499]
[New LWP 1500]
[New LWP 1501]
[New LWP 1514]
[New LWP 1506]
[New LWP 1504]
[New LWP 1515]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".
Core was generated by `/home/mark/hb/build/HandBrakeCLI -v2 -N eng -a 1 -i atpm.iso -t 1 -o /home/mark'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  ff_vector_fmul_scalar_sse.loop () at libavutil/x86/float_dsp.asm:124
124 VECTOR_FMUL_SCALAR
(gdb) bt
#0  ff_vector_fmul_scalar_sse.loop () at libavutil/x86/float_dsp.asm:124
#1  0xb560b040 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
(gdb) 

Sorry if I'm missing something obvious. (I put the build output here: https://gist.github.com/mhl/3f4809f109e149434b96c8aa5a06f17a )

@jstebbins
Contributor

I don't think you are missing anything. My guess is that the call stack is getting completely obliterated.

It may be that libdvdread or libdvdnav is running off the rails (since encoding a pre-ripped stream works). Could you give us a backtrace of all threads? Where the other threads are may give a clue.
(gdb) thread apply all bt

@mhl
mhl commented Jun 27, 2016

(Thanks for such quick replies, by the way!)

(gdb) thread apply all bt

Thread 34 (Thread 0xa446eb40 (LWP 1515)):
#0  0xb76f6428 in __kernel_vsyscall ()
#1  0xb6f77d0b in pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_wait.S:187
#2  0x085f9cfa in worker (v=0xa629560) at libavcodec/pthread_slice.c:80
#3  0xb6f73f72 in start_thread (arg=0xa446eb40) at pthread_create.c:312
#4  0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

Thread 33 (Thread 0xa9573b40 (LWP 1504)):
#0  0xb76f6428 in __kernel_vsyscall ()
#1  0xb6f7ab12 in __lll_unlock_wake ()
    at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/lowlevellock.S:386
#2  0xb6f771c7 in _L_unlock_669 () from /lib/i386-linux-gnu/libpthread.so.0
#3  0xb6f770f4 in __pthread_mutex_unlock_usercnt (mutex=mutex@entry=0xb5603c40, decr=decr@entry=1)
    at pthread_mutex_unlock.c:57
#4  0xb6f771be in __GI___pthread_mutex_unlock (mutex=0xb5603c40) at pthread_mutex_unlock.c:310
#5  0x080ef015 in hb_unlock (l=0xb5603c40) at ../libhb/ports.c:1033
#6  0x080d4db7 in hb_fifo_get_wait (f=0xb56012a0) at ../libhb/fifo.c:1016
#7  0x08112bff in hb_work_loop (_w=0xb56042a8) at ../libhb/work.c:1854
#8  0x080ee561 in hb_thread_func (_t=0xb560b178) at ../libhb/ports.c:855
#9  0xb6f73f72 in start_thread (arg=0xa9573b40) at pthread_create.c:312
#10 0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

Thread 32 (Thread 0xa8d72b40 (LWP 1506)):
#0  0xb76f6428 in __kernel_vsyscall ()
#1  0xb6f780ed in pthread_cond_timedwait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_timedwait.S:245
#2  0x080ef178 in hb_cond_timedwait (c=0xb5603d60, lock=0xb5603d08, msec=msec@entry=200)
    at ../libhb/ports.c:1143
#3  0x080d4e00 in hb_fifo_get_wait (f=0xb5603cd0) at ../libhb/fifo.c:1000
#4  0x08112bff in hb_work_loop (_w=0xb5604308) at ../libhb/work.c:1854
#5  0x080ee561 in hb_thread_func (_t=0xb560fb78) at ../libhb/ports.c:855
#6  0xb6f73f72 in start_thread (arg=0xa8d72b40) at pthread_create.c:312
#7  0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

Thread 31 (Thread 0xa4c6fb40 (LWP 1514)):
#0  0xb76f6428 in __kernel_vsyscall ()
#1  0xb6f77d0b in pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_wait.S:187
#2  0x085f9cfa in worker (v=0xa629560) at libavcodec/pthread_slice.c:80
#3  0xb6f73f72 in start_thread (arg=0xa4c6fb40) at pthread_create.c:312
#4  0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

Thread 30 (Thread 0xaad76b40 (LWP 1501)):
#0  0xb76f6428 in __kernel_vsyscall ()
#1  0xb6f780ed in pthread_cond_timedwait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_timedwait.S:245
#2  0x080ef178 in hb_cond_timedwait (c=0xb5604210, lock=0xb56041b8, msec=msec@entry=200)
    at ../libhb/ports.c:1143
#3  0x080d4e00 in hb_fifo_get_wait (f=0xb5604180) at ../libhb/fifo.c:1000
#4  0x08112bff in hb_work_loop (_w=0xb04c7950) at ../libhb/work.c:1854
#5  0x080ee561 in hb_thread_func (_t=0xb560ffc0) at ../libhb/ports.c:855
#6  0xb6f73f72 in start_thread (arg=0xaad76b40) at pthread_create.c:312
---Type <return> to continue, or q <return> to quit---
#7  0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

Thread 29 (Thread 0xab577b40 (LWP 1500)):
#0  0xb76f6428 in __kernel_vsyscall ()
#1  0xb6f77d0b in pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_wait.S:187
#2  0xb71e4d52 in ?? () from /usr/lib/i386-linux-gnu/i686/sse2/libx264.so.142
#3  0xb6f73f72 in start_thread (arg=0xab577b40) at pthread_create.c:312
#4  0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

Thread 28 (Thread 0xae0efb40 (LWP 1499)):
#0  0xb76f6428 in __kernel_vsyscall ()
#1  0xb6f77d0b in pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_wait.S:187
#2  0xb71e565c in ?? () from /usr/lib/i386-linux-gnu/i686/sse2/libx264.so.142
#3  0xb6f73f72 in start_thread (arg=0xae0efb40) at pthread_create.c:312
#4  0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

Thread 27 (Thread 0xaf98fb40 (LWP 1498)):
#0  0xb76f6428 in __kernel_vsyscall ()
#1  0xb6f77d0b in pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_wait.S:187
#2  0xb71e565c in ?? () from /usr/lib/i386-linux-gnu/i686/sse2/libx264.so.142
#3  0xb6f73f72 in start_thread (arg=0xaf98fb40) at pthread_create.c:312
#4  0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

Thread 26 (Thread 0xb0190b40 (LWP 1497)):
#0  0xb76f6428 in __kernel_vsyscall ()
#1  0xb6f77d0b in pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_wait.S:187
#2  0xb71e565c in ?? () from /usr/lib/i386-linux-gnu/i686/sse2/libx264.so.142
#3  0xb6f73f72 in start_thread (arg=0xb0190b40) at pthread_create.c:312
#4  0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

Thread 25 (Thread 0xb18f6b40 (LWP 1495)):
#0  0xb76f6428 in __kernel_vsyscall ()
#1  0xb6f77d0b in pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_wait.S:187
#2  0x080ef095 in hb_cond_wait (c=0xb5600a10, lock=0xb56009f0) at ../libhb/ports.c:1108
#3  0x0811195a in taskset_thread_wait4start (ts=ts@entry=0xb56008d0, thr_idx=thr_idx@entry=1)
    at ../libhb/taskset.c:172
#4  0x080b7432 in yadif_decomb_filter_thread (thread_args_v=0xb5600998) at ../libhb/decomb.c:708
#5  0x080ee561 in hb_thread_func (_t=0xb5600ae0) at ../libhb/ports.c:855
#6  0xb6f73f72 in start_thread (arg=0xb18f6b40) at pthread_create.c:312
#7  0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

Thread 24 (Thread 0xb28f8b40 (LWP 1493)):
#0  0xb76f6428 in __kernel_vsyscall ()
#1  0xb6f77d0b in pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_wait.S:187
#2  0x080ef095 in hb_cond_wait (c=0xb56005d8, lock=0xb56005b8) at ../libhb/ports.c:1108
#3  0x0811195a in taskset_thread_wait4start (ts=ts@entry=0xb5602e90, thr_idx=thr_idx@entry=1)
    at ../libhb/taskset.c:172
#4  0x08119afc in mask_dilate_thread (thread_args_v=0xb5604b50) at ../libhb/comb_detect.c:683
---Type <return> to continue, or q <return> to quit---
#5  0x080ee561 in hb_thread_func (_t=0xb5600738) at ../libhb/ports.c:855
#6  0xb6f73f72 in start_thread (arg=0xb28f8b40) at pthread_create.c:312
#7  0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

Thread 23 (Thread 0xb30f9b40 (LWP 1492)):
#0  0xb76f6428 in __kernel_vsyscall ()
#1  0xb6f77d0b in pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_wait.S:187
#2  0x080ef095 in hb_cond_wait (c=0xb56005d8, lock=0xb56005b8) at ../libhb/ports.c:1108
#3  0x0811195a in taskset_thread_wait4start (ts=ts@entry=0xb5602e90, thr_idx=thr_idx@entry=0)
    at ../libhb/taskset.c:172
#4  0x08119afc in mask_dilate_thread (thread_args_v=0xb5604b30) at ../libhb/comb_detect.c:683
#5  0x080ee561 in hb_thread_func (_t=0xb5600648) at ../libhb/ports.c:855
#6  0xb6f73f72 in start_thread (arg=0xb30f9b40) at pthread_create.c:312
#7  0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

Thread 22 (Thread 0xb38fab40 (LWP 1491)):
#0  0xb76f6428 in __kernel_vsyscall ()
#1  0xb6f77d0b in pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_wait.S:187
#2  0x080ef095 in hb_cond_wait (c=0xb5604980, lock=0xb56030f0) at ../libhb/ports.c:1108
#3  0x0811195a in taskset_thread_wait4start (ts=ts@entry=0xb5602e64, thr_idx=thr_idx@entry=1)
    at ../libhb/taskset.c:172
#4  0x08119cfc in mask_erode_thread (thread_args_v=0xb5603098) at ../libhb/comb_detect.c:785
#5  0x080ee561 in hb_thread_func (_t=0xb5604ae0) at ../libhb/ports.c:855
#6  0xb6f73f72 in start_thread (arg=0xb38fab40) at pthread_create.c:312
#7  0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

Thread 21 (Thread 0xb40fbb40 (LWP 1490)):
#0  0xb76f6428 in __kernel_vsyscall ()
#1  0xb6f77d0b in pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_wait.S:187
#2  0x080ef095 in hb_cond_wait (c=0xb5604980, lock=0xb56030f0) at ../libhb/ports.c:1108
#3  0x0811195a in taskset_thread_wait4start (ts=ts@entry=0xb5602e64, thr_idx=thr_idx@entry=0)
    at ../libhb/taskset.c:172
#4  0x08119cfc in mask_erode_thread (thread_args_v=0xb5603078) at ../libhb/comb_detect.c:785
#5  0x080ee561 in hb_thread_func (_t=0xb56049f0) at ../libhb/ports.c:855
#6  0xb6f73f72 in start_thread (arg=0xb40fbb40) at pthread_create.c:312
#7  0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

Thread 20 (Thread 0xb48fcb40 (LWP 1489)):
#0  0xb76f6428 in __kernel_vsyscall ()
#1  0xb6f77d0b in pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_wait.S:187
#2  0x080ef095 in hb_cond_wait (c=0xb56103d0, lock=0xb56103b0) at ../libhb/ports.c:1108
#3  0x0811195a in taskset_thread_wait4start (ts=ts@entry=0xb5602e38, thr_idx=thr_idx@entry=1)
    at ../libhb/taskset.c:172
#4  0x08119efc in mask_filter_thread (thread_args_v=0xb5610358) at ../libhb/comb_detect.c:887
#5  0x080ee561 in hb_thread_func (_t=0xb5610480) at ../libhb/ports.c:855
#6  0xb6f73f72 in start_thread (arg=0xb48fcb40) at pthread_create.c:312
#7  0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

Thread 19 (Thread 0xb50fdb40 (LWP 1488)):
#0  0xb76f6428 in __kernel_vsyscall ()
---Type <return> to continue, or q <return> to quit---
#1  0xb6f77d0b in pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_wait.S:187
#2  0x080ef095 in hb_cond_wait (c=0xb56103d0, lock=0xb56103b0) at ../libhb/ports.c:1108
#3  0x0811195a in taskset_thread_wait4start (ts=ts@entry=0xb5602e38, thr_idx=thr_idx@entry=0)
    at ../libhb/taskset.c:172
#4  0x08119efc in mask_filter_thread (thread_args_v=0xb5610338) at ../libhb/comb_detect.c:887
#5  0x080ee561 in hb_thread_func (_t=0xb5610440) at ../libhb/ports.c:855
#6  0xb6f73f72 in start_thread (arg=0xb50fdb40) at pthread_create.c:312
#7  0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

Thread 18 (Thread 0xaf0f1b40 (LWP 1485)):
#0  0xb76f6428 in __kernel_vsyscall ()
#1  0xb6f77d0b in pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_wait.S:187
#2  0x080ef095 in hb_cond_wait (c=0xb56048b0, lock=0xb5610298) at ../libhb/ports.c:1108
#3  0x0811195a in taskset_thread_wait4start (ts=ts@entry=0xb5602de0, thr_idx=thr_idx@entry=1)
    at ../libhb/taskset.c:172
#4  0x0811a6d8 in decomb_filter_thread (thread_args_v=0xb56046f0) at ../libhb/comb_detect.c:1040
#5  0x080ee561 in hb_thread_func (_t=0xb56044f0) at ../libhb/ports.c:855
#6  0xb6f73f72 in start_thread (arg=0xaf0f1b40) at pthread_create.c:312
#7  0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

Thread 17 (Thread 0xaa575b40 (LWP 1502)):
#0  dvdread_getbits (state=0xaa575030, number_of_bits=4) at src/bitreader.c:78
#1  0x086a2127 in navRead_PCI (pci=0xb5601d58, buffer=0xa848002d "") at src/nav_read.c:125
#2  0x0867faf6 in dvdnav_decode_packet (p=0xa848002c "", nav_dsi=0xb560212b, nav_pci=0xb5601d58)
    at src/dvdnav.c:357
#3  0x08680aa7 in dvdnav_get_next_cache_block (this=0xb5601ce0, buf=0xaa575148, event=0xaa575180, 
    len=0xaa575190) at src/dvdnav.c:846
#4  0x0867fcc0 in dvdnav_get_next_block (this=0xb5601ce0, buf=0xa6299b0 "", 
    event=event@entry=0xaa575180, len=len@entry=0xaa575190) at src/dvdnav.c:460
#5  0x080be642 in hb_dvdnav_read (e=0xb5603aa0) at ../libhb/dvdnav.c:1622
#6  0x080f7e70 in reader_work (w=0xb5604470, buf_in=0xaa5752f0, buf_out=0xaa575300)
    at ../libhb/reader.c:463
#7  0x08112c2a in hb_work_loop (_w=0xb5604470) at ../libhb/work.c:1869
#8  0x080ee561 in hb_thread_func (_t=0xb03f0f50) at ../libhb/ports.c:855
#9  0xb6f73f72 in start_thread (arg=0xaa575b40) at pthread_create.c:312
#10 0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

Thread 16 (Thread 0xae8f0b40 (LWP 1484)):
#0  0xb76f6428 in __kernel_vsyscall ()
#1  0xb6f77d0b in pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_wait.S:187
#2  0x080ef095 in hb_cond_wait (c=0xb56048b0, lock=0xb5610298) at ../libhb/ports.c:1108
#3  0x0811195a in taskset_thread_wait4start (ts=ts@entry=0xb5602de0, thr_idx=thr_idx@entry=0)
    at ../libhb/taskset.c:172
#4  0x0811a6d8 in decomb_filter_thread (thread_args_v=0xb56046d0) at ../libhb/comb_detect.c:1040
#5  0x080ee561 in hb_thread_func (_t=0xb5604738) at ../libhb/ports.c:855
#6  0xb6f73f72 in start_thread (arg=0xae8f0b40) at pthread_create.c:312
#7  0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

Thread 15 (Thread 0xb10f5b40 (LWP 1496)):
#0  0xb76f6428 in __kernel_vsyscall ()
#1  0xb6f780ed in pthread_cond_timedwait@@GLIBC_2.3.2 ()
---Type <return> to continue, or q <return> to quit---
    at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_timedwait.S:245
#2  0x080ef178 in hb_cond_timedwait (c=0xb5604080, lock=0xb5604028, msec=msec@entry=200)
    at ../libhb/ports.c:1143
#3  0x080d4e00 in hb_fifo_get_wait (f=0xb5603ff0) at ../libhb/fifo.c:1000
#4  0x08112bff in hb_work_loop (_w=0xb5608ac0) at ../libhb/work.c:1854
#5  0x080ee561 in hb_thread_func (_t=0xb5608990) at ../libhb/ports.c:855
#6  0xb6f73f72 in start_thread (arg=0xb10f5b40) at pthread_create.c:312
#7  0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

Thread 14 (Thread 0xb696bb40 (LWP 1457)):
#0  0xb76f6428 in __kernel_vsyscall ()
#1  0xb6d22b96 in nanosleep () at ../sysdeps/unix/syscall-template.S:81
#2  0xb6d51c0d in usleep (useconds=50000) at ../sysdeps/unix/sysv/linux/usleep.c:32
#3  0x080ee767 in hb_snooze (delay=<optimised out>, delay@entry=50) at ../libhb/ports.c:217
#4  0x080d6846 in thread_func (_h=0xa623340) at ../libhb/hb.c:1911
#5  0x080ee561 in hb_thread_func (_t=0xa623900) at ../libhb/ports.c:855
#6  0xb6f73f72 in start_thread (arg=0xb696bb40) at pthread_create.c:312
#7  0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

Thread 13 (Thread 0xb20f7b40 (LWP 1494)):
#0  0xb76f6428 in __kernel_vsyscall ()
#1  0xb6f77d0b in pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_wait.S:187
#2  0x080ef095 in hb_cond_wait (c=0xb5600a10, lock=0xb56009f0) at ../libhb/ports.c:1108
#3  0x0811195a in taskset_thread_wait4start (ts=ts@entry=0xb56008d0, thr_idx=thr_idx@entry=0)
    at ../libhb/taskset.c:172
#4  0x080b7432 in yadif_decomb_filter_thread (thread_args_v=0xb5600978) at ../libhb/decomb.c:708
#5  0x080ee561 in hb_thread_func (_t=0xb5600a80) at ../libhb/ports.c:855
#6  0xb6f73f72 in start_thread (arg=0xb20f7b40) at pthread_create.c:312
#7  0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

Thread 12 (Thread 0xb6acd700 (LWP 1456)):
#0  0xb76f6428 in __kernel_vsyscall ()
#1  0xb6d511b1 in select () at ../sysdeps/unix/syscall-template.S:81
#2  0x080a719e in EventLoop (h=0xa623340, preset_dict=0xa624140) at ../test/test.c:307
#3  0x08094e05 in main (argc=18, argv=0xbffe7f14) at ../test/test.c:521

Thread 11 (Thread 0xad0edb40 (LWP 1487)):
#0  0xb76f6428 in __kernel_vsyscall ()
#1  0xb6f77d0b in pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_wait.S:187
#2  0x080ef095 in hb_cond_wait (c=0xb5612a70, lock=0xb5612a50) at ../libhb/ports.c:1108
#3  0x0811195a in taskset_thread_wait4start (ts=ts@entry=0xb5602e0c, thr_idx=thr_idx@entry=1)
    at ../libhb/taskset.c:172
#4  0x0811a127 in decomb_check_thread (thread_args_v=0xb56129f8) at ../libhb/comb_detect.c:990
#5  0x080ee561 in hb_thread_func (_t=0xb5612b40) at ../libhb/ports.c:855
#6  0xb6f73f72 in start_thread (arg=0xad0edb40) at pthread_create.c:312
#7  0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

Thread 10 (Thread 0xac8ecb40 (LWP 1486)):
#0  0xb76f6428 in __kernel_vsyscall ()
#1  0xb6f77d0b in pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_wait.S:187
#2  0x080ef095 in hb_cond_wait (c=0xb5612a70, lock=0xb5612a50) at ../libhb/ports.c:1108
---Type <return> to continue, or q <return> to quit---
#3  0x0811195a in taskset_thread_wait4start (ts=ts@entry=0xb5602e0c, thr_idx=thr_idx@entry=0)
    at ../libhb/taskset.c:172
#4  0x0811a127 in decomb_check_thread (thread_args_v=0xb56129d8) at ../libhb/comb_detect.c:990
#5  0x080ee561 in hb_thread_func (_t=0xb5612ae0) at ../libhb/ports.c:855
#6  0xb6f73f72 in start_thread (arg=0xac8ecb40) at pthread_create.c:312
#7  0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

Thread 9 (Thread 0xa5470b40 (LWP 1513)):
#0  0xb76f6428 in __kernel_vsyscall ()
#1  0xb6f780ed in pthread_cond_timedwait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_timedwait.S:245
#2  0x080ef178 in hb_cond_timedwait (c=0xb56034a0, lock=0xb5603448, msec=msec@entry=200)
    at ../libhb/ports.c:1143
#3  0x080d4e00 in hb_fifo_get_wait (f=0xb5603410) at ../libhb/fifo.c:1000
#4  0x08112a52 in filter_loop (_f=0xb564f1a8) at ../libhb/work.c:1927
#5  0x080ee561 in hb_thread_func (_t=0xb07a0298) at ../libhb/ports.c:855
#6  0xb6f73f72 in start_thread (arg=0xa5470b40) at pthread_create.c:312
#7  0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

Thread 8 (Thread 0xb616ab40 (LWP 1460)):
#0  0xb76f6428 in __kernel_vsyscall ()
#1  0xb6f75178 in pthread_join (threadid=2806467392, thread_return=thread_return@entry=0x0)
    at pthread_join.c:92
#2  0x080eeecf in hb_thread_close (_t=0xb5603638) at ../libhb/ports.c:922
#3  0x081152fd in do_job (job=0xb5635f88) at ../libhb/work.c:1735
#4  work_func (_work=0xa6238b0) at ../libhb/work.c:136
#5  0x080ee561 in hb_thread_func (_t=0xa7637a8) at ../libhb/ports.c:855
#6  0xb6f73f72 in start_thread (arg=0xb616ab40) at pthread_create.c:312
#7  0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

Thread 7 (Thread 0xa5c71b40 (LWP 1512)):
#0  0xb76f6428 in __kernel_vsyscall ()
#1  0xb6f780ed in pthread_cond_timedwait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_timedwait.S:245
#2  0x080ef178 in hb_cond_timedwait (c=0xb56033d8, lock=0xb5603380, msec=msec@entry=200)
    at ../libhb/ports.c:1143
#3  0x080d4e00 in hb_fifo_get_wait (f=0xb5603348) at ../libhb/fifo.c:1000
#4  0x08112a52 in filter_loop (_f=0xb564f078) at ../libhb/work.c:1927
#5  0x080ee561 in hb_thread_func (_t=0xb0780218) at ../libhb/ports.c:855
#6  0xb6f73f72 in start_thread (arg=0xa5c71b40) at pthread_create.c:312
#7  0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

Thread 6 (Thread 0xa6472b40 (LWP 1511)):
#0  0xb76f6428 in __kernel_vsyscall ()
#1  0xb6f780ed in pthread_cond_timedwait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_timedwait.S:245
#2  0x080ef178 in hb_cond_timedwait (c=0xb5603310, lock=0xb56032b8, msec=msec@entry=200)
    at ../libhb/ports.c:1143
#3  0x080d4e00 in hb_fifo_get_wait (f=0xb56043c8) at ../libhb/fifo.c:1000
#4  0x08112a52 in filter_loop (_f=0xb564efb0) at ../libhb/work.c:1927
#5  0x080ee561 in hb_thread_func (_t=0xb0760198) at ../libhb/ports.c:855
#6  0xb6f73f72 in start_thread (arg=0xa6472b40) at pthread_create.c:312
#7  0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

---Type <return> to continue, or q <return> to quit---
Thread 5 (Thread 0xa9d74b40 (LWP 1503)):
#0  0xb76f6428 in __kernel_vsyscall ()
#1  0xb6f780ed in pthread_cond_timedwait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_timedwait.S:245
#2  0x080ef178 in hb_cond_timedwait (c=0xb5603fb8, lock=0xb5603f60, msec=msec@entry=200)
    at ../libhb/ports.c:1143
#3  0x080d4e00 in hb_fifo_get_wait (f=0xb5603f28) at ../libhb/fifo.c:1000
#4  0x08112bff in hb_work_loop (_w=0xb5604248) at ../libhb/work.c:1854
#5  0x080ee561 in hb_thread_func (_t=0xb04c7b10) at ../libhb/ports.c:855
#6  0xb6f73f72 in start_thread (arg=0xa9d74b40) at pthread_create.c:312
#7  0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

Thread 4 (Thread 0xa6c73b40 (LWP 1510)):
#0  0xb76f6428 in __kernel_vsyscall ()
#1  0xb6f780ed in pthread_cond_timedwait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_timedwait.S:245
#2  0x080ef178 in hb_cond_timedwait (c=0xb5603e28, lock=0xb5603dd0, msec=msec@entry=200)
    at ../libhb/ports.c:1143
#3  0x080d4e00 in hb_fifo_get_wait (f=0xb5603d98) at ../libhb/fifo.c:1000
#4  0x08112a52 in filter_loop (_f=0xb564ecd0) at ../libhb/work.c:1927
#5  0x080ee561 in hb_thread_func (_t=0xb0740118) at ../libhb/ports.c:855
#6  0xb6f73f72 in start_thread (arg=0xa6c73b40) at pthread_create.c:312
#7  0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

Thread 3 (Thread 0xa7474b40 (LWP 1509)):
#0  0xb76f6428 in __kernel_vsyscall ()
#1  0xb6f780ed in pthread_cond_timedwait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_timedwait.S:245
#2  0x080ef178 in hb_cond_timedwait (c=0xb5603ef0, lock=0xb5603e98, msec=msec@entry=200)
    at ../libhb/ports.c:1143
#3  0x080d4e00 in hb_fifo_get_wait (f=0xb5603e60) at ../libhb/fifo.c:1000
#4  0x08112bff in hb_work_loop (_w=0xb5603600) at ../libhb/work.c:1854
#5  0x080ee561 in hb_thread_func (_t=0xb0720098) at ../libhb/ports.c:855
#6  0xb6f73f72 in start_thread (arg=0xa7474b40) at pthread_create.c:312
#7  0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

Thread 2 (Thread 0xa7c75b40 (LWP 1508)):
#0  0xb76f6428 in __kernel_vsyscall ()
#1  0xb6f780ed in pthread_cond_timedwait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_timedwait.S:245
#2  0x080ef178 in hb_cond_timedwait (c=0xb5603568, lock=0xb5603510, msec=msec@entry=200)
    at ../libhb/ports.c:1143
#3  0x080d4e00 in hb_fifo_get_wait (f=0xb56034d8) at ../libhb/fifo.c:1000
#4  0x08112bff in hb_work_loop (_w=0xb56035a0) at ../libhb/work.c:1854
#5  0x080ee561 in hb_thread_func (_t=0xb5609198) at ../libhb/ports.c:855
#6  0xb6f73f72 in start_thread (arg=0xa7c75b40) at pthread_create.c:312
#7  0xb6d58f8e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

Thread 1 (Thread 0xa8476b40 (LWP 1507)):
#0  ff_vector_fmul_scalar_sse.loop () at libavutil/x86/float_dsp.asm:124
#1  0xb560b040 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
(gdb) 
@jstebbins
Contributor

Yeah, interestingly, all threads except one that is in libdvdread are waiting on condition variables or mutexes. So points to a bug in libdvdread. Unfortunately, it's going to be a tough one to debug. It's not obvious why it would be hammering the stack. At the point it is in the call stack, it is accessing memory that is allocated from the heap.

I usually debug such memory issues by running the app under valgrind. But in order to do so, I usually have to add a bunch of ignore rules since x264 provokes a bunch of warnings and spams the output. It's a slow and monotonous process, but it usually finds this kind of thing.

@cehoyos
cehoyos commented Jun 27, 2016

Please cut the file atpm.iso, test that the cut file still allows to reproduce the crash and upload.

@mhl
mhl commented Jul 1, 2016

@jstebbins I've tried rerunning with (hopefully!) useful valgrind options - there are only a couple of errors from x264, so I haven't tried to suppress them, but I can re-run with suppressions if that's helpful. The command I used is:

valgrind --leak-check=full --show-reachable=yes --error-limit=no --gen-suppressions=all --log-file=valgrind.log /home/mark/hb/build/HandBrakeCLI -v2 -N eng -a 1 -i atpm.iso -t 1 -o '/home/mark/Videos/ALL_THE_PRESIDENTS_MEN_PAL1_2016-06-26T10:29:07/title-01.mkv' --preset-import-gui --preset='Android 720p30 MP3' -s scan --subtitle-forced --subtitle-default

The results from that are here: https://gist.github.com/mhl/4a4f41985c9fa22d8fb026895246aee5

@cehoyos I've got a cut down version of the iso which still errors for me; I'll contact you separately about that.

I realised that I should mention that the machine this is running on is slightly unusual, in that it's an Intel Mac Mini, but in the brief period where they used 32-bit processors, in case that makes a difference. (I suspect it might, since I've since tried this on a 64-bit Ubuntu trusty machine with the same ISO, same commit, and don't get the segfault, so maybe it's due to some code assuming 64bit pointers?)

@bradleysepos
Contributor

That's a helpful mention as we haven't offered 32-bit Mac builds of HandBrake for some time. I don't remember whether we officially deprecated support for compiling for 32-bit Mac, but if not it will probably be the case in the near future.

Anyway, hopefully John will have some suggestions regarding the valgrind output.

@cehoyos Cutting an ISO seems ill-advised? 😕

@bradleysepos
Contributor

@mhl To clarify further, this is a 32-bit Intel Mac mini running Linux?

@jstebbins
Contributor

Well, I was hoping to see an "Invalid write" error or something like that before the crash that would indicate if the stack is actually being smashed and who is doing it. Unfortunately, there is none. So I'm not sure what to try next here.

@mhl
mhl commented Jul 1, 2016

@bradleysepos It's a Mac Mini running Linux, yes.

@cehoyos
cehoyos commented Aug 5, 2016

Sorry for the delay: I was able to reproduce the crash with the dump mhl uploaded but the backtrace looks different here. Debugging Handbrake is non-trivial, I suggest you provide the dump to the developers here, they should be able to trace this down.

@palfrey
palfrey commented Sep 11, 2016

I've what looks like a similar failure, also on a 32-bit machine (it's a 64-bit machine, but for historical reasons it's running a 32-bit kernel). I'm using the Debian builds of 0.10.5.

Command: HandBrakeCLI -e x264 -q 19 -a 1 -E lame -B 128 -6 dpl2 -R Auto -D 0.0 -X 720 --loose-anamorphic -i /media/disk_by-id_usb-Initio_BD_ROM_BC-5500A_0000000000000000W --denoise weak --decomb --title 3 -o tracks7a41919d_4b338e11-3.mp4 --no-dvdnav

(gdb) bt
#0  0xb68919b2 in ff_vector_fmul_scalar_sse () from /usr/lib/i386-linux-gnu/libavutil.so.55
#1  0xb6d75d5a in mp3lame_encode_frame () from /usr/lib/i386-linux-gnu/libavcodec.so.57
#2  0xb6f5c629 in avcodec_encode_audio2 () from /usr/lib/i386-linux-gnu/libavcodec.so.57
#3  0x08097c13 in Encode ()
#4  0x08097d7d in encavcodecaWork ()
#5  0x0808c17a in work_loop ()
#6  0x0805c12e in hb_thread_func ()
#7  0xb600629a in start_thread (arg=0x994f8b40) at pthread_create.c:333
#8  0xb5eba1be in clone () from /lib/i386-linux-gnu/libc.so.6

Full backtrace

@sr55
Contributor
sr55 commented Sep 11, 2016

Can you please compile from source and confirm the issue still exists. The debian builds are known to be broken in many ways and we don't want to waste time tracking down another one of their issues.

@palfrey
palfrey commented Sep 11, 2016

Right now, I can give you a backtrace from a recompile from source of the Debian build (which I've got to hand before I figured out there was easier ways to get the debug symbols). On Tuesday evening I've probably got time to get a clean recompile direct from the Github sources. Is current master or the 0.10.5 tag more useful? Also, this would be a recompile from source of Handbrake, but still the existing libavcodec/libavutil library versions.

(gdb) bt
#0  0xb68919b2 in ff_vector_fmul_scalar_sse () from /usr/lib/i386-linux-gnu/libavutil.so.55
#1  0xb6d75d5a in mp3lame_encode_frame () from /usr/lib/i386-linux-gnu/libavcodec.so.57
#2  0xb6f5c629 in avcodec_encode_audio2 () from /usr/lib/i386-linux-gnu/libavcodec.so.57
#3  0x080709c7 in Encode (w=0xb3055da8) at ../libhb/encavcodecaudio.c:388
#4  0x08070b1d in encavcodecaWork (w=0xb3055da8, buf_in=0x98bfc2a4, buf_out=0x98bfc2a8) at ../libhb/encavcodecaudio.c:469
#5  0x080597bb in work_loop (_w=0xb3055da8) at ../libhb/work.c:1718
#6  0x0806e05e in hb_thread_func (_t=0xa02f9f18) at ../libhb/ports.c:738
#7  0xb600c29a in start_thread (arg=0x98bfcb40) at pthread_create.c:333
#8  0xb5ec01be in clone () from /lib/i386-linux-gnu/libc.so.6
@sr55
Contributor
sr55 commented Sep 12, 2016

current master please.

@palfrey
palfrey commented Sep 13, 2016

So, with the same set of build deps as Debian's 0.10.5 + libopus-dev and libjansson-dev, but running off master, I get the following

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xa9bfdb40 (LWP 18776)]
ff_vector_fmul_scalar_sse.loop () at libavutil/x86/float_dsp.asm:124
124     VECTOR_FMUL_SCALAR
(gdb) bt
#0  ff_vector_fmul_scalar_sse.loop () at libavutil/x86/float_dsp.asm:124
#1  0xb5a1aca0 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

thread all apply bt full is at https://gist.github.com/palfrey/5012c242958f1249ad168d9fc4b9eab1

Configure was ./configure --prefix=/usr --build build --debug=std --disable-fdk-aac --enable-x265 --disable-gtk-update-checks (as per Debian packages, because I knew that'd work)

Command line was ./HandBrakeCLI -e x264 -q 19 -a 1 -E lame -B 128 -6 dpl2 -R Auto -D 0.0 -X 720 --loose-anamorphic -i /media/disk_by-id_usb-Initio_BD_ROM_BC-5500A_0000000000000000W --denoise weak --decomb --title 3 -o ~/dvds/tracks7a41919d_4b338e11-3.mp4 --no-dvdnav

@sr55 sr55 added the bug label Sep 13, 2016
@blairfancy

Hi i got similar issue on Ubuntu 16..4 and latest version of handbrake-cli, any workarround?

@sr55
Contributor
sr55 commented Sep 15, 2016

@blairfancy Can you also provide a log so we can confirm that it is infact the same problem please

@bradleysepos
Contributor

I'm curious whether the following patch makes any difference.

diff --git a/contrib/ffmpeg/module.defs b/contrib/ffmpeg/module.defs
index 6e13442..0e92a8f 100644
--- a/contrib/ffmpeg/module.defs
+++ b/contrib/ffmpeg/module.defs
@@ -128,6 +128,10 @@ ifeq (darwin-i386,$(BUILD.system)-$(BUILD.machine))
     ifneq (none,$(FFMPEG.GCC.g))
         FFMPEG.GCC.args.extra += -fomit-frame-pointer
     endif
+else
+    ifneq (,$(filter $(BUILD.machine),i386 i486 i586 i686))
+        FFMPEG.GCC.args.extra += -fomit-frame-pointer
+    endif
 endif

 ## For POSIX.1-2001

@palfrey
palfrey commented Sep 15, 2016

@bradleysepos Made that patch, ran make, nothing happens. What needs to go away, or is it just plain "make clean"?

@bradleysepos
Contributor

make ffmpeg.xclean && make

@palfrey
palfrey commented Sep 15, 2016

Did that, sadly exactly the same failure (certainly from a bt command in gdb perspective)

@bradleysepos
Contributor

Okay. At least we know it's not running out of registers due to useless pointers.

@palfrey
palfrey commented Nov 13, 2016

Just retried with current git master, still seeing exactly the same results.

@sr55
Contributor
sr55 commented Dec 18, 2016

Maybe we can re-test this now that we've bumped to libav12?

@palfrey
palfrey commented Dec 18, 2016

I can't any more, as I've upgraded my machine from 32 to 64-bit

@bradleysepos
Contributor

Closing. @mhl if you can test this with HandBrake 1.0.0 and report back, please do so.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment