OBS crashes in libswscale when starting with changed resolution

[dbrgn]

When the canvas/output resolution is set to 1920x1080, v4l2sink works fine. But when I stop, change the canvas/output resolution to 1620x1080 and then start v4l2sink again, OBS segfaults:

        Signal: 11 (SEGV)
     Timestamp: Sun 2020-04-05 00:29:56 CEST (17s ago)
  Command Line: obs
    Executable: /usr/bin/obs
 Control Group: /user.slice/user-1000.slice/session-2.scope
          Unit: session-2.scope
         Slice: user-1000.slice
       Session: 2
     Owner UID: 1000 (user)
       Boot ID: 269f686e36dc488a99dbd078988ec827
    Machine ID: e69d99a0d54a4b27a05fb1eb0a876c83
      Hostname: x2000
       Storage: /var/lib/systemd/coredump/core.obs.1000.269f686e36dc488a99dbd078988ec827.90018.1586039396000000000000.lz4
       Message: Process 90018 (obs) of user 1000 dumped core.

                Stack trace of thread 90024:
                #0  0x00007fe06fb77f36 n/a (libswscale.so.5 + 0x57f36)

Not sure if this is related to #13.

[dbrgn]

Let me know if I should build the tool with debug symbols to get a nicer stack trace.

[ejohnsonbegin]

I have the same problems. Here the back trace:
backtrace.log
And debugger output:
Reading symbols from obs...
Reading symbols from /usr/lib64/debug//usr/bin/obs.debug...
(gdb) r
Starting program: /usr/bin/obs
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
[New Thread 0x7fffc9b13700 (LWP 21827)]
[New Thread 0x7fffbde67700 (LWP 21831)]
Attempted path: share/obs/obs-studio/locale/en-US.ini
Attempted path: /usr/share/obs/obs-studio/locale/en-US.ini
Attempted path: share/obs/obs-studio/locale.ini
Attempted path: /usr/share/obs/obs-studio/locale.ini
Attempted path: share/obs/obs-studio/locale/fr-FR.ini
Attempted path: /usr/share/obs/obs-studio/locale/fr-FR.ini
info: Using preferred locale 'fr-FR'
Attempted path: share/obs/obs-studio/themes/System.qss
Attempted path: /usr/share/obs/obs-studio/themes/System.qss
info: CPU Name: AMD Phenom(tm) II X6 1100T Processor
info: CPU Speed: 3350.034MHz
info: Physical Cores: 6, Logical Cores: 6
info: Physical Memory: 15887MB Total, 2126MB Free
info: Kernel Version: Linux 4.19.72-gentoo
info: Distribution: Gentoo Unknown
info: Window System: X11.0, Vendor: The X.Org Foundation, Version: 1.20.7
[New Thread 0x7fffb6d61700 (LWP 21832)]
info: Portable mode: false
Attempted path: share/obs/obs-studio/themes/Dark/no_sources.svg
Attempted path: /usr/share/obs/obs-studio/themes/Dark/no_sources.svg
QMetaObject::connectSlotsByName: No matching signal for on_advAudioProps_clicked()
QMetaObject::connectSlotsByName: No matching signal for on_advAudioProps_destroyed()
QMetaObject::connectSlotsByName: No matching signal for on_actionGridMode_triggered()
QMetaObject::connectSlotsByName: No matching signal for on_program_customContextMenuRequested(QPoint)
info: OBS 25.0.3 (linux)
info: ---------------------------------
info: ---------------------------------
info: audio settings reset:
samples per sec: 44100
speakers: 2
[New Thread 0x7fffb5ea5700 (LWP 21833)]
info: ---------------------------------
info: Initializing OpenGL...
[New Thread 0x7fffa1f76700 (LWP 21885)]
[New Thread 0x7fffa1634700 (LWP 21886)]
[New Thread 0x7fffa0e33700 (LWP 21887)]
[New Thread 0x7fff93fff700 (LWP 21888)]
[New Thread 0x7fff937fe700 (LWP 21889)]
info: Loading up OpenGL on adapter X.Org AMD TURKS (DRM 2.50.0 / 4.19.72-gentoo, LLVM 9.0.1)
info: OpenGL loaded successfully, version 3.3 (Core Profile) Mesa 19.3.5, shading language 3.30
info: ---------------------------------
info: video settings reset:
base resolution: 1920x1080
output resolution: 1148x646
downscale filter: Bicubic
fps: 60/1
format: NV12
YUV mode: 601/Partial
[New Thread 0x7fff92ffd700 (LWP 21890)]
info: NV12 texture support not available
[New Thread 0x7fff925dc700 (LWP 21891)]
info: Audio monitoring device:
name: Par défaut
id: default
info: ---------------------------------
warning: Failed to load 'en-US' text for module: 'decklink-ouput-ui.so'
[New Thread 0x7fff90e4c700 (LWP 21892)]
libDeckLinkAPI.so: Ne peut ouvrir le fichier d'objet partagé: Aucun fichier ou dossier de ce type
warning: A DeckLink iterator could not be created. The DeckLink drivers may not be installed
info: No blackmagic support
info: FFMPEG VAAPI supported
[New Thread 0x7fff7b127700 (LWP 21893)]
[New Thread 0x7fff7a926700 (LWP 21894)]
[Thread 0x7fff7a926700 (LWP 21894) exited]
warning: Failed to load 'fr-FR' text for module: 'v4l2sink.so'
[Thread 0x7fff7b127700 (LWP 21893) exited]
info: VLC found, VLC video source enabled
info: ---------------------------------
info: Loaded Modules:
info: vlc-video.so
info: v4l2sink.so
info: text-freetype2.so
info: rtmp-services.so
info: obs-x264.so
info: obs-transitions.so
info: obs-outputs.so
info: obs-libfdk.so
info: obs-filters.so
info: obs-ffmpeg.so
info: linux-v4l2.so
info: linux-pulseaudio.so
info: linux-decklink.so
info: linux-capture.so
info: linux-alsa.so
info: image-source.so
info: frontend-tools.so
info: decklink-ouput-ui.so
info: ---------------------------------
info: ==== Startup complete ===============================================
info: All scene data cleared
info: ------------------------------------------------
[New Thread 0x7fff7a926700 (LWP 21895)]
info: pulse-input: Server name: 'pulseaudio 13.0'
info: pulse-input: Audio format: s16le, 44100 Hz, 3 channels
info: pulse-input: Started recording from 'alsa_output.pci-0000_00_14.2.analog-surround-21.monitor'
info: pulse-input: Server name: 'pulseaudio 13.0'
info: pulse-input: Audio format: s16le, 8000 Hz, 1 channels
info: pulse-input: Started recording from 'alsa_input.usb-046d_08da-01.mono-fallback'
info: v4l2-input: Start capture from /dev/video0
info: v4l2-input: Input: 0
info: v4l2-input: Resolution: 640x480
info: v4l2-input: Pixelformat: 21UY
info: v4l2-input: Linesize: 640 Bytes
info: v4l2-input: Framerate: -nan fps
[New Thread 0x7fff7922d700 (LWP 21896)]
[New Thread 0x7fff78a2c700 (LWP 21897)]
info: Switched to scene 'Scène'
info: ------------------------------------------------
info: Loaded scenes:
info: - scene 'Scène':
info: - source: 'Périphérique de capture vidéo (V4L2)' (v4l2_input)
info: ------------------------------------------------
Attempted path: share/obs/obs-studio/images/overflow.png
Attempted path: /usr/share/obs/obs-studio/images/overflow.png
info: adding 69 milliseconds of audio buffering, total audio buffering is now 69 milliseconds (source: Mic/Aux)

info: adding 23 milliseconds of audio buffering, total audio buffering is now 92 milliseconds (source: Mic/Aux)

info: adding 23 milliseconds of audio buffering, total audio buffering is now 116 milliseconds (source: Mic/Aux)

Thread 11 "video-io: video" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fff92ffd700 (LWP 21890)]
0x00007fffec347187 in ff_nv12ToUV_sse2.loop_u () from /usr/lib64/libswscale.so.5
(gdb) bt
#0 0x00007fffec347187 in ff_nv12ToUV_sse2.loop_u () at src/libswscale/x86/rgb2rgb.c:183
#1 0x00007fffec34cd83 in deinterleaveBytes_sse2 (src=, dst1=0x7fff5f90b35e "ss", dst2=0x7fff5f93879e "\204\204", width=574, height=323, srcStride=1148, dst1Stride=574, dst2Stride=574) at src/libswscale/x86/rgb2rgb_template.c:1916
#2 0x00007fffec33f9c3 in nv12ToPlanarWrapper (c=0x555556fe8a70, src=0x7fff92ffca08, srcStride=0x7fff92ffc9e8, srcSliceY=, srcSliceH=646, dstParam=, dstStride=0x7fff92ffc9f8) at src/libswscale/swscale_unscaled.c:177
#3 0x00007fffec3376c3 in sws_scale (c=0x555556fe8a70, srcSlice=srcSlice@entry=0x7fff92ffcad0, srcStride=srcStride@entry=0x7fff92ffcb10, srcSliceY=srcSliceY@entry=0, srcSliceH=646, dst=dst@entry=0x555556de44c0, dstStride=0x555556de4500) at src/libswscale/swscale.c:986
#4 0x00007ffff4f2a789 in video_scaler_scale (scaler=, output=output@entry=0x555556de44c0, out_linesize=out_linesize@entry=0x555556de4500, input=input@entry=0x7fff92ffcad0, in_linesize=in_linesize@entry=0x7fff92ffcb10)
at /usr/src/debug/media-video/obs-studio-25.0.3/obs-studio-25.0.3/libobs/media-io/video-scaler-ffmpeg.c:184
#5 0x00007ffff4f28036 in scale_video_output (data=0x7fff92ffcad0, input=0x555556de4440) at /usr/src/debug/media-video/obs-studio-25.0.3/obs-studio-25.0.3/libobs/media-io/video-io.c:100
#6 video_output_cur_frame (video=0x5555567c8d40) at /usr/src/debug/media-video/obs-studio-25.0.3/obs-studio-25.0.3/libobs/media-io/video-io.c:140
#7 video_thread (param=0x5555567c8d40) at /usr/src/debug/media-video/obs-studio-25.0.3/obs-studio-25.0.3/libobs/media-io/video-io.c:187
#8 0x00007ffff369f268 in start_thread (arg=) at pthread_create.c:486
#9 0x00007ffff33c7e2f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

[aidangoettsch]

Interesting observation I had is that it seems to be something to do with the OBS config files. These steps got me predictable behavior.

1. Delete the OBS config (~/.config/obs-studio, this will delete scenes so you may want to make a backup if you care about them)
2. Run OBS and change the resolution of the scene
3. Add at least one source to the scene
4. Start v4l2sink, it works without crashing
5. Stop v4l2sink/close OBS, it crashes due to Obs Studio crashes when stopping the v4l2 sink #10
6. Restart OBS
7. Start v4l2sink with the same configuration, it now crashes
8. Restart OBS
9. Change the output resolution to 1920x1080
10. Start v4l2sink, it doesn't crash

I suspect it may be linked to OBS being unable to gracefully exit due to #10, I'm going to try to apply the patch for that issue and see if I can still reproduce.

[aidangoettsch]

Applying #21 to allow OBS to exit gracefully seems to make the non-crash behavior persist across multiple launches. I suspect this doesn't actually solve the underlying issue (configs being corrupted causing a crash), but it makes it usable in 99.9% of situations.

[guicaulada]

Hello, I had a problem today where OBS would crash when I started V4L2 Video Output, I can confirm that going on Settings and changing Base Canvas Resolution and Output Scaled Resolution to 1920x1080 solved the problem for me and I am now able to start V4L2 Video Output, without any problems.