Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Segfault on container.close() with debugpy #1070

Closed
5 of 6 tasks
king-cony opened this issue Jan 5, 2023 · 5 comments
Closed
5 of 6 tasks

Segfault on container.close() with debugpy #1070

king-cony opened this issue Jan 5, 2023 · 5 comments
Labels
bug invalid stale

Comments

@king-cony
Copy link

king-cony commented Jan 5, 2023 •
edited

Overview

Hi all. I'm experiencing some weird behavior when closing an InputContainer. When debugging with debugpy, I'll get a segmentation fault if I call container.close() or let the object go out of scope with a context manager. I've encountered this issue on different machines, OSes, vms, and it only seems to happen while debugging and after accessing data from a Stream object.

Expected behavior

InputContainer objects should be garbage collected with no segfault

Actual behavior

A SIGSEGV signal is emitted when an InputContainer is closed

Traceback:

============================= test session starts ==============================
platform linux -- Python 3.10.6, pytest-7.2.0, pluggy-1.0.0
rootdir: /home/vm/Desktop/pyav-test
collected 1 item

test_av.py
Fatal Python error: Segmentation fault

Thread 0x00007ff86e5fb640 (most recent call first):
  File "/usr/lib/python3.10/threading.py", line 324 in wait
  File "/usr/lib/python3.10/threading.py", line 607 in wait
  File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 261 in _on_run
  File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_daemon_thread.py", line 49 in run
  File "/usr/lib/python3.10/threading.py", line 1016 in _bootstrap_inner
  File "/usr/lib/python3.10/threading.py", line 973 in _bootstrap

Thread 0x00007ff86edfc640 (most recent call first):
  File "/usr/lib/python3.10/threading.py", line 324 in wait
  File "/usr/lib/python3.10/threading.py", line 607 in wait
  File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 215 in _on_run
  File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_daemon_thread.py", line 49 in run
  File "/usr/lib/python3.10/threading.py", line 1016 in _bootstrap_inner
  File "/usr/lib/python3.10/threading.py", line 973 in _bootstrap

Thread 0x00007ff86f5fd640 (most recent call first):
  File "/usr/lib/python3.10/threading.py", line 320 in wait
  File "/usr/lib/python3.10/threading.py", line 607 in wait
  File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_timeout.py", line 43 in _on_run
  File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_daemon_thread.py", line 49 in run
  File "/usr/lib/python3.10/threading.py", line 1016 in _bootstrap_inner
  File "/usr/lib/python3.10/threading.py", line 973 in _bootstrap
Thread 0x00007ff86fdfe640 (most recent call first):
  File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_comm.py", line 204 in _read_line
  File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_comm.py", line 222 in _on_run
  File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_daemon_thread.py", line 49 in run
  File "/usr/lib/python3.10/threading.py", line 1016 in _bootstrap_inner
  File "/usr/lib/python3.10/threading.py", line 973 in _bootstrap

Thread 0x00007ff8705ff640 (most recent call first):
  File "/usr/lib/python3.10/threading.py", line 324 in wait
  File "/usr/lib/python3.10/queue.py", line 180 in get
  File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_comm.py", line 367 in _on_run
  File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_daemon_thread.py", line 49 in run
  File "/usr/lib/python3.10/threading.py", line 1016 in _bootstrap_inner
  File "/usr/lib/python3.10/threading.py", line 973 in _bootstrap

Current thread 0x00007ff872668000 (most recent call first):
  File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_safe_repr.py", line 128 in _is_long_iter
  File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_safe_repr.py", line 112 in _repr
  File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_safe_repr.py", line 196 in _repr_iter
  File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_safe_repr.py", line 77 in __call__
  File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_xml.py", line 315 in get_variable_details
  File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_suspended_frames.py", line 51 in get_var_data
  File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_comm.py", line 753 in internal_get_variable_json
  File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_constants.py", line 475 in new_func
  File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_comm.py", line 527 in do_it
  File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 1772 in process_internal_commands
  File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 2097 in _do_wait_suspend
  File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 2062 in do_wait_suspend
  File "/home/vm/Desktop/pyav-test/test_av.py", line 11 in test_av
  File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/_pytest/python.py", line 195 in pytest_pyfunc_call
  File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/_pytest/python.py", line 1789 in runtest
  File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/_pytest/runner.py", line 167 in pytest_runtest_call
  File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/_pytest/runner.py", line 260 in <lambda>
  File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/_pytest/runner.py", line 339 in from_call
  File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/_pytest/runner.py", line 259 in call_runtest_hook
  File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/_pytest/runner.py", line 220 in call_and_report
  File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/_pytest/runner.py", line 131 in runtestprotocol
  File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/_pytest/runner.py", line 112 in pytest_runtest_protocol
  File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/_pytest/main.py", line 349 in pytest_runtestloop
  File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/_pytest/main.py", line 324 in _main
  File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/_pytest/main.py", line 270 in wrap_session
  File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/_pytest/main.py", line 317 in pytest_cmdline_main
  File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/_pytest/config/__init__.py", line 167 in main
  File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/testlauncher.py", line 36 in run
  File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/testlauncher.py", line 44 in <module>
  File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 124 in _run_code
  File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 135 in _run_module_code
  File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 321 in run_path
  File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 284 in run_file
  File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 430 in main
  File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/__main__.py", line 39 in <module>
  File "/usr/lib/python3.10/runpy.py", line 86 in _run_code
  File "/usr/lib/python3.10/runpy.py", line 196 in _run_module_as_main

Extension modules: _pydevd_bundle.pydevd_cython, _pydevd_frame_eval.pydevd_frame_evaluator, av._core, av.logging, av.bytesource, av.buffer, av.audio.format, av.enum, av.error, av.utils, av.option, av.descriptor, av.container.pyio, av.dictionary, av.format, av.stream, av.container.streams, av.sidedata.motionvectors, av.sidedata.sidedata, av.packet, av.container.input, av.container.output, av.container.core, av.codec.context, av.video.format, av.video.reformatter, av.plane, av.video.plane, av.video.frame, av.video.stream, av.codec.codec, av.frame, av.audio.layout, av.audio.plane, av.audio.frame, av.audio.stream, av.audio.fifo, av.filter.pad, av.filter.link, av.filter.context, av.filter.graph, av.filter.filter, av.audio.resampler, av.video.codeccontext (total: 44)

Investigation

I've been able to reproduce this behavior on the following platforms

  • Windows 11 22H2
  • Ubuntu 20.04 (WSL2)
  • Ubuntu 22.04 (VM)
  • MacOS Ventura 13.0.1

Each segfault happens after closing an InputContainer after accessing any of its Stream objects

Reproduction

The following code should reproduce the error

import av
import av.datasets

content = av.datasets.curated("pexels/time-lapse-video-of-night-sky-857195.mp4")
input_container = av.open(content)
video_stream = input_container.streams.video[0]
total_frames = video_stream.frames

input_container.close() # <--- segfault happens here

print(total_frames)

or

import av
import av.datasets

content = av.datasets.curated("pexels/time-lapse-video-of-night-sky-857195.mp4")
with av.open(content) as input_container:
    video_stream = input_container.streams.video[0]
    total_frames = video_stream.frames
    # segfault happens as this goes out of scope

print(total_frames)

Versions

  • OS: Ubuntu 20.04 (WSL2)
  • PyAV runtime:
PyAV v10.0.0
library configuration: --disable-static --enable-shared --libdir=/tmp/vendor/lib --prefix=/tmp/vendor --disable-alsa --disable-doc --disable-mediafoundation --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-lzma --enable-version3 --enable-zlib
library license: GPL version 3 or later
libavcodec     59. 37.100
libavdevice    59.  7.100
libavfilter     8. 44.100
libavformat    59. 27.100
libavutil      57. 28.100
libswresample   4.  7.100
libswscale      6.  7.100
  • PyAV build:
pip installed
  • FFmpeg:
ffmpeg version 4.2.7-0ubuntu0.1 Copyright (c) 2000-2022 the FFmpeg developers
built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.1)
configuration: --prefix=/usr --extra-version=0ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
libavutil      56. 31.100 / 56. 31.100
libavcodec     58. 54.100 / 58. 54.100
libavformat    58. 29.100 / 58. 29.100
libavdevice    58.  8.100 / 58.  8.100
libavfilter     7. 57.100 /  7. 57.100
libavresample   4.  0.  0 /  4.  0.  0
libswscale      5.  5.100 /  5.  5.100
libswresample   3.  5.100 /  3.  5.100
libpostproc    55.  5.100 / 55.  5.100

Research

I have done the following:

Additional context

Strangely enough, I've only encountered this issue while debugging. The code runs fine if no debugger is attached.
@king-cony king-cony added the bug label Jan 5, 2023
@jlaine
Copy link
Collaborator

jlaine commented Jan 30, 2023

I've tried to run your example, but I don't even get as far as the incriminated line, because video_stream contains a list of streams, not a stream. So the following line:

 total_frames = video_stream.frames

blows up with AttributeError: 'list' object has no attribute 'frames'

@jlaine jlaine added the invalid label Jan 30, 2023
@king-cony
Copy link
Author

king-cony commented Jan 31, 2023 •
edited

I've tried to run your example, but I don't even get as far as the incriminated line, because video_stream contains a list of streams, not a stream. So the following line:

 total_frames = video_stream.frames

blows up with AttributeError: 'list' object has no attribute 'frames'

Whoops, small error. I assumed that streams.get(video=0) and streams.video[0] were analogous, I've edited my snippet.

@jlaine
Copy link
Collaborator

jlaine commented Jan 31, 2023

I've not been able to reproduce the crash (I'm on Debian/bookworm):

(env) me@mycomputer:~/Development/pyav-org/PyAV$ python -m debugpy --listen localhost:5678 test_av.py 
0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
168

@github-actions
Copy link

github-actions bot commented Jun 1, 2023

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale label Jun 1, 2023
This was referenced Jun 26, 2023
Closed
Closed
@gepz
Copy link

gepz commented Aug 13, 2023

I encountered a very similar issue while debugging in Jupyter (or in Jupyter code cells within VSCode). I believe this might be due to Jupyter's internal use of debugpy.
Specifically, when I set a breakpoint on any line after the input is closed, Jupyter reports a kernel crash when the debugger pauses at that line.

Here's a simple code snippet to reproduce the problem:

# %%
import av

with av.open("input.mp4") as input:
    stream = input.streams.video[0]

x = 5 # set breakpoint here

This issue might be related to #1337

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug invalid stale
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jlaine @king-cony @gepz