fix(process_video): deadlock on callback exception + improve shutdown robustness #2102
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
The updated
process_videofunction implements a three-stage threaded pipeline consisting of a reader, a main-thread processor, and a writer, while using bounded queues to strictly control memory consumption during video processing.When the user-supplied callback raises an exception, the function logs the error along with the affected frame index and, by default, re-raises it after performing a clean shutdown of all components.
With the optional
skip_on_error=Trueparameter, problematic frames are silently discarded instead of interrupting the entire job, which is especially useful for noisy or partially corrupted videos.The reader thread runs as non-daemon so that
cv2.VideoCapture.release()is reliably called even during early termination, preventing file handle or resource leaks.Shutdown is managed through a combination of a
stop_event, non-blocking queue operations with small timeouts, and a light sleep-based backoff that avoids both deadlocks and excessive CPU usage.The writer thread is kept as a daemon because disk writes can usually be safely interrupted, while both threads receive sensible join timeouts with logging warnings if they fail to complete in time.
Type of Change
🐛 Bug fix (non-breaking change which fixes an issue)
Motivation and Context
This change fixes a critical bug in the
process_videofunction where an unhandled exception thrown inside the user-providedcallbackwould cause the entire processing pipeline to deadlock indefinitely.The main thread would exit its processing loop but then block forever on
reader_worker.join()while the reader thread (a daemon) was stuck trying toput()into a fullframe_read_queue.Testing