App crash when using detect-threshold

[mrobe]

I may be doing this wrong, but when I add "detect-threshold" to the command line, I get the following error:

Traceback (most recent call last):
  File "/usr/local/bin/scenedetect", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.6/site-packages/scenedetect/__main__.py", line 58, in main
    cli.main(obj=cli_ctx)   # Parse CLI arguments with registered callbacks.
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 727, in main
    ctx.exit()
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 351, in __exit__
    self.close()
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 440, in close
    cb()
  File "/usr/local/lib/python3.6/site-packages/scenedetect/cli/context.py", line 342, in process_input
    show_progress=not self.quiet_mode)
  File "/usr/local/lib/python3.6/site-packages/scenedetect/scene_manager.py", line 469, in detect_scenes
    self._process_frame(self._num_frames + start_frame, frame_im)
  File "/usr/local/lib/python3.6/site-packages/scenedetect/scene_manager.py", line 366, in _process_frame
    self._add_cuts(detector.process_frame(frame_num, frame_im))
  File "/usr/local/lib/python3.6/site-packages/scenedetect/detectors/threshold_detector.py", line 177, in process_frame
    frame_avg = compute_frame_average(frame_img)
  File "/usr/local/lib/python3.6/site-packages/scenedetect/detectors/threshold_detector.py", line 54, in compute_frame_average
    frame.shape[0] * frame.shape[1] * frame.shape[2])
AttributeError: 'NoneType' object has no attribute 'shape'

Command line:

scenedetect --input D-2019-2.mp4 --output D-2019-2 --stats D-stats.csv detect-content list-scenes save-images detect-threshold

Computing Environment:

• OS: macOS 10.13.6
• Python Version: 3.6.5
• OpenCV Version: 4.1.1

Additional

I tried following the instructions here:

https://pyscenedetect.readthedocs.io/en/latest/examples/usage-example/#finding-optimal-thresholdsensitivity-value

But it seems like something is missing. How do I determine "an optimal value"? I.e., which file should I be looking at for this, and what am I supposed to look for in that file?

[Breakthrough]

At first glance this looks like a bug, however I noticed in your command you specified both detect-content as well as detect-threshold. Was this intentional, and if you specify just one detector, does it resolve the issue?

Thank you for your submission, will triage/look into this further whenever I get some spare time!

[mrobe]

Adding both detect-content and detect-threshold wasn't intentional. I don't understand the documentation, or what I should be doing exactly to identify the threshold. I tried again without detect-content and it works without crashing.

Again, I don't follow the documentation on how I'm to determine an optimum value for the threshold. Any tips would be appreciated.

[Breakthrough]

Hi @mrobe;

To identify the exact threshold required, you can plot the values you obtain in the stats-file when running detect-content. There is no automatic way to do this right now other than manual analysis of the statsfile (the default sensitivity of 30 works for most videos however). You would then specify the desired threshold as an argument to detect-content:

scenedetect --input D-2019-2.mp4 --output D-2019-2 --stats D-stats.csv detect-content --threshold VALUE list-scenes save-images

detect-threshold is a separate detection algorithm which uses a thresholding operation, which also has an associated threshold argument (there is usually a threshold/sensitivity value for each detect- command separately).

That being said, your original command line argument list looks like it indeed does trigger a bug, so I will keep this open for eventually fixing that (or providing a more appropriate error message to the user).

[mrobe]

Hi @Breakthrough,

I've looked at the stats file and I see the delta_rgb value, but I have no idea how am I supposed to do "manual analysis" as you suggest. Am I looking for an average of all these values? Minimum? Maximum? Etc.

I could write a simple app (or probably use Excel) to do a summation of the values, but I need some tips on what I should do to compute the threshold argument.

[Breakthrough]

@mrobe unfortunately I have not been successful in such a development as of yet, which is why automatic threshold determination is still marked as a possible long term project goal on the features page. If you could do so with an algorithm that works on all input videos of all types, I would love to see what you come up with.

I don't mean to sound discouraging, but that is a non-trivial problem that I would like a robust solution for that works on a wide variety of input test cases. If you would like to discuss this further, or have any solutions that you would like to propose, please feel free to post them here.

The best I've seen so far was referenced here in #53, with a threshold of T = μ + ασ, where α = 5 (I was unable to get a response from the author on why the value of 5 for alpha was selected, nor was it suitable for all ad-hoc test cases I was doing). This also did not seem to work well with the output of detect-content versus using a histogram for some reason, but haven't had time to look further into that aspect for a while.

This has also been suggested in #35 as a possible solution to the camera flash question, but again, no actual algorithm, design, or implementation has materialized as of yet in terms of automatic threshold determination. A solution for the camera flash question was proposed in this comment, but still requires manual adjustment by the end user for optimal performance on all input videos (it is still a significant improvement in detection accuracy, but does not directly deal with automatic parameter determination).

Thank you.

[rjhornsby]

I'm running into the same issue ...AttributeError: 'NoneType' object has no attribute 'shape' when trying to use detect-threshold like so:

$ scenedetect --input 2020W11_KON_Wisdom_2_Message.mp4 -s dt_stats detect-threshold -m 800 list-scenes

However, it seems like the problem is somehow related to loading the cached stats in this mode. If dt_stats does not exist, or I don't specify a stats file, it runs fine. When it fails, it doesn't matter if the stats file was generated by a previous detect-threshold run, or a detect-content run.

As expected, the stats file generated by each mode, for a given video file, are both generated fine and are identical.

It looks like the same problem as the OP, but there are some different line numbers so posting the stack trace here for completeness:

$ scenedetect --input 2020W11_KON_Wisdom_2_Message.mp4 -s dt_stats detect-threshold -m 800 list-scenes
[PySceneDetect] Loaded 1 video, framerate: 29.97 FPS, resolution: 1280 x 720
[PySceneDetect] Downscale factor set to 5, effective resolution: 256 x 144
[PySceneDetect] Loading frame metrics from stats file: dt_stats
[PySceneDetect] Loaded 1 metrics for 146491 frames.
[PySceneDetect] Scene list CSV file name format:
  $VIDEO_NAME-Scenes.csv
[PySceneDetect] Detecting scenes...
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 727, in main
    ctx.exit()
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 504, in exit
    raise Exit(code)
click.exceptions.Exit: 0

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/scenedetect", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/site-packages/scenedetect/__main__.py", line 58, in main
    cli.main(obj=cli_ctx)   # Parse CLI arguments with registered callbacks.
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 727, in main
    ctx.exit()
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 351, in __exit__
    self.close()
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 440, in close
    cb()
  File "/usr/local/lib/python3.7/site-packages/scenedetect/cli/context.py", line 342, in process_input
    show_progress=not self.quiet_mode)
  File "/usr/local/lib/python3.7/site-packages/scenedetect/scene_manager.py", line 469, in detect_scenes
    self._process_frame(self._num_frames + start_frame, frame_im)
  File "/usr/local/lib/python3.7/site-packages/scenedetect/scene_manager.py", line 366, in _process_frame
    self._add_cuts(detector.process_frame(frame_num, frame_im))
  File "/usr/local/lib/python3.7/site-packages/scenedetect/detectors/threshold_detector.py", line 202, in process_frame
    if self.frame_under_threshold(frame_img):
  File "/usr/local/lib/python3.7/site-packages/scenedetect/detectors/threshold_detector.py", line 129, in frame_under_threshold
    num_pixel_values = float(frame.shape[0] * frame.shape[1] * frame.shape[2])
AttributeError: 'NoneType' object has no attribute 'shape'

[Breakthrough]

Sorry this one seems to have slipped past my radar - do you know if this is still happening on v0.5.3? My apologies for the lack of progress on this one. Will make sure this makes it in for the next release.

[sksavant]

I can confirm that the same bug is observed in v0.5.3 when using detect-threshold and run using the cached stats file.

[Breakthrough]

This seems to be a very long standing issue due to how the actual detection logic was refactored to improve performance. Unfortunately, using a statsfile with detect-threshold will be slightly slower for now (~5% slowdown), but is still useful for determining an approximate threshold.

This is because the average itself is not used for threshold calculations, but rather, summation of the amount of pixels that meet the target threshold based on the minimum pixel percentage. The performance issue will be fixed in #178. Pushing a fix to the v0.5.4 branch now.