Extra frames sometimes appear at the end of split videos

[typoman]

First of all thank you for such a great tool! It seems that the precise flag (-p or --precise) has been changed to preset and does not work anymore although it's still written in the readme of the repo. I used the content detect option but still the cuts are not precise and there are some frames that overlap with the previous scene. Is there any way to cut the scenes more precisely?

[Breakthrough]

Hi @typoman;

The default behaviour of PySceneDetect now is what precise mode was before. Scenes should be split accurately and re-encoded by ffmpeg when using the split-video command. To disable this and go back to the old behaviour, supply the -c/--copy flag to the split-video command.

Are you seeing a different/unexpected result? How many frames are overlapping? Could you provide some more details, like the complete command/example output?

Thanks!

[Breakthrough]

Hi @typoman;

Were you able to validate that frame cuts were correct or not with the new version? I forgot to mention, you also need to have ffmpeg installed for precise cutting to work.

Thanks!

[typoman]

Hi @Breakthrough,

I had installed ffmpeg when I was doing the tests. I'm sorry for the delayed answer but since I was in a hurry I just used another solution to split the videos manually. My config is OS X 10.13.6 and I think I used default flags. In the split videos, I saw frames from the previous scene just for the first or two frames. The codec was h264 and the format was mp4. If you want I can perform a test on a video (or a link) you provide and send you back what I get then you can compare results?

[Breakthrough]

Hi @typoman;

You can try with the goldeneye.mp4 file from here:
https://github.com/Breakthrough/PySceneDetect/tree/resources/tests/goldeneye

It should match the expected output on the examples page on readthedocs.

Would you be able to share with me one of the files where you're experiencing this behaviour? Is the video fixed-framerate or variable framerate? (One thing PySceneDetect does not deal with is varaible framerate videos...)

Thank you!

[typoman]

I can't share my files but I encounter the same issue with your sample. The command I used:

scenedetect --input goldeneye.mp4 detect-content split-video

When I open the first scene in QuickTime it contains two frames of the the next scene at the end. Here is the result of all scenes:

Archive.zip

[Breakthrough]

@typoman can you also provide all other information as per this link (e.g. operating system, output log in debug mode)?

You can copy and paste the template into a comment here, or attach the relevant information to the first post.

Thank you.

[typoman]

Bug/Issue Description:
A clear and concise description of what the bug or issue is - in other words, what the unexpected behavior or output is.

Imprecise cutting (as mentioned above)

Required Information:
Provide the following information to assist with reporting the bug:

scenedetect -v debug -l BUG_REPORT.txt --input goldeneye.mp4 detect-content split-video

Output:

ffmpeg version 3.4.2 Copyright (c) 2000-2018 the FFmpeg developers
  built with Apple LLVM version 9.0.0 (clang-900.0.39.2)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/3.4.2 --enable-shared --enable-pthreads --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --disable-jack --enable-gpl --enable-libmp3lame --enable-libx264 --enable-libxvid --enable-opencl --enable-videotoolbox --disable-lzma
  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libavresample   3.  7.  0 /  3.  7.  0
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
  libpostproc    54.  7.100 / 54.  7.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/Users/bahmaneslami/Downloads/goldeneye.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2016-02-03T04:12:42.000000Z
  Duration: 00:01:22.59, start: 0.000000, bitrate: 1533 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1280x544 [SAR 1:1 DAR 40:17], 1338 kb/s, 23.98 fps, 23.98 tbr, 24k tbn, 47.95 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 191 kb/s (default)
    Metadata:
      creation_time   : 2016-02-03T04:12:42.000000Z
      handler_name    : IsoMedia File Produced by Google, 5-11-2011
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
[libx264 @ 0x7fe04c810a00] using SAR=1/1
[libx264 @ 0x7fe04c810a00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 @ 0x7fe04c810a00] profile High, level 3.1
[libx264 @ 0x7fe04c810a00] 264 - core 152 r2854 e9a5903 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=2 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=12 lookahead_threads=4 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=1 keyint=250 keyint_min=23 scenecut=40 intra_refresh=0 rc_lookahead=10 rc=crf mbtree=1 crf=22.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'goldeneye-Scene-001.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    encoder         : Lavf57.83.100
    Stream #0:0(und): Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 1280x544 [SAR 1:1 DAR 40:17], q=-1--1, 23.98 fps, 24k tbn, 23.98 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc57.107.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 191 kb/s (default)
    Metadata:
      creation_time   : 2016-02-03T04:12:42.000000Z
      handler_name    : IsoMedia File Produced by Google, 5-11-2011
frame=   91 fps=0.0 q=-1.0 Lsize=     448kB time=00:00:03.73 bitrate= 982.6kbits/s speed=8.14x    
video:356kB audio:88kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.938643%
[libx264 @ 0x7fe04c810a00] frame I:1     Avg QP:19.18  size: 28013
[libx264 @ 0x7fe04c810a00] frame P:24    Avg QP:19.86  size:  8254
[libx264 @ 0x7fe04c810a00] frame B:66    Avg QP:21.60  size:  2085
[libx264 @ 0x7fe04c810a00] consecutive B-frames:  3.3%  0.0%  0.0% 96.7%
[libx264 @ 0x7fe04c810a00] mb I  I16..4: 19.5% 58.6% 21.9%
[libx264 @ 0x7fe04c810a00] mb P  I16..4: 12.5% 10.1%  0.5%  P16..4: 30.0% 11.2%  4.7%  0.0%  0.0%    skip:30.9%
[libx264 @ 0x7fe04c810a00] mb B  I16..4:  1.6%  1.3%  0.0%  B16..8: 14.2%  3.0%  0.1%  direct: 7.8%  skip:72.0%  L0:45.5% L1:45.1% BI: 9.4%
[libx264 @ 0x7fe04c810a00] 8x8 transform intra:45.7% inter:48.6%
[libx264 @ 0x7fe04c810a00] coded y,uvDC,uvAC intra: 31.0% 57.2% 8.9% inter: 3.4% 17.8% 0.1%
[libx264 @ 0x7fe04c810a00] i16 v,h,dc,p: 45% 40% 13%  3%
[libx264 @ 0x7fe04c810a00] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 28% 29% 35%  1%  1%  1%  1%  1%  2%
[libx264 @ 0x7fe04c810a00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 27% 39%  8%  4%  5%  5%  4%  5%  4%
[libx264 @ 0x7fe04c810a00] i8c dc,h,v,p: 41% 33% 23%  3%
[libx264 @ 0x7fe04c810a00] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0x7fe04c810a00] kb/s:766.73
 bahmaneslami@B-M-E  ~/Downloads   master ?                                                                                                              ✔  1383  17:09:51

2. Add -v debug -l BUG_REPORT.txt to the beginning of the command, then re-run PySceneDetect and attach the generated BUG_REPORT.txt file.

Expected Behavior:
A clear and concise description of what you expected to happen.

Precise cutting!

Computing Environment:

• OSX: 10.13.6 BuildVersion: 17G6030
• Python Version: 3.7.3
• OpenCV Version: 4.0.0

BUG_REPORT.txt

[Breakthrough]

Hi @typoman;

I've submitted an updated version of PySceneDetect, v0.5.1, which includes various bugfixes. My apologies however, but I'm having trouble replicating the behavior you're seeing (I'm only able to test on Windows/Linux). Can you double-check if this issue is fixed in v0.5.1 on OSX?

Thank you!

Edit: Would you also be able to provide the output scene list, in .csv format, for goldeneye.mp4? I would like to compare the detected cut positions to those in the resources branch.

[typoman]

The result is bigger than it could be uploaded here, so here is a temporary link:
https://we.tl/t-6CAqCUzg0A

If there is something missing, please tell me what command to use and I will send you the result.

[Breakthrough]

Hi @typoman;

Thanks for the response. Am looking at the files now, and they seem correct - are there any files in particular where you're noticing the 1-2 frames at the beginning, or is it in every file? (I'm just looking at them visually, but to double check, I'll run them through ffmpeg to export each frame just to be sure.)

Can you add the list-scenes option at the end of your command line, and attach the goldeneye-Scenes.csv file it generates?

This will help me to narrow down where this issue is being caused. Right now I think it might have to do with either timecode misalignment between PySceneDetect and ffmpeg, or ffmpeg is including extra frames at the beginning due to differences in seeking between versions (might affect I/B/P frame seeks). Once you provide the goldeneye-Scenes.csv file, I may also ask you to run some additional commands with ffmpeg to make sure everything is working as expected there.

Thank you for your help!

[typoman]

I see a frame from the next scene in the last frames of the file "goldeneye-Scene-001.mp4". I use QuickTime to check the files. What app do you use to see the frames?

[Breakthrough]

@typoman I see now, I was looking at the beginning of the videos not the end.

That being said, at first glance, this just looks like it has to do with how ffmpeg splits up the videos, or possibly due to a timecode conversion error (when passing the frame number to ffmpeg). I only say this because not all scenes exhibit this problem, thus the error doesn't add up like I would expect if the actual detection was wrong (or it was an off-by-one in which case every video would be affected).

Thanks for the report, will definitely have to do some more investigation into this. If anyone has any feedback or possible causes for this issue, it would be much appreciated.

[Breakthrough]

Also @typoman, do you not see this behavior using an older version of PySceneDetect in particular? (e.g. v0.5-beta-1)

[typoman]

I haven't used that version but I can test if you tell me how can I install it.

[Breakthrough]

@typoman Sorry I thought you meant you initially used an old version that did not have the behavior you were experiencing. If that's not the case than don't worry - I don't think anything has changed to make this any different, I think this was how it always was. BTW, to install an older version, first uninstall PySceneDetect via pip, then download a development release, extract it, and type python setup.py install - you can also use a virtualenv to avoid having to uninstall PySceneDetect.

---

If anyone has any ideas why ffmpeg might be including the extra frames at the end, it would be much appreciated, because at the moment I'm not really sure why this is happening to be honest.

[Breakthrough]

Hey all;

I've just released v0.5.2 which has a fix for the improper starting timecode being passed to ffmpeg. This resolves all of the issues I was able to see in the original issue reported by @typoman (thanks again for providing the video output for me to compare against).

Grab the latest version from pip and all videos should definitely be starting at the correct frames now. Additionally I was unable to replicate extra frames being shown at the end, however, should anyone else be able to replicate this on v0.5.2 let me know and I will re-open this issue. Re-running all tests confirmed no extra frames in the videos now on my end.

Thank you all for your assistance! If this crops up again, feel free to post a new comment here, or re-open a new issue.

[Breakthrough]

Looks like this is cropping up again for some reason. If anyone still runs into this, feel free to provide an update in #159 (discussion will continue there).

[Breakthrough]

Hey all;

I pushed one more fix for this issue to master just now, it seems in some cases the actual timecode formatting was skewing the end time by an extra frame. This should reduce the occurrences of this issue significantly.

Note that it has not been completely resolved, so if this does still happen to anyone, I encourage you to provide more test cases to validate against, and provide any additional details you can. I've identified some options that can be added to ffmpeg which seems to help with videos that exhibit this issue:

scenedetect -i VIDEO_FILE detect-content split-video -a "-r VIDEO_FRAMERATE -vsync 0"

Where VIDEO_FRAMERATE must be set to the exact framerate of the video. If you are still experiencing this issue but this command resolves it, please let me know by responding to this issue or #159.

Thank you all for your assistance and patience with this issue.

[snwfdhmp]

Thanks for the update and the fix, I'll try it out soon !

[Breakthrough]

@typoman Not sure how I was able to replicate this on my end - I tried with both an old and new version of ffmpeg on my system (Windows) just now, and don't see the same output (I get frame perfect cuts on the Goldeneye test clip). Starting to wonder if this only affects a particular version of ffmpeg or a particular OS/distribution.

I definitely see the 1-2 extra frames in the output you provided me, am just having trouble replicating that on my end for some reason.

Would you be able to re-run that test with PySceneDetect v0.5.2, as well as a newer version of ffmpeg? Thanks!

Edit: Will try to run this on Linux instead now to see if that changes anything.

Edit: Same result on both Windows/Linux using PySceneDetect v0.5.2...

[typoman]

Hey I tried to use it but I get this:

Downloads % scenedetect --input goldeneye.mp4 detect-content split-video
Traceback (most recent call last):
  File "/Users/bman/.pyenv/versions/3.8.2/bin/scenedetect", line 5, in <module>
    from scenedetect import main
  File "/Users/bman/.pyenv/versions/3.8.2/lib/python3.8/site-packages/scenedetect/__init__.py", line 52, in <module>
    from scenedetect.scene_manager import SceneManager
  File "/Users/bman/.pyenv/versions/3.8.2/lib/python3.8/site-packages/scenedetect/scene_manager.py", line 56, in <module>
    import cv2
ModuleNotFoundError: No module named 'cv2'
Downloads % pip install opencv
ERROR: Could not find a version that satisfies the requirement opencv (from versions: none)
ERROR: No matching distribution found for opencv
zsh: no matches found: scenedetect[opencv,progress_bar]
bman@Bahmans-MacBook-Pro Downloads % pip install scenedetect[opencv,progress_bar]                
zsh: no matches found: scenedetect[opencv,progress_bar]

I'm on Catalina and a new system.

[Breakthrough]

@typoman Interesting, I was able to just install it on a new Linux system without issue... I think the package is called opencv-python, can you try installing that one? (e.g. pip install opencv-python)

(The "option" I added to PySceneDetect is called just openCV, but if it's causing an issue like this, I'll rename it to match the opencv-python package name)

[typoman]

Hey I installed the opencv and it works now. Unfortunately I still get the extra frames at the end of some segments.

[Breakthrough]

@typoman Could you test the development version of v0.6 ( https://github.com/Breakthrough/PySceneDetect/archive/v0.6.x.zip )? It includes another potential fix for this issue related to the audio encoding.

Also which media were you testing with, the Goldeneye test video, or just your own media? Could you share the results you're getting as well as the source material if possible?

Thank you so much for your assistance!

Please provide as much info as you can where possible, I really want to fix this for the next release!

[typoman]

Here is the command I used with pip installed version and the output. Source is the golden eye video:

Downloads % scenedetect --input goldeneye.mp4 detect-content split-video
[PySceneDetect] Loaded 1 video, framerate: 23.98 FPS, resolution: 1280 x 544
[PySceneDetect] Downscale factor set to 5, effective resolution: 256 x 108
[PySceneDetect] FFmpeg codec args set: -c:v libx264 -preset veryfast -crf 22 -c:a copy
[PySceneDetect] Video output file name format: $VIDEO_NAME-Scene-$SCENE_NUMBER
[PySceneDetect] Detecting scenes...
[PySceneDetect] Processed 1980 frames in 2.2 seconds (average 891.83 FPS).
[PySceneDetect] Detected 21 scenes, average shot length 3.9 seconds.
[PySceneDetect] Comma-separated timecode list:
  00:00:03.754,00:00:08.759,00:00:10.802,00:00:15.599,00:00:27.110,00:00:34.117,00:00:36.536,00:00:42.501,00:00:44.002,00:00:45.837,00:00:48.966,00:00:51.134,00:00:52.552,00:00:53.428,00:00:55.639,00:00:56.932,00:01:10.779,00:01:18.036,00:01:19.913,00:01:21.999
[PySceneDetect] Splitting input video using ffmpeg, output path template:
  $VIDEO_NAME-Scene-$SCENE_NUMBER.mp4
ffmpeg version 4.2.2 Copyright (c) 2000-2019 the FFmpeg developers
  built with Apple clang version 11.0.0 (clang-1100.0.33.17)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.2.2_2 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-videotoolbox --disable-libjack --disable-indev=jack
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/Users/bman/Downloads/goldeneye.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2016-02-03T04:12:42.000000Z
  Duration: 00:01:22.59, start: 0.000000, bitrate: 1533 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1280x544 [SAR 1:1 DAR 40:17], 1338 kb/s, 23.98 fps, 23.98 tbr, 24k tbn, 47.95 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 191 kb/s (default)
    Metadata:
      creation_time   : 2016-02-03T04:12:42.000000Z
      handler_name    : IsoMedia File Produced by Google, 5-11-2011
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
[libx264 @ 0x7f97be00aa00] using SAR=1/1
[libx264 @ 0x7f97be00aa00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x7f97be00aa00] profile High, level 3.1
[libx264 @ 0x7f97be00aa00] 264 - core 155 r2917 0a84d98 - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=2 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=17 lookahead_threads=4 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=1 keyint=250 keyint_min=23 scenecut=40 intra_refresh=0 rc_lookahead=10 rc=crf mbtree=1 crf=22.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'goldeneye-Scene-001.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    encoder         : Lavf58.29.100
    Stream #0:0(und): Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 1280x544 [SAR 1:1 DAR 40:17], q=-1--1, 23.98 fps, 24k tbn, 23.98 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc58.54.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 191 kb/s (default)
    Metadata:
      creation_time   : 2016-02-03T04:12:42.000000Z
      handler_name    : IsoMedia File Produced by Google, 5-11-2011
frame=   91 fps=0.0 q=-1.0 Lsize=     455kB time=00:00:03.73 bitrate= 997.2kbits/s speed=18.2x    
video:363kB audio:88kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.931592%
[libx264 @ 0x7f97be00aa00] frame I:1     Avg QP:19.18  size: 28013
[libx264 @ 0x7f97be00aa00] frame P:24    Avg QP:19.87  size:  8449
[libx264 @ 0x7f97be00aa00] frame B:66    Avg QP:21.60  size:  2117
[libx264 @ 0x7f97be00aa00] consecutive B-frames:  3.3%  0.0%  0.0% 96.7%
[libx264 @ 0x7f97be00aa00] mb I  I16..4: 19.5% 58.6% 21.9%
[libx264 @ 0x7f97be00aa00] mb P  I16..4: 12.8% 15.8%  0.7%  P16..4: 26.3%  9.7%  4.0%  0.0%  0.0%    skip:30.7%
[libx264 @ 0x7f97be00aa00] mb B  I16..4:  1.6%  1.9%  0.0%  B16..8: 13.9%  2.9%  0.1%  direct: 7.4%  skip:72.1%  L0:45.9% L1:44.8% BI: 9.3%
[libx264 @ 0x7f97be00aa00] 8x8 transform intra:54.5% inter:45.4%
[libx264 @ 0x7f97be00aa00] coded y,uvDC,uvAC intra: 34.7% 58.0% 8.8% inter: 3.0% 16.5% 0.1%
[libx264 @ 0x7f97be00aa00] i16 v,h,dc,p: 43% 39% 13%  5%
[libx264 @ 0x7f97be00aa00] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 29% 29% 33%  1%  1%  2%  1%  1%  2%
[libx264 @ 0x7f97be00aa00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 27% 37%  8%  4%  5%  6%  4%  5%  4%
[libx264 @ 0x7f97be00aa00] i8c dc,h,v,p: 40% 33% 24%  3%
[libx264 @ 0x7f97be00aa00] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0x7f97be00aa00] kb/s:781.00
[PySceneDetect] Output from ffmpeg for Scene 1 shown above, splitting remaining scenes...
[PySceneDetect] Video splitting completed, individual scenes written to disk.

The splitted output:
goldeneye.zip

When I use the debuggin mode I get this:

Downloads % scenedetect -v debug -l BUG_REPORT.txt --input goldeneye.mp4 detect-content split-video
Traceback (most recent call last):
  File "/Users/bman/.pyenv/versions/3.8.2/bin/scenedetect", line 8, in <module>
    sys.exit(main())
  File "/Users/bman/.pyenv/versions/3.8.2/lib/python3.8/site-packages/scenedetect/__main__.py", line 58, in main
    cli.main(obj=cli_ctx)   # Parse CLI arguments with registered callbacks.
  File "/Users/bman/.pyenv/versions/3.8.2/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/Users/bman/.pyenv/versions/3.8.2/lib/python3.8/site-packages/click/core.py", line 1268, in invoke
    Command.invoke(self, ctx)
  File "/Users/bman/.pyenv/versions/3.8.2/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/bman/.pyenv/versions/3.8.2/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/Users/bman/.pyenv/versions/3.8.2/lib/python3.8/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/bman/.pyenv/versions/3.8.2/lib/python3.8/site-packages/scenedetect/cli/__init__.py", line 261, in scenedetect_cli
    logfile = ctx.obj.get_output_file_path(logfile)
AttributeError: 'CliContext' object has no attribute 'get_output_file_path'
Downloads %

I will post the second try with the zipped version you linked in the next comment.

[typoman]

Here is the output from the development version of v0.6. Everything looks correct this time except one cut was missed in the goldeneye-Scene-017.mp4. At least no extra frames anymore.

Downloads % scenedetect --input goldeneye.mp4 detect-content split-video
[PySceneDetect] Loaded 1 video, framerate: 23.98 FPS, resolution: 1280 x 544
[PySceneDetect] Downscale factor set to 5, effective resolution: 256 x 108
[PySceneDetect] FFmpeg codec args set: -c:v libx264 -preset veryfast -crf 22 -c:a aac
[PySceneDetect] Video output file name format: $VIDEO_NAME-Scene-$SCENE_NUMBER
[PySceneDetect] Detecting scenes...
[PySceneDetect] Processed 1980 frames in 2.2 seconds (average 919.96 FPS).
[PySceneDetect] Detected 21 scenes, average shot length 3.9 seconds.
[PySceneDetect] Comma-separated timecode list:
  00:00:03.754,00:00:08.759,00:00:10.802,00:00:15.599,00:00:27.110,00:00:34.117,00:00:36.536,00:00:42.501,00:00:44.002,00:00:45.837,00:00:48.966,00:00:51.134,00:00:52.552,00:00:53.428,00:00:55.639,00:00:56.932,00:01:10.779,00:01:18.036,00:01:19.913,00:01:21.999
[PySceneDetect] Splitting input video using ffmpeg, output path template:
  $VIDEO_NAME-Scene-$SCENE_NUMBER.mp4
ffmpeg version 4.2.2 Copyright (c) 2000-2019 the FFmpeg developers
  built with Apple clang version 11.0.0 (clang-1100.0.33.17)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.2.2_2 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-videotoolbox --disable-libjack --disable-indev=jack
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/Users/bman/Downloads/goldeneye.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2016-02-03T04:12:42.000000Z
  Duration: 00:01:22.59, start: 0.000000, bitrate: 1533 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1280x544 [SAR 1:1 DAR 40:17], 1338 kb/s, 23.98 fps, 23.98 tbr, 24k tbn, 47.95 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 191 kb/s (default)
    Metadata:
      creation_time   : 2016-02-03T04:12:42.000000Z
      handler_name    : IsoMedia File Produced by Google, 5-11-2011
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[libx264 @ 0x7fc865808200] using SAR=1/1
[libx264 @ 0x7fc865808200] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x7fc865808200] profile High, level 3.1
[libx264 @ 0x7fc865808200] 264 - core 155 r2917 0a84d98 - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=2 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=17 lookahead_threads=4 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=1 keyint=250 keyint_min=23 scenecut=40 intra_refresh=0 rc_lookahead=10 rc=crf mbtree=1 crf=22.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'goldeneye-Scene-001.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    encoder         : Lavf58.29.100
    Stream #0:0(und): Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 1280x544 [SAR 1:1 DAR 40:17], q=-1--1, 23.98 fps, 24k tbn, 23.98 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc58.54.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      creation_time   : 2016-02-03T04:12:42.000000Z
      handler_name    : IsoMedia File Produced by Google, 5-11-2011
      encoder         : Lavc58.54.100 aac
frame=   90 fps=0.0 q=-1.0 Lsize=     415kB time=00:00:03.76 bitrate= 903.9kbits/s speed=17.8x    
video:350kB audio:61kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.024442%
[libx264 @ 0x7fc865808200] frame I:1     Avg QP:19.18  size: 28013
[libx264 @ 0x7fc865808200] frame P:23    Avg QP:19.83  size:  8245
[libx264 @ 0x7fc865808200] frame B:66    Avg QP:21.60  size:  2117
[libx264 @ 0x7fc865808200] consecutive B-frames:  2.2%  0.0%  0.0% 97.8%
[libx264 @ 0x7fc865808200] mb I  I16..4: 19.5% 58.6% 21.9%
[libx264 @ 0x7fc865808200] mb P  I16..4: 11.3% 14.5%  0.3%  P16..4: 27.5% 10.1%  4.2%  0.0%  0.0%    skip:32.1%
[libx264 @ 0x7fc865808200] mb B  I16..4:  1.6%  1.9%  0.0%  B16..8: 13.9%  2.9%  0.1%  direct: 7.4%  skip:72.1%  L0:45.9% L1:44.8% BI: 9.3%
[libx264 @ 0x7fc865808200] 8x8 transform intra:55.6% inter:45.4%
[libx264 @ 0x7fc865808200] coded y,uvDC,uvAC intra: 35.1% 61.0% 9.0% inter: 3.0% 16.5% 0.1%
[libx264 @ 0x7fc865808200] i16 v,h,dc,p: 43% 39% 13%  5%
[libx264 @ 0x7fc865808200] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 28% 29% 34%  1%  1%  2%  1%  1%  2%
[libx264 @ 0x7fc865808200] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 30% 30%  9%  5%  5%  7%  4%  6%  4%
[libx264 @ 0x7fc865808200] i8c dc,h,v,p: 39% 34% 24%  3%
[libx264 @ 0x7fc865808200] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0x7fc865808200] kb/s:761.64
[aac @ 0x7fc865809a00] Qavg: 1044.723
[PySceneDetect] Output from ffmpeg for Scene 1 shown above, splitting remaining scenes...
[PySceneDetect] Video splitting completed, individual scenes written to disk.
Downloads % 

goldeneye2.zip

[Breakthrough]

Awesome, that is exactly the expected result. Thank you so much for your assistance @typoman, I really appreciate it! This confirms that #166 closes this issue (#93), as well as #159.

Also thanks for letting me know about the issues using debug mode, I've opened up a new defect to track that (#169).

If you run into any other issues, or this happens to reoccur in the future, please feel free to create a new defect again. Thank you all so much for your assistance in closing this one out, will try to get a maintenance release out before v0.6 officially into pip as soon as possible.

[adibMosharrof]

I am facing the problem where extra frames appear at the end of detect scenes.

I am using PySceneDetect v0.5.4, Python 3.7.7
Here is a sample code that I created to show the extra frames error
Please let me know if you need more information that might assist in helping to solve the issue.

[Breakthrough]

Hi @adibMosharrof;

Hmm I see, do you mean in the video files it generates, or images?

In your script, it looks like you're comparing the frame index to the total number of frames. The last index will always be one less than the total # of frames, because if we have a 1-frame video, it starts and ends at frame 0. However, I will try to replicate this on my end with the video you provided first.

Would you be able to upload it to this issue directly? I want to make sure I'm using the exact same source input as you. If you could also zip and attach the output from PySceneDetect showing the extra frames, that would also be helpful. Lastly, could you also provide the version of ffmpeg you have installed?

Thank you!

[adibMosharrof]

Hello @Breakthrough

Thank you very much for the quick response.

I took a look at the indexes. The starting frame number of the first scene that is detected has a frame number of 0, and the ending frame number of the last scene that is detected has a frame number 1799, whereas the total number of frames in the video was 1788.

I am not using ffmpeg, when i check my pip env for ffmpeg -version, i get ffmpeg command not found.

As for the outputs, do you want me to provide a csv file for the scenes that were generated? I can provide that as well as the video that I am using to this issue
tmp.zip

[Breakthrough]

Hi @adibMosharrof;

How are you validating the output? I know that the OpenCV frame count isn't 100% accurate for every single type of video/container (i.e. calling get(cv2.CAP_PROP_FRAME_COUNT) may return a number lower than the actual number of frames in a video). I verified this by using ffmpeg to extract all frames from the video you provided:

ffmpeg -i [video.mp4] thumb%04d.jpg

This resulted in exactly 1799 frames being extracted. Looking frame by frame, the cuts PySceneDetect provided in the CSV you gave seem accurate to me. I've attached a .ZIP containing the last 10 frames as well as the output from split-images for you to compare on your machine:
last-10-frames.zip
save-images-output.zip

Note that the numbers in the last-10-frames.zip start from 1 instead of 0, so in the .CSV, you have to add +1 to the starting frame to get the filename. You can also verify the splits are correct using the save-images command.

Thanks!

[adibMosharrof]

Thank you very much for the response and debugging the example video. I think I will have to switch to ffmpeg for extracting frames instead of opencv.