zero-length files when circular buffer recording

[chconnor]

Hi -- posted this in the forums but the radio silence makes me think this may be the better place. :-)

I've been developing a security camera with a pi zero and the pi camera -- latest versions of both.

Following the splitting to/from a circular stream recipe, I record to the circular buffer. When motion is detected (using comparisons between low-res jpeg captures from the same stream) it splits to a file and writes the circular buffer to another file.

The trouble I'm having is that the "pre-record" outputs of the circular buffer are usually empty, but not always. A pattern I've noticed is that it seems to work fine when the camera is looking at a very simple low-detail image (e.g. the ceiling, or if I put a piece of paper in front of it that fills the frame, or it's night and the frame is mostly black). When the scene is complex scene, I get only empty pre-record files.

I saw in the documentation notes about SPS frames (which I only loosely understand) being required when writing out the circular buffer, so I tried increasing the pre-record time, and tried setting intra_period to 60, but neither helped. I tried setting first_frame to "None" and the pre-record files would be >0 file size, but small and apparently still empty of real data.

A simplified version of my code looks like this:

with picamera.PiCamera(sensor_mode=4) as camera:
    camera.resolution=(1640, 1232)
    camera.framerate=30
    stream = picamera.PiCameraCircularIO(camera, seconds=5, bitrate=15*1000000)
    camera.start_recording(stream, format='h264', profile='high', level='4.2', bitrate=15*1000000, quality=25)

    try:
        while not term_prog:
            try:
                addTimestamp(camera) # sets camera.annotate_text
                camera.wait_recording(0.5)

                if detect_motion(camera, False):
                    if spaceRemaining() < disk_warning_bytes:
                        print('Almost out of space! Not recording and waiting extra 5 seconds. Free bytes: ' + str(spaceRemaining()))
                        camera.wait_recording(5)
                    else:
                        camera.split_recording('filename.h264') # real code makes unique filenames here
                        stream.copy_to('pre-record-filename.h264', seconds=5)
                        stream.clear()

                        while detect_motion(camera, True):
                            addTimestamp(camera)
                            camera.wait_recording(iteration_delay)

                        camera.split_recording(stream)

            except:
                e = sys.exc_info()[0]
                print(getTimestamp() + " - Error in loop! Will proceed anyway:\n\n%s" % e)

    finally:
        camera.stop_recording()
        print('All done at ' + getTimestamp())

Can anyone see what I'm doing wrong? Or maybe it's a bug? We just had another potential thief up on our porch so I'm eager to get these cameras going. :-)

Thanks!

[waveform80]

Hi -- posted this in the forums but the radio silence makes me think this may be the better place. :-)

Unfortunately I haven't found the time to track the forums recently so I've more or less fallen back on GitHub as the sole place I check (and sometimes the RPi stack exchange which notifies me of camera-related questions). Still, questions are very welcome here! Often times, it lets me know if I've got the design right :)

A simplified version of my code looks like this:...

Hmm, could you try something? Try removing second=5 from the copy_to call so that the pre-record file gets everything from the circular buffer. I suspect this is something to do with the copy_to code not finding an SPS header within the last 5 seconds of the buffer. Speaking of which...

I saw in the documentation notes about SPS frames (which I only loosely understand) being required when writing out the circular buffer, so I tried increasing the pre-record time, and tried setting intra_period to 60, but neither helped. I tried setting first_frame to "None" and the pre-record files would be >0 file size, but small and apparently still empty of real data.

Ah, okay - might need to expand the docs there. Here's a brief run-down: the camera's H.264 encoder basically outputs three types of frame: SPS headers, I-frames and P-frames...

• I-frames - also known as key-frames or intra-frames. These are frames containing a complete picture (in a better-than-JPEG format).
• P-frames - these are frames containing the changes from the prior frame. As a result they're much smaller than I-frames (this is where the majority of H.264/MPEG's compression comes from) but they can only be decoded if you've already decoded the prior frame. An MPEG file usually looks something like IPPPIPPPIPPP etc. Hence if you skip to a P-frame you'll see a rather corrupted display moving around until you hit the next I-frame and everything clears up.
• SPS headers - this is a tiny non-image frame which contains configuration information including how big the frames of the video stream are. Without this information, decoding the rest of the stream is tough. The H.264 encoder usually produces an SPS header immediately prior to an I-frame, so the stream actually looks like SIPPPSIPPPSIPPP, etc.

This is why copy_to tries to find the first SPS header frame within the limits you've specified. Now, you've specified seconds=5 so it'll seek backward from the end of the stream looking for SPS headers. As soon as it hits the specified 5 second limit it'll copy from the last found SPS header to the end of the stream. However, if you've not found any SPS headers you'll naturally copy nothing.

Why might it find no SPS headers? I've been a bit untruthful in my illustration of the frame layout above. It's not really 3 P-frames and then another I-frame. Usually it's more like 29 P-frames and then an I-frame. The rate at which I-frames are produced (and thus SPS headers) is dictated by the intra_period parameter. If this is set to 30 then every 30th frame will be an I-frame (preceded by an SPS header); if the framerate is 30 then that means there'll be an I-frame (and an SPS header) at the start of each second of video.

I forget what the default for intra_period is, but that's another thing to try: stick seconds=5 back into the copy_to call, and then add something like intra_period=30 to the start_recording call.

Naturally, you can decrease intra_period to 1 if you want, in which case every frame will be an I-frame but you can obviously see how that'll ruin the compression ratio. So, like most things in video compression, it's a trade-off: decrease intra_period and you lose compression, but gain fidelity in being able to copy from a particular location. Increase intra_period and you gain compression, but the points from which you can copy become further apart.

[waveform80]

Oh, another thing to try: you're setting the buffer size to 5 seconds and wanting to copy 5 seconds which in the case of high-motion scenes might stretch things to their limit. Give it a little leeway: set the buffer to, say, 7 seconds but only copy 5.

[chconnor]

Thanks for the thoughtful and detailed reply! I will try removing the seconds= from the copy_to, and I'll try the 7/5 trick as well. I did already try the intra_period thing, though (see first post -- I set it to 60, and I'm recording at 30fps so that'd be 2 seconds which should be well within the 5 second window?)

[waveform80]

Yeah, with a 5 sec buffer, 30fps recording, and an i-period of 60 or less there should always be a couple of SPS headers in the buffer (once it's filled). I'll see if I can reproduce the blank copies here.

[6by9]

Daft thought - are you getting any SPS/PPS header buffers at all? Is the inline_headers option actually working as expected?
I see that the default appears to be "inline_headers=True", but that doesn't necessarily mean that it is being complied with. (It's also a bit of a strange thing to be set if saving directly to disc, but never mind).

[chconnor]

Well, I removed seconds= from the copy_to, and it works! The length of the pre-record is highly variable, and sometimes quite long; e.g. I set it to 5 seconds but copy_to generates 60-second long clip; I assume that's because of the documented assumption of of full-bitrate video, and I'm recording a mostly static scene. And the more static the scene the more likely it was to work (e.g. the mostly-black night recordings worked), so maybe that's a clue for your debugging.

For some reason I was reticent to remove "seconds=5" when originally reading the documentation: Maybe i was afraid that "all frames in the buffer" would include frames not necessarily in the current recording (if e.g. the current recording to the stream hadn't yet filled one loop's worth of frames) ?

Anyway, i'm happy to have a solution. Thanks for the help, as always! (Feel free to close this bug unless you want to leave it open for tracking the intra_frame stuff or whatever else.)

(and fingers crossed for 315 some day. :-)

[waveform80]

Ah, glad that worked - and yes, you're absolutely right that a mostly static scene compresses really well (lots of damn near empty P-frames) so you wind up with tons of stuff in the buffer. No need to worry about stuff not in the current recording: the circular buffer isn't really pre-allocated - it grows from empty up to the maximum size and then lops off chunks that exceed the maximum (this probably sounds horribly inefficient to anyone used to working with "classic" circular buffers but actually it's rather fast because all we're dealing with are pointers to chunks; it does making reading slow but that's the minority operation here - writing is far more common).

I'm still slightly concerned that copy_to didn't find any SPS headers at all in the 5 second window it was given to search. I haven't managed to recreate that but I'll have another go today and see if I can force it somehow. For now, I'll leave this open until I'm sure I can't reproduce.

[chconnor]

If it helps: the zero-length file issue didn't happen when I first fired up the code -- it would work fine the first time it generated a set of files, which led me to believe everything was fine. It was when I had the thing running over a couple days that I noticed the pattern: 0-length pre-record files during the day and content in the pre-record files at night. Maybe the demo code posted above needs a few iterations, or a different time between motion detection, or some time to pass, before it will reproduce the issue?

[emmanuel-contreras]

Hello, happy to have found this issue, which I am also having. It was working fine for a few days, but now everything recorded is has a zero size/empty file. No updates to my code and not sure what is causing it. Removing the "seconds=20" in copy_to() did fix the issue, but now output videos vary quite a bit on the buffer size.

As a workaround, I am testing using a 5 second buffer, which I know will record much more, so that the output file is around 30 seconds which is what we want it to be.

Here is my python file:
https://github.com/emmanuel-contreras/rpi_usb/blob/master/circularBuffer.py

Thank you!

[helmutka]

For me, this seems to be the same issue as #302. A bug in "copy_to" with parameter "seconds".