[BUG] 'queue=False' appears not to disable queue prepopulation #787
Comments
alexw-im
changed the title
alexw-im
changed the title
|
Not directly related to the bug but in my mission to get accurate timestamps: I just came across #703 and it seems my notion that the "SensorTimestamp" and time.monotonic_ns() come from different domains (VPU vs CPU) is actually incorrect. With that knowledge, I modified my test code a little, not WITHOUT the workaround present: You see it now reports the What's throwing me for a loop now is that if you look at the output below, the "frame to next line" delta is always between 17 and 21ms, even when the Does this imply that the "SensorTimestamp" is simply when the frame got returned, rather than being precisely tied to when the frame hit the SOC coming from the camera? |
|
A couple of things to consider. The Pi won't respond in a particularly real-time way, so measuring elapsed times is going to give you very variable results. So you need to use the SensorTimestamp in the metadata. The SensorTimestamp records the frame start time from the camera, so you can generally expect it to be earlier than the time at which you try to capture the request (even with queue=False). Even this, though, indicates the start of the frame being read out, so the time at which it started being exposed is even earlier. I guess you need to be very clear what you want, because the start of exposure, the start of readout and the end of readout are all different!! The SensorTimestamp is based on CPU times, so this should line up with what Python's time.monotonic() reports. If you really want to guarantee that the frame starts being read out after a specific time, I think you'd have to capture every request and check that the SensorTimestamp is late enough. If you care about the start of exposure, then you probably need to account for the current exposure time too. You'll need to set the buffer_count to at least 2, otherwise you're guaranteed to drop every other camera frame. Hope that helps - sorry if it's all a bit complicated! |
|
That all jives with info from this issue I opened with libcamera (since their documentation about the timestamp is a little ambiguous and not technically quite accurate) https://bugs.libcamera.org/show_bug.cgi?id=196, thank you!
I think what I'm trying to do is exactly this. I explicitly don't want a frame that was started earlier than I asked for, and I don't care about dropping frames. For instance, say my exposure time is a full second, and "processing" takes half a second. I'm trying to minimize total latency from frame start to computation finish, NOT trying to "return data as soon as possible" For example A, let's say it's T0 and I'm "ready" so I command a capture, wait 1s, get a frame, process for half a second, and 1.5s plus or minus some jitter later, I return data with a total latency of around 1.5s from image start to data out. For example B, let's say it's T0 and I'm "ready" so I command a capture, but there's already a frame in the buffer that finished exposure 100ms ago, and that frame is returned to me instantly to process on, so I take my half second and do that. In this case, since the frame STARTED 1.1s before it was returned to me, I do my processing and return data with a full latency of 1.6s. The data in B gets returned sooner than the data in A, BUT the data corresponds to a more-stale sample than in A. My understanding with the queue=false scenario is that it SHOULD force scenario A, without my silly workaround of "bogart the buffer so the camera can't fill it until I say so" - which it seems not to be doing in my case. But maybe you're suggesting a best-of-both worlds? I never want to receive a buffer that's already filled, but I wouldn't mind receiving a buffer that's in the process of filling. As long as the buffer corresponds to an exposure-in-progress-or-readout, that still implies a minimized total shutter-to-processed latency, rather than a filled buffer which means "image is no longer fresh" |
|
If you just want to minimise the time from the frame starting to processing it, then I think that setting queue=False probably does this, and the frames you get will be the very next ones to pop out of libcamera. But the behaviour of the system will still be jittery (Linux is not very real-time after all, and Python even less so), and interpreting the timestamps is not so straightforward, as per the other details in this discussion. Maybe you could set a pre_callback which records the current time in the CompletedRequest object. That should give you the time (as best we can get hold of it) at which the frame pops out of libcamera. Would that help you to sort out what is going on? |
Describe the bug
While using "queue=False" in i.e.
config = picam2.create_still_configuration({"size": (1280, 800)}, raw={'format': 'R10'}, queue=False, controls=controls), it doesn't appear to accomplish the design intent of keeping the output buffer from pre-filling.To Reproduce
I'm using an OV9281 sensor (arducam module in this case) with a Pi 4B.
Expected behaviour
This snippet SHOULD take one 100ms capture on-demand once per while loop, then sleep a while (a second). The idea is that I want a single frame as close to right-after-when-I-request-it as possible.
Having read 4.2.1.4 in the Picamera2 manual, I figured I'd use the queue=False option in my configuration to make sure the queue was never pre-filled with prior-to-trigger image data.
As written, the call to
picam2.capture_request()should be blocking and take at least as long as the exposure. I'd expect that the "elapsed time" counter should always be >> 100ms: 100ms to expose, plus however much to read out, process, compute, update apt, check stocks, phone home, etc.In this case, however, my "elapsed" timer SOMETIMES takes >> 100ms, and sometimes returns within single-digit milliseconds. See console output below.
Console Output, Screenshots
Note below that all snapshots take well over 100ms to return, except the 3rd trigger which returns in only 4.9ms
Workaround :
I believe the above is a bug: perhaps "queue=False" gets ignored if there's only one allocated memory queue, assuming that the user won't want a frame "only very occasionally"?
In any event, the following configuration of the script works around the issue by deliberately holding the queue until right before the desired capture. We start by capturing a request right before entering the loop, then from that point on we only release that request right before making a new one.
The fact that this workaround works as expected makes me think the base case is indeed buggy and I'm not simply misunderstanding usage.
And the output. Note that the minimum elapsed capture time is 152ms:
Hardware :
Pi 4B with ArduCam OV9281 B0165 module.
Additional context
My true motive for this is to timestamp the frame capture as accurately as possible. I know that neither python nor linux are real-time and thus there are any number of preemptions that could cause jitter in my "timestamp-then-snap" delay, but I'm hunting for ways to minimize that. I know the returned "SensorTimestamp" is quite precise and accurate to the true second-shutter timing, but as far as I'm aware there's no way to accurately translate that to system time, correct? Whatever I can do to get a precise
time.time()for the start/center/end of the shutter-open, I'd quite like to do, up to and including preempt_rt, but I'm not sure how dramatically that would help.The text was updated successfully, but these errors were encountered: