Unexpected Behavior When Using Camera

[SuperN1ck]

Hey,
I have one problem when using the camera of the AI-Deck I couldn't fix properly. I tested out the test_camera-example for the GAP8-chip and got some weird results (but this is also applicable to other examples as far as it seems to me).

When I try to capture an image it gets tiled in a weird way like so:

What got me thinking is that this only occurs for the first image. If you wrap it in a loop and continuously capture images like in the wifi streaming example or the GAP8-IO example the problem is gone starting from the second image. So I was investigating what caused this issue and how to solve it. So far the only solution that worked for me is to start and stop the camera once before starting the actual capture process but also after setting the registers for the orientation or other parameters (for instance around line 95)

pi_camera_control(&camera, PI_CAMERA_CMD_START, 0);
pi_camera_control(&camera, PI_CAMERA_CMD_STOP, 0);

Doing so is exactly what happens when capturing images continuously but leaving out the capture call.

Therefore this resolves the issue and generates a correct image

But doesn't seem like the correct or proper solution. This also happened on two different AI-Decks so it don't think my specific board might be faulty.

So this results in a couple of open questions:

• Does this only happen for our AI-Deck(s)? Or did anyone else experience a similar issue?
• Is there a better way to fix the problem? I also looked at another repository providing an implementation for interfacing the camera but couldn't see any major differences
• Is this a general issue which should be posted in the gap_sdk repository?

Cheers,
-Nick

[knmcguire]

Hi Nick,

It is probably something that is for the regular SDK I think. I will try to nudge one of the developers to look at this issue. Maybe they know more!

[SuperN1ck]

Hey Kim,
thank you for your help. If there is a public discussion or you need more info let me know.

[Yaooooo]

Hi Nick,

Sorry for my late reply for your issue.
About your problem of camera, it's not the SDK/HW problem, but it's a known hw limitation:

• For single short mode: "start" and "stop" are necessary to capture a clean image.
• For continuous mode: you don't need "start" and "stop" for each frame, but just before/after the streaming.

This continous mode means you need to keep "always 2 capture task in uDMA (even each task is not a full image)", like what we do in this example: gap_sdk/examples/pmsis/bsp/cameras/test_camera_stream

In this case, the uDMA will keep working and you will get the full image streaming.

[SuperN1ck]

Hey Yao,
thank you for your reply. So, if I understand you correctly, there is no good way to retrieve a single image without the tiling in single shot mode?
Let me summarize what I tried:

Single Shot Mode

This is what should be achieved in this example: https://github.com/bitcraze/AIdeck_examples/blob/master/GAP8/test_functionalities/test_camera/test.c -- Capture a single image from the camera and save it to the disk.
But I was only able to capture a clean image with a double start/stop command. Something among the lines (throwing out other stuff):

Sync

...
pi_camera_control(&camera, PI_CAMERA_CMD_START, 0); // I added this
pi_camera_control(&camera, PI_CAMERA_CMD_STOP, 0); // I added this
pi_camera_control(&camera, PI_CAMERA_CMD_START, 0);
pi_camera_capture(&camera, buff, BUFF_SIZE);
pi_camera_control(&camera, PI_CAMERA_CMD_STOP, 0);
pi_camera_close(&camera);
...

Async

...
pi_camera_control(&camera, PI_CAMERA_CMD_START, 0); // I added this
pi_camera_control(&camera, PI_CAMERA_CMD_STOP, 0); // I added this
pi_camera_capture_async(&camera, buff, BUFF_SIZE, pi_task_callback(&task, handle_transfer_end, NULL)); // handle_transfer_end sets done to true.
pi_camera_control(&camera, PI_CAMERA_CMD_START, 0);
while(!done){pi_yield();}
pi_camera_control(&camera, PI_CAMERA_CMD_STOP, 0);
pi_camera_close(&camera);
...

Continuous Mode

For the continuous mode I looked at the example you mentioned. But since in test_camera_stream the buffer is just dumped to disk I can not confirm that it is a clean image (or is there an easy way?).

Therefore, I looked at this other example: https://github.com/GreenWaves-Technologies/gap_sdk/blob/release-v3.7/examples/pmsis/bsp/cameras/test_camera_lcd/test_camera_lcd.c
Here, the image is saved as an actual image. But the images are only clean starting from the 2nd image. So the first one is also tiled (basically unusable) in the same way as the one I already uploaded in my initial post. But I guess in continuous mode you could neglect that, given it's a HW limitation. It's just something what needs to be noted down somewhere then I guess.

Question

So basically my questions boil down to:

• @Yaooooo How to ensure that you get a clean image when performing a single shot capture? --> Is my solution (double start/stop) valid due to the hardware Iimitation or is there a better way to do it?
• @knmcguire If you want I can update the test_camera-example according to test_camera_lcd and test_camera_stream to (also) capture continuously?

[knmcguire]

Hi @SuperN1ck, You have my blessing to update the test camera examples :) If you haven't done it already

[Yaooooo]

Hi @SuperN1ck ,
I am from GreenWave, actually, I may found your issue and fixed in the test_functionalities/test_camera
Since I cannot push my fixes in Bitcraze's repo, I do a pull request for you: SuperN1ck#1
Then you can redo a pull request for Bitcraze to include this changes.
Please tell me if it works for you.

[knmcguire]

I guess this already was fixed in in PR #23 right?

[SuperN1ck]

Yes it was fixed to a degree which don't imply any modifications in the GAP SDK itself (see: SuperN1ck#1). The problem still occurs sometimes for me when not using the default values, but I think this is more related to the GAP SDK so it might make sense to reopen an issue there but I'm fine for now.