Feature Request: Useful Camera Examples

[hpssjellis]

Feature Request: Useful Camera Examples

Since the Portenta Vision Shields (Lora and Ethernet) work for both the PortentaH7 (lite, connected) and soon the PortentaX8, it would be useful to have a few camera examples that are more complete than the raw demo already supplied.

I have successfully made Portenta Examples for most other functions on my "Portenta Pro Community Solutions" Arduino IDE library just search for "Community", but the camera eludes me. The MBED Portenta is slightly different than other Arduinos making out-of-the-box Arduino examples not work.

Best would be to have several small examples such as:

1. Portenta Camera QR-code reader
2. Portenta Camera to Grayscale OLED (Already done)
3. Portenta Camera to PNG or JPEG
4. Portenta Camera to SD-card

but might be easier to group these all together into one example:

Grouped example: Camera QR-code reader that shows the QR code on a Grayscale OLED and then converts to JPEG/PNG and saves to the Portenta SDCard. An example like that would show all the necessary tricks needed to build from there.

@facchinm @sebromero @giulcioffi @manchoz @per1234 @pennam

I would be willing to work with an Arduino Dev who could see my code and give some suggestions. My basic Portenta Camera examples are here . I have no idea where to start with converting the code to the MBED Portenta for QR-code, PNG or JPEG or saving a compressed image to the Portenta-SD-card

[avibrown]

Hear, hear!

[eivholt]

+1

[sebromero]

We're currently having a look at 3. and 4..

[hpssjellis]

@sebromero not sure if it is helpful, but this repo has camera saving to sd card and then cellular transfer for the sony spressense https://github.com/edgeimpulse/ei-spresense-4g-wildlife-camera/blob/main/wildlife-camera-sketch/4g_camera.ino

[sebromero]

Hi all! There is an update on 4. @marqdevx wrote a great tutorial on how to do this. Check it out here: https://docs.arduino.cc/tutorials/portenta-vision-shield/camera-to-bitmap-sd-card

[marqdevx]

You can find the sketch also on the Arduino_Pro_Tutorials

https://github.com/arduino-libraries/Arduino_Pro_Tutorials/blob/main/examples/Vision%20Shield%20to%20SD%20Card%20bmp/visionShieldBitmap/visionShieldBitmap.ino

[hpssjellis]

@marqdevx Thank you so much for the excellent example. I notice that the code is only for the PortentaH7 and not for both the Portenta and Nicla Vision like the standard examples:

#include "camera.h"

#ifdef ARDUINO_NICLA_VISION
  #include "gc2145.h"
  GC2145 galaxyCore;
  Camera cam(galaxyCore);
  #define IMAGE_MODE CAMERA_RGB565
#else
  #include "himax.h"
  HM01B0 himax;
  Camera cam(himax);
  #define IMAGE_MODE CAMERA_GRAYSCALE
#endif

Any suggestions for running a similar example for the color/grayscale camera on the Nicla Vision?

[marqdevx]

Thanks!! @hpssjellis

We did not add the Nicla Voice compatibility cause it does not have an SD Card out of the box.

But you could use the same settings:

• same resolution CAMERA_R320x240 just change the class
• and maintain the IMAGE_MODE to CAMERA_GRAYSCALE

You could also use the RGB565, but you may tweak the sketch.

I need to check it out, I think it needs some additions to be compatible with the RGB 565 format

[hpssjellis]

@marqdevx Thanks so much for the quick reply.

My actual goal is to transfer a very small 96x96 image offsite (perhaps using WiFi on the Nicla Vision ), so saving to the SD Card is not as relevant as the bigger goal. That is why the original request included some kind of standard compression such as PNG or JPEG so that the image from the Portenta or Nicla Vision is more widely useable. Presently the generated bitmap can not be imported into edgeimpulse.com without some extra image manipulation which makes the process to unwieldy to teach high school kids.

If you can have a look at it that would be awesome, I will also try some research on my side.

Link to good PNG manipulation site https://github.com/bitbank2/PNGenc

Link to good JPEG manipulation site https://github.com/bitbank2/JPEGENC

[marqdevx]

I've tested some converters to convert the bitmap to a PNG but it didn't have compatibility for the mbed Core, I will check those out to see if we are lucky.

Also I would love to see that transfer sketch hehe 👀

[hpssjellis]

@marqdevx

The data transfer to webpage sketch is being worked on with a friend from Italy, not sure if it is ready for viewing yet. I will check with him. The sketch is just for sensor data, I am the one that wants to make it web transfer a small image.

[hpssjellis]

@marqdevx

WebSave of camera data here https://github.com/hpssjellis/my-examples-for-the-arduino-portentaH7/blob/master/research/camera-webserver-save/cam07-websave2.ino

I eventually want grayscale 96x96 but this is bigger and very slow to load almost 30 seconds and the base64 encoded image is not correct. The save text file feature is kind of cool.

[sebromero]

@hpssjellis Interesting example. I'd love to see this working. What makes it slow, the transfer over WiFi or the base64 encoding? We could consider adding GZIP to the mix to make it faster if it's the WiFi that makes it slow.

[hpssjellis]

@sebromero

@hpssjellis Interesting example. I'd love to see this working. What makes it slow, the transfer over WiFi or the base64 encoding? We could consider adding GZIP to the mix to make it faster if it's the WiFi that makes it slow.

The main example is loading data into the textarea and then easily saving the text to a .txt file. I thought that was a security issue and not allowed, but it works fine.

My code does not actually encode to base64 and that is why the image does not yet work, so the slow point is simply the client.write() function, and JZIP probably won't help too much. I will be working on the code over the next while.

[hpssjellis]

@sebromero @marqdevx

The big problem is that the Portenta fb camera data is not in PNG format. That is all that is needed now for this method to work.

The new version has almost no delay.

https://github.com/hpssjellis/my-examples-for-the-arduino-portentaH7/blob/master/research/camera-webserver-save/cam08-web-image-save.ino

I am now testing it on the Nicla Vision

If you put raw PNG data into the textarea and click view image as base64 image it works fine.

[hpssjellis]

Interesting that edgeimpulse has converted portenta images to JPEG. Also using Larry's JPEG converter.

https://github.com/edgeimpulse/firmware-arduino-portenta-h7/tree/f629358c116429b6462563ad6eff760d1000b42e/src/firmware-sdk/jpeg

Link to good PNG manipulation site https://github.com/bitbank2/PNGenc

Link to good JPEG manipulation site https://github.com/bitbank2/JPEGENC

[hpssjellis]

@pnndra @marqdevx

Does anyone know where the definitions for the class FILE are? An example of it's use is here

    FILE *file = fopen(imagePath, "w");

I am trying to get PNG conversion of Camera data working on both the Portenta and NiclaVision which would be huge since it is a web standard, and I am fairly close, what would also help is a link to the old Arduino File class, as that is what I am trying to convert.

I am working on getting this code here working but then need to do much more to get the camera and web server an active part of the code.

[hpssjellis]

My issues is how to use fprintf, fread, fseek, fopen, fclose and fwrite. I am having success with fprintf, fwrite and fclose. Also finding examples for the others such as:

fseek

  fseek(update_file, 0, SEEK_END);
  long update_file_size = ftell(update_file);
  fseek(update_file, 0, SEEK_SET);

for fread here

 /* Read the OTA header ... */
  fread(ota_header.buf, 1, sizeof(ota_header.buf), update_file);

and fwrite here

    int ret = fwrite(&wifi_firmware_image_data[byte_count], chunck_size, 1, fp);

to me it looks like fprintf and fseek have the file as the first argument and fread and fwrite have the file as the last argument.

[hpssjellis]

So I can convert a Portenta or Nicla Vision image data into a PNG and then convert it to Base64 and upload a 16x16 PNG image to a website here Note: larger images work but the website often times-out after 30 seconds, but that whole process is kind of ridiculous. You have a low power device doing all the work and sending data to a high power device to view a webpage.

What would make more sense is to send the raw data from the Portenta or Nicla Vision to the website and let the website javascript do the conversions to BMP PNG or JPEG and then to Base64 if needed and the webpage to allow saving of the images either to disk or local storage. I can probably do the last part but what I am having trouble with is sending the raw byte data through the arduino webserver to a webpage. Whenever I google search I get info on how to convert to Base64 or how to load a saved image.

I will look into littleFS to save the image to a file system in memory, that simplifies some things especially for the Nicla Vision which does not have an SDCard but also adds a potentially slow step. Best would be to load raw image bytes directly to the generated webpage. I know javascript Blobs can process it, I just don't have any working examples. Any arduino people experts with Javascript and working with Byte arrays? I thought I was the Javascript expert :) , but seem a bit stumped here.

[hpssjellis]

I will close this as I have updated my Arduino IDE library "Portenta Pro Community Solutions" (search for community ). The libary has lots of examples of using the PortentaH7 Vision Shield camera.

See the folder dot36 here for most of them