Processing failed, potentially unsupported config

[f-alemauro]

Hello,
can someone please explain me why this script does not work? It gives me

[14442C10D13799CD00] [55.888] [ImageManip(3)] [error] Processing failed, potentially unsupported config

Here is the code

import depthai as dai

pipeline = dai.Pipeline()
pipeline.setOpenVINOVersion(version=dai.OpenVINO.Version.VERSION_2021_3)

cam = pipeline.create(dai.node.ColorCamera)
cam.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)
cam.setIspScale(2,3)
cam.setInterleaved(False)
cam.setPreviewSize(1280, 720)    


preview_rotate_manip = pipeline.create(dai.node.ImageManip)
rotation_cfg = dai.RotatedRect()
rotation_cfg.center.x, rotation_cfg.center.y = 0.5, 0.5
rotation_cfg.size.width, rotation_cfg.size.height = 1,1
rotation_cfg.angle = 0
preview_rotate_manip.initialConfig.setCropRotatedRect(rotation_cfg, True)
preview_rotate_manip.setMaxOutputFrameSize(1280*720*3)
cam.preview.link(preview_rotate_manip.inputImage)


preview_zoom_manip = pipeline.create(dai.node.ImageManip)
zoom_cfg = dai.RotatedRect()
zoom_cfg.center.x, rotation_cfg.center.y = 0.5, 0.5
zoom_cfg.size.width, rotation_cfg.size.height = 1,1
preview_zoom_manip.initialConfig.setCropRotatedRect(zoom_cfg, True)
preview_zoom_manip.setMaxOutputFrameSize(1280*720*3)
preview_rotate_manip.out.link(preview_zoom_manip.inputImage)


preview_resize_manip = pipeline.create(dai.node.ImageManip)
preview_resize_manip.initialConfig.setResize(300,300)
preview_resize_manip.initialConfig.setKeepAspectRatio(False)
preview_resize_manip.setMaxOutputFrameSize(1280*720*3)
preview_zoom_manip.out.link(preview_resize_manip.inputImage)


cam_xout = pipeline.create(dai.node.XLinkOut)
cam_xout.setStreamName("RGB")
preview_resize_manip.out.link(cam_xout.input)

with dai.Device(pipeline) as device:
    q_rgb_out = device.getOutputQueue("RGB", maxSize=1, blocking=True)
    while True: 
        in_rgb = q_rgb_out.get()
        cv2.imshow("RGB", in_rgb.getCvFrame())
        cv2.waitKey(5)

I know this is a very weird example, but that would be the base for my application.
I'm on

• Windows 10
• VSCode
• Depthai 2.13.1.0
• Python 3.9.2

Thank you

[f-alemauro]

This is the output with debug on:

[2022-03-10 13:37:35.339] [debug] Library information - version: 2.13.1, commit: 40a79d31881c48bdd20fac8c2fb021de96c775b4 from 2021-11-24 20:04:37 +0100, build: 2021-11-24 22:06:17 +0000
[2022-03-10 13:37:35.342] [debug] Initialize - finished
[2022-03-10 13:37:35.468] [debug] Resources - Archive 'depthai-bootloader-fwp-0.0.15.tar.xz' open: 2ms, archive read: 123ms
[2022-03-10 13:37:35.866] [debug] Resources - Archive 'depthai-device-fwp-d492416efbfe1961620697df8a4470b122bd3cf3.tar.xz' open: 2ms, archive read: 522ms
[2022-03-10 13:37:36.022] [debug] Device - OpenVINO version: 2021.3
[2022-03-10 13:37:36.023] [debug] Patching OpenVINO FW version from 2021.4 to 2021.3
[14442C10D13799CD00] [18.717] [system] [info] Memory Usage - DDR: 0.12 / 358.66 MiB, CMX: 2.05 / 2.50 MiB, LeonOS Heap: 6.78 / 78.59 MiB, LeonRT Heap: 2.88 / 23.84 MiB
[14442C10D13799CD00] [18.717] [system] [info] Temperatures - Average: 41.61 ┬░C, CSS: 42.58 ┬░C, MSS 41.20 ┬░C, UPA: 41.20 ┬░C, DSS: 41.43 ┬░C
[14442C10D13799CD00] [18.717] [system] [info] Cpu Usage - LeonOS 10.07%, LeonRT: 2.21%
[2022-03-10 13:37:37.639] [debug] Schema dump: {"connections":[{"node1Id":0,"node1Output":"preview","node2Id":1,"node2Input":"inputImage"},{"node1Id":1,"node1Output":"out","node2Id":2,"node2Input":"inputImage"},{"node1Id":2,"node1Output":"out","node2Id":3,"node2Input":"inputImage"},{"node1Id":3,"node1Output":"out","node2Id":4,"node2Input":"in"}],"globalProperties":{"calibData":null,"cameraTuningBlobSize":null,"cameraTuningBlobUri":"","leonCssFrequencyHz":700000000.0,"leonMssFrequencyHz":700000000.0,"pipelineName":null,"pipelineVersion":null,"xlinkChunkSize":-1},"nodes":[[0,{"id":0,"ioInfo":{"inputConfig":{"blocking":false,"name":"inputConfig","queueSize":8,"type":3},"inputControl":{"blocking":true,"name":"inputControl","queueSize":8,"type":3},"isp":{"blocking":false,"name":"isp","queueSize":8,"type":0},"preview":{"blocking":false,"name":"preview","queueSize":8,"type":0},"raw":{"blocking":false,"name":"raw","queueSize":8,"type":0},"still":{"blocking":false,"name":"still","queueSize":8,"type":0},"video":{"blocking":false,"name":"video","queueSize":8,"type":0}},"name":"ColorCamera","properties":{"boardSocket":-1,"colorOrder":0,"fp16":false,"fps":30.0,"imageOrientation":-1,"initialControl":{"aeLockMode":false,"aeRegion":{"height":0,"priority":0,"width":0,"x":0,"y":0},"afRegion":{"height":0,"priority":0,"width":0,"x":0,"y":0},"antiBandingMode":0,"autoFocusMode":3,"awbLockMode":false,"awbMode":0,"brightness":0,"chromaDenoise":0,"cmdMask":0,"contrast":0,"effectMode":0,"expCompensation":0,"expManual":{"exposureTimeUs":0,"frameDurationUs":0,"sensitivityIso":0},"lensPosition":0,"lumaDenoise":0,"saturation":0,"sceneMode":0,"sharpness":0},"inputConfigSync":false,"interleaved":false,"ispScale":{"horizDenominator":3,"horizNumerator":2,"vertDenominator":3,"vertNumerator":2},"previewHeight":720,"previewKeepAspectRatio":true,"previewWidth":1280,"resolution":0,"sensorCropX":-1.0,"sensorCropY":-1.0,"stillHeight":-1,"stillWidth":-1,"videoHeight":-1,"videoWidth":-1}}],[1,{"id":1,"ioInfo":{"inputConfig":{"blocking":true,"name":"inputConfig","queueSize":8,"type":3},"inputImage":{"blocking":true,"name":"inputImage","queueSize":8,"type":3},"out":{"blocking":false,"name":"out","queueSize":8,"type":0}},"name":"ImageManip","properties":{"initialConfig":{"cropConfig":{"cropRatio":1.0,"cropRect":{"xmax":0.0,"xmin":0.0,"ymax":0.0,"ymin":0.0},"cropRotatedRect":{"angle":0.0,"center":{"x":0.5,"y":0.5},"size":{"height":1.0,"width":1.0}},"enableCenterCropRectangle":false,"enableRotatedRect":true,"normalizedCoords":true,"widthHeightAspectRatio":1.0},"enableCrop":true,"enableFormat":false,"enableResize":false,"formatConfig":{"flipHorizontal":false,"type":7},"resizeConfig":{"bgBlue":0,"bgGreen":0,"bgRed":0,"enableRotation":false,"enableWarp4pt":false,"enableWarpMatrix":false,"height":0,"keepAspectRatio":true,"lockAspectRatioFill":false,"normalizedCoords":true,"rotationAngleDeg":0.0,"warpBorderReplicate":false,"warpFourPoints":[],"warpMatrix3x3":[],"width":0},"reusePreviousImage":false,"skipCurrentImage":false},"inputConfigSync":false,"numFramesPool":4,"outputFrameSize":2764800}}],[2,{"id":2,"ioInfo":{"inputConfig":{"blocking":true,"name":"inputConfig","queueSize":8,"type":3},"inputImage":{"blocking":true,"name":"inputImage","queueSize":8,"type":3},"out":{"blocking":false,"name":"out","queueSize":8,"type":0}},"name":"ImageManip","properties":{"initialConfig":{"cropConfig":{"cropRatio":1.0,"cropRect":{"xmax":0.0,"xmin":0.0,"ymax":0.0,"ymin":0.0},"cropRotatedRect":{"angle":0.0,"center":{"x":0.5,"y":0.0},"size":{"height":0.0,"width":1.0}},"enableCenterCropRectangle":false,"enableRotatedRect":true,"normalizedCoords":true,"widthHeightAspectRatio":1.0},"enableCrop":true,"enableFormat":false,"enableResize":false,"formatConfig":{"flipHorizontal":false,"type":7},"resizeConfig":{"bgBlue":0,"bgGreen":0,"bgRed":0,"enableRotation":false,"enableWarp4pt":false,"enableWarpMatrix":false,"height":0,"keepAspectRatio":true,"lockAspectRatioFill":false,"normalizedCoords":true,"rotationAngleDeg":0.0,"warpBorderReplicate":false,"warpFourPoints":[],"warpMatrix3x3":[],"width":0},"reusePreviousImage":false,"skipCurrentImage":false},"inputConfigSync":false,"numFramesPool":4,"outputFrameSize":2764800}}],[3,{"id":3,"ioInfo":{"inputConfig":{"blocking":true,"name":"inputConfig","queueSize":8,"type":3},"inputImage":{"blocking":true,"name":"inputImage","queueSize":8,"type":3},"out":{"blocking":false,"name":"out","queueSize":8,"type":0}},"name":"ImageManip","properties":{"initialConfig":{"cropConfig":{"cropRatio":1.0,"cropRect":{"xmax":0.0,"xmin":0.0,"ymax":0.0,"ymin":0.0},"cropRotatedRect":{"angle":0.0,"center":{"x":0.0,"y":0.0},"size":{"height":0.0,"width":0.0}},"enableCenterCropRectangle":false,"enableRotatedRect":false,"normalizedCoords":true,"widthHeightAspectRatio":1.0},"enableCrop":false,"enableFormat":false,"enableResize":true,"formatConfig":{"flipHorizontal":false,"type":7},"resizeConfig":{"bgBlue":0,"bgGreen":0,"bgRed":0,"enableRotation":false,"enableWarp4pt":false,"enableWarpMatrix":false,"height":300,"keepAspectRatio":false,"lockAspectRatioFill":false,"normalizedCoords":true,"rotationAngleDeg":0.0,"warpBorderReplicate":false,"warpFourPoints":[],"warpMatrix3x3":[],"width":300},"reusePreviousImage":false,"skipCurrentImage":false},"inputConfigSync":false,"numFramesPool":4,"outputFrameSize":2764800}}],[4,{"id":4,"ioInfo":{"in":{"blocking":true,"name":"in","queueSize":8,"type":3}},"name":"XLinkOut","properties":{"maxFpsLimit":-1.0,"metadataOnly":false,"streamName":"RGB"}}]]}
[2022-03-10 13:37:37.645] [debug] Asset map dump: {"map":{}}
[14442C10D13799CD00] [18.762] [system] [info] ImageManip internal buffer size '203904'B, shave buffer size '23552'B
[14442C10D13799CD00] [18.762] [system] [info] SIPP (Signal Image Processing Pipeline) internal buffer size '16384'B
[14442C10D13799CD00] [18.797] [system] [info] ColorCamera allocated resources: no shaves; cmx slices: [13-15]
[14442C10D13799CD00] [18.797] [system] [info] ImageManip allocated resources: shaves: [15-15] no cmx slices.
[14442C10D13799CD00] [18.914] [ImageManip(3)] [error] Processing failed, potentially unsupported config
[14442C10D13799CD00] [19.718] [system] [info] Memory Usage - DDR: 64.26 / 358.66 MiB, CMX: 2.29 / 2.50 MiB, LeonOS Heap: 20.71 / 78.59 MiB, LeonRT Heap: 3.73 / 23.84 MiB
[14442C10D13799CD00] [19.718] [system] [info] Temperatures - Average: 42.81 ┬░C, CSS: 43.27 ┬░C, MSS 42.58 ┬░C, UPA: 42.58 ┬░C, DSS: 42.81 ┬░C
[14442C10D13799CD00] [19.718] [system] [info] Cpu Usage - LeonOS 12.04%, LeonRT: 6.38%
[14442C10D13799CD00] [20.719] [system] [info] Memory Usage - DDR: 64.26 / 358.66 MiB, CMX: 2.29 / 2.50 MiB, LeonOS Heap: 20.71 / 78.59 MiB, LeonRT Heap: 3.73 / 23.84 MiB
[14442C10D13799CD00] [20.719] [system] [info] Temperatures - Average: 43.16 ┬░C, CSS: 44.18 ┬░C, MSS 43.04 ┬░C, UPA: 42.81 ┬░C, DSS: 42.58 ┬░C
[14442C10D13799CD00] [20.719] [system] [info] Cpu Usage - LeonOS 7.00%, LeonRT: 0.33%```

[themarpe]

Hi @f-alemauro
On the first glance it seems like the crop is out of the picture? This is what might be causing the issues.

Regardless, feel free to try the same on the image_manip_refactor branch

[f-alemauro]

Thank @themarpe.
How could the crop be out of the picture? I'm using normalized coordinates... And I'm not even cropping indeed.

rotation_cfg = dai.RotatedRect()
rotation_cfg.center.x, rotation_cfg.center.y = 0.5, 0.5
rotation_cfg.size.width, rotation_cfg.size.height = 1,1
rotation_cfg.angle = 0

[themarpe]

@f-alemauro

You are cropping in the first ImageManip - can you try a smaller rect, maybe:

review_rotate_manip = pipeline.create(dai.node.ImageManip)
rotation_cfg = dai.RotatedRect()
rotation_cfg.center.x, rotation_cfg.center.y = 0.5, 0.5
rotation_cfg.size.width, rotation_cfg.size.height = 0.5, 0.5
rotation_cfg.angle = 0
preview_rotate_manip.initialConfig.setCropRotatedRect(rotation_cfg, True)
preview_rotate_manip.setMaxOutputFrameSize(1280*720*3)
cam.preview.link(preview_rotate_manip.inputImage)

Or just regular crop if you don't need a rotation.

Also, can you test against the before mentioned image_manip_branch?
The following will install the python library from that branch:

git checkout image_manip_refactor
python3 examples/install_dependencies.py

[f-alemauro]

this is strange:

import cv2
import depthai as dai

pipeline = dai.Pipeline()
pipeline.setOpenVINOVersion(version=dai.OpenVINO.Version.VERSION_2021_3)

cam = pipeline.create(dai.node.ColorCamera)
cam.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)
cam.setIspScale(2,3)
cam.setInterleaved(False)
cam.setPreviewSize(1280, 720)    


preview_rotate_manip = pipeline.create(dai.node.ImageManip)
rotation_cfg = dai.RotatedRect()
rotation_cfg.center.x, rotation_cfg.center.y = 0.5, 0.5
rotation_cfg.size.width, rotation_cfg.size.height = 0.5,0.5
rotation_cfg.angle = 0
preview_rotate_manip.initialConfig.setCropRotatedRect(rotation_cfg, True)
preview_rotate_manip.setMaxOutputFrameSize(1280*720*3)
cam.preview.link(preview_rotate_manip.inputImage)


preview_zoom_manip = pipeline.create(dai.node.ImageManip)
zoom_cfg = dai.RotatedRect()
zoom_cfg.center.x, rotation_cfg.center.y = 0.5, 0.5
zoom_cfg.size.width, rotation_cfg.size.height = 0.5,0.5
rotation_cfg.angle = 0
preview_zoom_manip.initialConfig.setCropRotatedRect(zoom_cfg, True)
preview_zoom_manip.setMaxOutputFrameSize(1280*720*3)
preview_rotate_manip.out.link(preview_zoom_manip.inputImage)

cam_xout = pipeline.create(dai.node.XLinkOut)
cam_xout.setStreamName("RGB")
preview_zoom_manip.out.link(cam_xout.input)

with dai.Device(pipeline) as device:
    device.setLogLevel(dai.LogLevel.DEBUG)
    device.setLogOutputLevel(dai.LogLevel.DEBUG)
    q_rgb_out = device.getOutputQueue("RGB", maxSize=1, blocking=True)
    while True: 
        in_rgb = q_rgb_out.get()
        cv2.imshow("RGB", in_rgb.getCvFrame())
        cv2.waitKey(5)

This code just produce an image with a weird shape (1, 321, 3)

I'll try with that branch. But I would like to also continue these tests on regular branch to help you solving this bug :)
Is it ok for you?

[f-alemauro]

Even this script fails:

import cv2
import depthai as dai

pipeline = dai.Pipeline()
pipeline.setOpenVINOVersion(version=dai.OpenVINO.Version.VERSION_2021_3)

cam = pipeline.create(dai.node.ColorCamera)
cam.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)
cam.setIspScale(2,3)
cam.setInterleaved(False)
cam.setPreviewSize(1280, 720)    


preview_zoom_n_rotate_manip = pipeline.create(dai.node.ImageManip)
rgbRr = dai.RotatedRect()
rgbRr.center.x, rgbRr.center.y = cam.getPreviewWidth()//2, cam.getPreviewHeight()//2
rgbRr.size.width, rgbRr.size.height = cam.getPreviewWidth(), cam.getPreviewHeight()
rgbRr.angle = 0
preview_zoom_n_rotate_manip.initialConfig.setCropRotatedRect(rgbRr, False)
preview_zoom_n_rotate_manip.initialConfig.setResizeThumbnail(320,320, 0, 0, 0)
preview_zoom_n_rotate_manip.initialConfig.setKeepAspectRatio(False)
preview_zoom_n_rotate_manip.setMaxOutputFrameSize(300*300*3)
cam.preview.link(preview_zoom_n_rotate_manip.inputImage)

cam_xout = pipeline.create(dai.node.XLinkOut)
cam_xout.setStreamName("RGB")
preview_zoom_n_rotate_manip.out.link(cam_xout.input)

with dai.Device(pipeline) as device:
    device.setLogLevel(dai.LogLevel.DEBUG)
    device.setLogOutputLevel(dai.LogLevel.DEBUG)
    q_rgb_out = device.getOutputQueue("RGB", maxSize=1, blocking=True)
    while True: 
        in_rgb = q_rgb_out.get()
        frame = in_rgb.getCvFrame()
        print(frame.shape)
        cv2.imshow("RGB", in_rgb.getCvFrame())
        cv2.waitKey(5)

Could you please try that and let me know if thats fail?
Unfortunately there is no way I can build the pipeline I would like: I would like to have:

1. center cropping
2. rotation
3. resize (with or without letterboxing)

and I would like to control the setting live, so for example I would like to change the cropping size, the rotation and switching from letterboxing to non-letterboxing.

[themarpe]

Thanks @f-alemauro for exposing the bugs - current ImageManip has a bunch of issues which we are in the works of replacing/fixing with the refactored ImageManip.
I'll raise this internally and we'll most likely have this fixed in image_manip_refactor branch which will be mainlined shortly after.

CC: @Erol444 can you recheck the above if that should work in current ImageManip already

[OanaMariaVatavu]

Hi, @f-alemauro!

I had tried your scripts on the latest develop, and they're run just fine, with little corrections:

1. at the first and second scripts, you set wrong the second crop:

zoom_cfg.center.x, rotation_cfg.center.y = 0.5, 0.5
zoom_cfg.size.width, rotation_cfg.size.height = 1,1

should be

zoom_cfg.center.x, zoom_cfg.center.y = 0.5, 0.5
zoom_cfg.size.width, zoom_cfg.size.height = 1,1

2. at the third scrip, you set wrong the total output size:

preview_zoom_n_rotate_manip.setMaxOutputFrameSize(300*300*3)

should be

 preview_zoom_n_rotate_manip.setMaxOutputFrameSize(320*320*3)

[f-alemauro]

Hello @OanaMariaVatavu
thank you for your feedback. Really appreciate your help!

First and second script now works perfectly.

Third script still have an error in the code:

preview_zoom_n_rotate_manip.initialConfig.setCropRotatedRect(rgbRr, False)

that should have True instead of False (am I correct?)

preview_zoom_n_rotate_manip.initialConfig.setCropRotatedRect(rgbRr, True)

but still fails with depthai 2.13.1.0

[14442C10D13799CD00] [49.964] [ImageManip(1)] [error] Output image width is bigger (65792 pixels) than maximum output frame width supported by ImageManip (1920 pixels) - skipping frame

[OanaMariaVatavu]

Hello @OanaMariaVatavu thank you for your feedback. Really appreciate your help!

First and second script now works perfectly.

Third script still have an error in the code:

preview_zoom_n_rotate_manip.initialConfig.setCropRotatedRect(rgbRr, False)

that should have True instead of False (am I correct?)

preview_zoom_n_rotate_manip.initialConfig.setCropRotatedRect(rgbRr, True)

but still fails with depthai 2.13.1.0

[14442C10D13799CD00] [49.964] [ImageManip(1)] [error] Output image width is bigger (65792 pixels) than maximum output frame width supported by ImageManip (1920 pixels) - skipping frame

@f-alemauro
If you set True for the second parameter of the setCropRotatedRect function, it will be required that the crop position and resolution be in a normalized format, so, in the current configuration, it is ok to be False.
Regarding the error you received, it seems to be a bug in the setResizeThumbnail API. As a workaround, I suggest you use the setResize API instead.
This version of script works on my side:

import cv2
import depthai as dai

pipeline = dai.Pipeline()
pipeline.setOpenVINOVersion(version=dai.OpenVINO.Version.VERSION_2021_3)

cam = pipeline.create(dai.node.ColorCamera)
cam.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)
cam.setIspScale(2,3)
cam.setInterleaved(False)
cam.setPreviewSize(1280, 720)    


preview_zoom_n_rotate_manip = pipeline.create(dai.node.ImageManip)
rgbRr = dai.RotatedRect()
rgbRr.center.x, rgbRr.center.y = cam.getPreviewWidth()//2, cam.getPreviewHeight()//2
rgbRr.size.width, rgbRr.size.height = cam.getPreviewWidth(), cam.getPreviewHeight()
rgbRr.angle = 0
preview_zoom_n_rotate_manip.initialConfig.setCropRotatedRect(rgbRr, False)
preview_zoom_n_rotate_manip.initialConfig.setResize(320,320)
preview_zoom_n_rotate_manip.initialConfig.setKeepAspectRatio(False)
preview_zoom_n_rotate_manip.setMaxOutputFrameSize(320*320*3)
cam.preview.link(preview_zoom_n_rotate_manip.inputImage)

cam_xout = pipeline.create(dai.node.XLinkOut)
cam_xout.setStreamName("RGB")
preview_zoom_n_rotate_manip.out.link(cam_xout.input)

with dai.Device(pipeline) as device:
    device.setLogLevel(dai.LogLevel.DEBUG)
    device.setLogOutputLevel(dai.LogLevel.DEBUG)
    q_rgb_out = device.getOutputQueue("RGB", maxSize=1, blocking=True)
    while True: 
        in_rgb = q_rgb_out.get()
        frame = in_rgb.getCvFrame()
        print(frame.shape)
        cv2.imshow("RGB", in_rgb.getCvFrame())
        cv2.waitKey(5)

In the current state of the image_manip_refactor branch, which we'll be integrating into develop/mainline soon, this issue is not present anymore.
Please let me know if the above workaround suits for you.

[f-alemauro]

@f-alemauro
If you set True for the second parameter of the setCropRotatedRect function, it will be required that the crop position and resolution be in a normalized format, so, in the current configuration, it is ok to be False.

Ok, thank you for the correction.

In the current state of the image_manip_refactor branch, which we'll be integrating into develop/mainline soon, this issue is not present anymore.
Please let me know if the above workaround suits for you.

Well, not at all. I mean, I'm using setResizeThumbnail because I would like to keep the aspect ratio, resize the image as required by the subsequent NN node (i.e. 300x300 square), and preserve the whole field of view of the sensor. Without letterboxing FOV would be reduced as resizing from 1920x1080 to 300x300.
Am I right?

[OanaMariaVatavu]

Well, not at all. I mean, I'm using setResizeThumbnail because I would like to keep the aspect ratio, resize the image as required by the subsequent NN node (i.e. 300x300 square), and preserve the whole field of view of the sensor. Without letterboxing FOV would be reduced as resizing from 1920x1080 to 300x300.
Am I right?

Yes, @f-alemauro , you are partially right.
Using the setResize config you will resize from input resolution for ImageManip(because of this link cam.preview.link(preview_zoom_n_rotate_manip.inputImage), the input for ImageManip node will be the preview output from ColorCamera node = 1280x720 resolution) to 320x320 output resolution, with distortion the aspect ratio(16:9 -> 1:1). With setResizeThumbnail you will keep the aspect ratio as you said.

I will investigate the root cause of the bug caused by enabling the setResizeThumbnail config, because I see that, if add some prints on leon, the error is gone, which seems to be a stack issue. Want to be sure that this bug is not hidden also in the image_manip_refactor branch.

[OanaMariaVatavu]

@f-alemauro
Looking deeper into the implementation, it seems that the setResizeThumbnail and setCropRotatedRect APIs are mutually exclusive, so they can't be used in the same ImageManip pipeline configuration. In this context, you have received this error [14442C10D13799CD00] [49.964] [ImageManip(1)] [error] Output image width is bigger (65792 pixels) than maximum output frame width supported by ImageManip (1920 pixels) - skipping frame because, in this unsupported configuration, some device class fields were uninitialized. Using the mainline version, I can suggest you only to use those APIs in different ImageManip pipelines.
In the image_manip_refactor version, the mentioned error is not reproducible because those device class fields are initialized, but using those 2 APIs in the same config, the resizing is made with distortion of the aspect ratio 16:9 -> 1:1(the field of view is kept, but having a "flattened" horizontally appearance. @themarpe, does it makes sense to have support for setResizeThumbnail and setCropRotatedRect APIs in the same config? If yes, I will add this support in the image_manip_refactor branch.

[f-alemauro]

@OanaMariaVatavu
That's a great feedback! Thank you!
So, are you stating that in the mainline version I cannot have a singe ImageManip node that calls both setResizeThumbnail and setCropRotatedRect?
And also that I should use two ImageManip nodes, one after the other, the first calling setCropRotatedRect and the second setResizeThumbnail? This would be a great deal!