Strange artifacts showing on right side of preview image

[ConnorChristie]

I just converted the vehicle-detection-adas-002 to the blob format and tried running it through a similar pipeline as the second tutorial. It partially works and it does manage to recognize vehicles quite accurately but seems to show some weird green and pink bars on the right hand side of the preview image:

I did re-compile the depthai-api module to run on my board which is a 64 bit version of linux: Linux NanoPi-Fire3 4.4.172-s5p6818 #1 SMP PREEMPT Mon Nov 11 11:24:09 CST 2019 aarch64 aarch64 aarch64 GNU/Linux

I am also using python 3.7 and opencv 3.4.9 if that makes any difference?

Here is the output from depthai after running the python script:

python3 vehicle_detection.py
Using Custom Calibration File: depthai.calib
XLink initialized.
Sending device firmware "cmd_file": /home/pi/depthai-python-extras/depthai.cmd
Successfully connected to device.
Loading config file
watchdog started Attempting to open stream config_d2h
1000
Successfully opened stream config_d2h with ID #0!
Closing stream config_d2h: ...
Closing stream config_d2h: DONE.
WARNING: Version (dev) does not match (unknown & 8c6328c8acd1086542b47d09b4b192ec1143914c)
CNN configurations read: vehicle-detection-adas-0002.json
depthai: Calibration file is not specified, will use default setting;
config_h2d json:
{"_board":{"_homography_right_to_left":[0.988068163394928,0.0029474012553691864,5.067617416381836,-0.008765067905187607,0.9921473264694214,-8.795275688171387,-8.449587767245248e-06,-3.603489403758431e-06,1.0]},"_load_inBlob":true,"_pipeline":{"_streams":[{"name":"metaout"},{"name":"previewout"},{"name":"meta_d2h"}]},"ai":{"calc_dist_to_bb":false},"board":{"clear-eeprom":false,"left_fov_deg":69.0,"left_to_rgb_distance_m":0.0,"left_to_right_distance_m":0.03500000014901161,"override-eeprom":false,"store-to-eeprom":false,"swap-left-and-right-cameras":false},"depth":{"padding_factor":0.30000001192092896}}
Attempting to open stream config_h2d
Successfully opened stream config_h2d with ID #1!
Writing 1000 bytes to config_h2d
!!! XLink write successful: config_h2d (1000)
Closing stream config_h2d: ...
Closing stream config_h2d: DONE.
Read: 4248000
Attempting to open stream inBlob
Successfully opened stream inBlob with ID #2!
Writing 4248000 bytes to inBlob
!!! XLink write successful: inBlob (4248000)
Closing stream inBlob: ...
Closing stream inBlob: DONE.
depthai: done sending Blob file vehicle-detection-adas-0002.blob
Attempting to open stream outBlob
Successfully opened stream outBlob with ID #0!
Closing stream outBlob: ...
Closing stream outBlob: DONE.
CNN input width: 672
CNN input height: 384
CNN input num channels: 3
Host stream start:meta_d2h
Opening stream for read: meta_d2h
Attempting to open stream meta_d2h
Successfully opened stream meta_d2h with ID #4!
Starting thread for stream: meta_d2h
Started thread for stream: meta_d2h
Host stream start:metaout
Opening stream for read: metaout
Attempting to open stream metaout
Successfully opened stream metaout with ID #5!
Starting thread for stream: metaout
Started thread for stream: metaout
Host stream start:previewout
Opening stream for read: previewout
Attempting to open stream previewout
Successfully opened stream previewout with ID #0!
Starting thread for stream: previewout
Started thread for stream: previewout
depthai: INIT OK!

[Luxonis-Brandon]

Hi @ConnorChristie ,

Yes we have seen that too on some models like at the bottom here:
https://docs.luxonis.com/tutorials/openvino_model_zoo_pretrained_model/#trying-other-models

There’s some alias occurring and we’re working to figure out why. If you notice those pixels are a strange remapping from the left side of the image where red and green come over but not blue.

Sorry we should have had this up as a known issue.

[Luxonis-Brandon]

For documentation purposes another model that does this is face-detection-adas-0001.

[Luxonis-Brandon]

Hi @ConnorChristie ,

Just wanted to update that this is on our shortlist, but we're cranking through some necessary improvements first. #74 #77 #76 #66 #62 in addition to updating our tensor-output processing so that it can be dynamic for support of other tensor formats.

And we have h.264 1080p video output support and 12MP still image support output to go to PR tomorrow.

Thoughts?

Thanks,
Brandon

[Luxonis-Brandon]

Hi @ConnorChristie ,

So the fix for this is in this PR (with some other fixes and related functionalities):
#85

[ConnorChristie]

@Luxonis-Brandon After updating to the latest version (more specifically, updating depthai.cmd) I am now getting this error after starting:
Nvm, I wasn't building the latest python module as indicated by the commit I was running under (3b41050). It's all better now :)

Using Custom Calibration File: depthai.calib
Using Arguments= {'config_overwrite': None, 'board': None, 'field_of_view': 71.86, 'rgb_field_of_view': 68.7938, 'baseline': 9.0, 'rgb_baseline': 2.0, 'swap_lr': True, 'store_eeprom': False, 'clear_eeprom': False, 'override_eeprom': False, 'device_id': '', 'dev_debug': None, 'force_usb2': None, 'cnn_model': 'mobilenet-ssd', 'disable_depth': False, 'draw_bb_depth': False, 'full_fov_nn': False, 'streams': ['metaout', 'previewout'], 'video': None}
depthai.__version__ == 0.0.10a
depthai.__dev_version__ == 3b41050fe9c2453408e24c49e15f5ac561595cf6
XLink initialized.
Sending device firmware "cmd_file": /home/pi/depthai/depthai.cmd
Successfully connected to device.
Loading config file
watchdog started Attempting to open stream config_d2h
3000
Successfully opened stream config_d2h with ID #0!
Closing stream config_d2h: ...
Closing stream config_d2h: DONE.
WARNING: Version (dev) does not match (30295b558351cd030408e12a220cdd55b5fb450e & 3b41050fe9c2453408e24c49e15f5ac561595cf6)
EEPROM data: invalid / unprogrammed
Available streams: ['meta_d2h', 'left', 'right', 'disparity', 'depth_sipp', 'metaout', 'previewout', 'jpegout', 'video']
CNN configurations read: /home/pi/depthai/resources/nn/mobilenet-ssd/mobilenet-ssd_depth.json
config_h2d json:
{"_board":{"_homography_right_to_left":[1.0012089014053345,0.006868398282676935,-5.654909610748291,-0.003709208220243454,1.0076549053192139,-8.794636726379395,-4.944150077790255e-06,8.152581358444877e-06,1.0]},"_load_inBlob":true,"_pipeline":{"_streams":[{"name":"metaout"},{"name":"previewout"}]},"ai":{"calc_dist_to_bb":true},"board":{"clear-eeprom":false,"left_fov_deg":71.86000061035156,"left_to_rgb_distance_m":0.019999999552965164,"left_to_right_distance_m":0.09000000357627869,"name":"","override-eeprom":false,"revision":"","rgb_fov_deg":68.7938003540039,"stereo_center_crop":false,"store-to-eeprom":false,"swap-left-and-right-cameras":true},"depth":{"padding_factor":0.30000001192092896}}
Attempting to open stream config_h2d
Successfully opened stream config_h2d with ID #1!
Writing 1000 bytes to config_h2d
!!! XLink write successful: config_h2d (1000)
Closing stream config_h2d: ...
Closing stream config_h2d: DONE.
Creating observer stream host_capture: ...
Attempting to open stream host_capture
Successfully opened stream host_capture with ID #0!
Creating observer stream host_capture: DONE.
Read: 14514560
Attempting to open stream inBlob
E: [global] [    991829] [python3] XLinkOpenStream:95   Cannot find stream id by the "inBlob" name
E: [global] [    991830] [python3] XLinkOpenStream:96   Max streamId reached!
Failed to open stream inBlob ! Retrying ...
E: [global] [    991885] [python3] XLinkOpenStream:95   Cannot find stream id by the "inBlob" name
E: [global] [    991885] [python3] XLinkOpenStream:96   Max streamId reached!
Failed to open stream inBlob ! Retrying ...
E: [global] [    991885] [Scheduler00Thr] dispatcherEventSend:53        Write failed (header) (err -1) | event XLINK_CREATE_STREAM_REQ

E: [xLink] [    991885] [Scheduler00Thr] sendEvents:1036        Event sending failed
E: [global] [    991940] [python3] XLinkOpenStream:95   Cannot find stream id by the "inBlob" name
E: [global] [    991940] [python3] XLinkOpenStream:96   Max streamId reached!
Failed to open stream inBlob ! Retrying ...
E: [global] [    991941] [Scheduler00Thr] dispatcherEventSend:53        Write failed (header) (err -1) | event XLINK_CREATE_STREAM_REQ

E: [xLink] [    991942] [Scheduler00Thr] sendEvents:1036        Event sending failed
E: [global] [    991996] [python3] XLinkOpenStream:95   Cannot find stream id by the "inBlob" name
E: [global] [    991996] [python3] XLinkOpenStream:96   Max streamId reached!
Failed to open stream inBlob ! Retrying ...
E: [global] [    991997] [Scheduler00Thr] dispatcherEventSend:53        Write failed (header) (err -1) | event XLINK_CREATE_STREAM_REQ

E: [xLink] [    991997] [Scheduler00Thr] sendEvents:1036        Event sending failed
E: [global] [    992051] [python3] XLinkOpenStream:95   Cannot find stream id by the "inBlob" name
E: [global] [    992051] [python3] XLinkOpenStream:96   Max streamId reached!
Failed to open stream inBlob ! Retrying ...
E: [global] [    992053] [Scheduler00Thr] dispatcherEventSend:53        Write failed (header) (err -1) | event XLINK_CREATE_STREAM_REQ

E: [xLink] [    992054] [Scheduler00Thr] sendEvents:1036        Event sending failed
Stream not opened: inBlob
depthai: pipelineConfig write error;
Pipeline is not created.
Stopping threads: ...
Stopping threads: DONE 0.000s.
Closing all observer streams: ...
Closing stream host_capture: ...
Closing stream host_capture: DONE.
E: [global] [    992245] [Scheduler00Thr] dispatcherEventSend:53        Write failed (header) (err -4) | event XLINK_CLOSE_STREAM_REQ

Closing all observer streams: DONE.
Reseting device: 0.
E: [xLink] [    992245] [Scheduler00Thr] sendEvents:1036        Event sending failed
E: [global] [    992245] [Scheduler00Thr] dispatcherEventSend:53        Write failed (header) (err -4) | event XLINK_RESET_REQ

E: [xLink] [    992245] [Scheduler00Thr] sendEvents:1036        Event sending failed
Reseting: DONE.
Return code:3

[alex-luxonis]

Hi @ConnorChristie
Please update the python lib (.so) as well, or rebuild it from sources.
The device firmware (.cmd) and python lib are interdependent, we will add some better checks and error messages if they are incompatible (different versions).
You can use the current master branch, the above PR was merged.

Edit: Ah, great! Thanks for the update!

[ConnorChristie]

@alex-luxonis Yup, figured it out. Thought I was building latest but it turns out I was not. Everything works as expected now, thanks!

[ConnorChristie]

This has been fixed by #85