Tracking Objects Example - camera setup and coordinate system issues

[koreahjg1]

Hello contributors,

I used your example of object tracker to build my iOS app with object tracker in Xamarin C#. It is very cool and useful, but I found some issues and bugs as below and want to let you know the solutions for them. I hope this can help you improving the example.

1. Camera setup
   Your approach to access the frames from the camera using a callback method given to the VideoOutDelegate as a parameter causes a significant delay (only ~3 FPS) and the frame in the callback method is cropped at the top and bottom of the frame if the camera provides 1920x1080 frames or higher resolution. (see line 106 of the file in the link below)
   https://github.com/xamarin/ios-samples/blob/master/ios11/VisionObjectTracker/ObjectTracker/ViewController.cs

I recommend accessing the frames from a built-in callback method in Xamarin C# as described in the link below.
https://docs.microsoft.com/en-us/xamarin/ios/user-interface/controls/intro-to-manual-camera-controls
This method allows you to access the frames x10 faster than your approach (~30FPS) and the frame can include the full image whatever resolution you use from the camera input.

2. Converting the coordinates from UI to object tracker
   Your code (line 70 in https://github.com/xamarin/ios-samples/blob/master/ios11/VisionObjectTracker/ObjectTracker/ObjectTracker.cs) is using an incorrect way to convert the coordinate system. The correct way to convert the coordinate the y coordinate is 1-box.y-box.width where box is the bounding box in the user interface.

[chamons]

Thanks for the feedback!

If you'd like, you are very welcome to post a PR against this repo with your suggested changes. Else the sample may be tweaked with your suggestions in the future.

[rolfbjarne]

Microsoft support for Xamarin will end on May 1, 2024 for all Xamarin SDKs. In preparation for this, all issues and PRs in this repository are being closed.