A Unity demo app for iOS using ARKit and Core ML running MobileNet SSD object detection
Switch branches/tags
Nothing to show
Clone or download
richardstechnotes
Latest commit 1526ff8 Nov 20, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
Assets Added more comments to Detection.cs Nov 20, 2018
ProjectSettings Initial commit Nov 20, 2018
UnityPackageManager Initial commit Nov 20, 2018
.gitignore Initial commit Nov 20, 2018
LICENSE Initial commit Nov 20, 2018
README.md Initial commit Nov 20, 2018

README.md

UnityARKitDetection

Introduction

This is a Unity demo app for iOS that uses ARKit and Core ML running MobileNet SSD object detection. It is based on a combination of two other repos:

It would have been impossible to put this together without these excellent projects - thanks!

It doesn't do much with ARKit at this stage - this is for future applications.

The project was tested using Unity 2017.4.5f1 and an 11 inch iPad Pro. It should also work with later versions of Unity. Using Core ML requires iOS 11.0 or above.

I will be the first to say that my modifications were put together quickly and may not be the most elegant but it does seem to work not too badly!

Build

Select the Detection scene and build for iOS. In the Player Settings, change the bundle identifier to match that on your iOS Provisioning Profile - saves time in Xcode.

Open the generated project in Xcode and set the Team correctly. The Core ML model needs to be added at this stage. Easiest way to do this is to clone https://github.com/vonholst/SSDMobileNet_CoreML and use SSDMobileNet/SSDMobileNet/ssd_mobilenet_feature_extractor.mlmodel. This file needs to be added to the Xcode project using the File, Add files to... command. Drag and drop does not seem to work. Unfortunately, every time the project is regenerated by Unity, the model file will need to be re-added.

If you get linker errors for ARKit when trying to build the XCode project, it means that the particular version of Unity you are using did not include the ARKit.framework in linked binaries for the generated project. Go to Build Phases / Link Binary With Libraries, and add ARKit.framework.

Run

Download to an iOS device and run. It does sometimes crash at line 49 of VisionBridge.mm if run from Xcode but this only happens occasionally.

An example of the code running can be found at https://richardstechnotes.com/2018/11/19/mobilenet-ssd-object-detection-with-unity-arkit-and-core-ml/

License

MIT license for original code and modification to the existing repos.