Skip to content

An example of using Tensorflow and ONNX models with Unity Barracuda inference engine for image classification and object detection.

License

Notifications You must be signed in to change notification settings

Syn-McJ/TFClassify-Unity-Barracuda

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

What

This is an example of using models trained with TensorFlow or ONNX in Unity application for image classification and object detection. It uses Barracuda inference engine - please note that Barracuda is still in development preview and changes frequently.

More details in my blogpost.

Classify results:

Detect results:

If you're looking for similar example but using TensorflowSharp plugin instead of Barracuda, see my previous repo.

How

You'll need Unity 2019.3 or above. Versions 2019.2.x seem to have a bug with WebCamTexture and Vulkan that causes memory leak.

  • Open the project in Unity.
  • Install Barracuda 0.4.0-preview plugin from Window -> Package Maanger (the sample didn't work with 0.5.0-preview last time I checked it).
  • In Edit -> Player Settings -> Other settings make sure that you have Vulkan in Graphics APIs for Android or Metal for iOS (remove Auto Graphics API check if neccessary). Barracuda is also suppose to work with OpenGLES3 + ES3.1, but I didn't have any luck with it.
  • Open Classify or Detect scene in Assets folder.
  • Make sure that Classifier or Detector object has Model file and Labels file set.
  • in File -> Build settings choose one of the scenes and hit Build and run.

For iOS, you might need to fix team settings and privacy request message for camera in Xcode.

Barracuda repository might be found here.

How to use your own model

There are limited range of neural network architectures that I managed to get to run with Barracuda. Read my blogpost to see what's working and what isn't.

Notes

I'm not a Unity expert, so if you found any problems with this example feel free to open an issue.

About

An example of using Tensorflow and ONNX models with Unity Barracuda inference engine for image classification and object detection.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages