Skip to content

v0.6

Latest
Latest

Choose a tag to compare

View all tags
@j20001970 j20001970 released this 16 Sep 12:21
· 41 commits to master since this release
v0.6
c8795da

GDMP v0.6 release! It's the Godot Engine plugin for MediaPipe framework and solutions, and published a new release shortly after the announcement of Godot Engine v4.5. There must have been people who want to deploy ML features on game engines, so if they find game engines, they find Godot, and if they find Godot, they find GDMP. Oh yeah, it's all coming together.

What's changed in this release

General

  • godot-cpp updated to v4.4
  • MediaPipe updated to v0.10.26
  • OpenCV is now built statically by default on all platforms, carrying the size of additional dynamic libraries is no longer required.
  • Continue the progress of deprecating MediaPipeCameraHelper, camera helper now build stub driver by default.
    Different from what was planned in the previous release note, this also affect GDNative branch of GDMP due to mainteinance burden of different build work flows. We encourage migrating to Godot 4.x if camera feeds functionality is needed. Camera helper will not be included in next release.

New classes

  • MediaPipeNodeSource and MediaPipeNodeDestination are added to represent the connection ports of MediaPipeGraphBuilder and MediaPipeGraphNode.
  • MediaPipeClassifierOptions and MediaPipeEmbedderOptions are added to encapsulate common task initialize parameters for classifier and embedder tasks.

New tasks

The following text and audio tasks have been added:

  • MediaPipeLanguageDetector
  • MediaPipeTextClassifier
  • MediaPipeTextEmbedder
  • MediaPipeAudioClassifier
  • MediaPipeAudioEmbedder

Framework improvements

  • MediaPipePacket now use different setters for different types.
  • A type registration mechanism around MediaPipePacket is implemented. Different classes can register packet type validation and getter methods for corresponding data type. MediaPipePacket::get iterate over registered types and return the first matching type.
  • Paritial implement MediaPipeProto setter for repeated fields.
  • Several calculators are included in the library, you can use these calculators with MediaPipeGraphBuilder and MediaPipeTaskRunner.

Tasks improvements

  • Task containers can now create proto of corresponding type.
  • The properties of MediaPipeGestureRecognizerResult are now converted from proto to task containers.

Android

  • Android plugin AAR and native libraries are built separately (again), and AAR is no longer required to use GDMP.
    The purpose of Android plugin was to provide GDMPCameraHelper Java class on Android, since MediaPipeCameraHelper is deprecated, we are removing the reliance of Android plugin.

iOS

  • iOS library are now built as xcframework to support both device and simulator on arm64.

Linux

  • arm64 support are now included.

Web

  • OpenCV is enabled, most vision tasks should work now.
    Image Segmenter is still not available as it require WebGPU to be enabled, we will try to address this in next release.

What's next?

Next release will focus on trying to bring GPU support to more platforms, especially WebGPU support on Web to enable all tasks, generative AI tasks (i.e. Image Generator, LLM Inference) are also on the roadmap.

Full Changelog: v0.5...v0.6

Assets 3
Loading