MediaPipe4U is an UnrealEngine plugin that integrated Google Mediapipe technology for capturing human motion (include body, fingers, face) and puppeteering 3D avatar through webcam, videos, and images in realtime.
| Unreal Engine | China Site | Global Site | Update |
|---|---|---|---|
| UE 5.0 | cowtransfer (China) | OneDrive | 2023-04-03 |
| UE 5.1 | cowtransfer (China) | OneDrive | 2023-04-03 |
Because the plugin is precompiled and contains a large number of C++ link symbols and debug symbols, it will cost 5G disk space after decompression (most files are UE-generated binaries in Intermediate).
Don't need to worry about disk usage, this is just disk usage during development, after the project is packaged, the plug-in disk usage is 200M only (most files are GStreamer dynamic library).
Demo Project
The demo project does not contain plugins, you need to download the plugin and copy content to the project's plugins folder to run.
Video Tutorials (English)
Video Tutorials (Chinese)
If you have any questiongs, please check FAQ first. The problems listed there may be also yours. If you can’t find an answer in the FAQ, please post an issue. Private message or emal may cause the question to be mised .
Since the Windows version of MediaPipe does not support GPU inference, Windows relies on the CPU to inferring human pose estimation (see MediaPipe offical site for more details).
Hardware Evnrioment
CPU： AMD 3600 CPU
RAM: 32GB
GPU: Nvidia 1660s
Evaluation
Frame Rate： 18-24 fps
CPU usage：20% (Based on DEMO project)
Follow Me
- [fix] LOD 变化时，如果骨骼变化，引发程序崩溃
- [fix] MediaPipe 表情解算 BrowDown 时右眼误差
- [fix] UE Editor 中 LiveLink管理器状态和 MediaPipeFaceLinkActor 不一致
- [fix] MediaPipe4U LiveLink Source 无法删除的问题
Google Translate
- [fix] When the LOD changes, if the skeleton changes, it causes the program to crash
- [fix] There is an error in the right eye when MediaPipe face solves BrowDown
- [fix] The LiveLink manager state in the UE Editor is inconsistent with the MediaPipeFaceLinkActor
- [fix] MediaPipe4U LiveLink Source could not be removed