orz0313-UnityDietSaber

A game like BeatSaber but use webcam as input.

Outline

This is the prototype of a BeatSaber-like game in the process of design thinking , which use webcam and Barracuda to capture and calculate your motion.
※You will need webcamera to input your image
※Without GPU, it does not work fine basically

Install

1. Download .onnx file "Resnet34_3inputs_448x448_20200609.onnx " 1.1 https://digital-standard.com/threedpose/models/Resnet34_3inputs_448x448_20200609.onnx

2. Make a reference to BarracudaRunner.NN_Model
   2.1 Put your file "Resnet34_3inputs_448x448_20200609.onnx" in the path DietSaber/Assets/Scripts/3DPose/Model
   2.2 Open the PlayScene in DietSaber/Assets/Scenes
   2.3 Find and drag the .onnx file into the "NN Model" slot in Gameobject named "BarracudaRunner" in Inspector view.
   

Tutorial

1. Open the MainScene in DietSaber/Assets/Scenes
2. Start Runtime
3. Select background music(you can put your mp3 to the address "Assets\Resources\Mp3")
4. Click the Start button to load PlayScene
5. Slice the cubes
6. press ESC to go back to MainScene

Note

1. Since the trained model "Resnet34_3inputs_448x448_20200609.onnx " could not support multi target estimation , keep only one person in the camera view.
   
2. Keep your whole body in your camera. If your limbs stick out , it will make incorrect calculation.
   2.1 You could make your input image size rearranged by change the number "Video Background Scale" which is in the script "Video Capture" of the Gameobject "MainTexture".
   
3. To avoid misestimation , do not wear cloth which is the same color as background.
   

※Seems that GPU memory leak leads to crash during scene transfers and it could not be easily fixed by manually dispose,not sure this problem is caused by Barracuda workflow or Unity Editor version.

License

Non-commercial use

The videos named as "onegai_darling.mp4" contained in this code for debuging is not copyright free.

UnityChan license is at the followeing URL: https://unity-chan.com/contents/license_en/