mocopi

You can use Sony mocopi trackers for body movement motion capture in VTube Studio.

VTube Studio will convert your 3D body movement into tracking parameters that you can easily use in your 2D models, including body movement, head movement, leg movement and arm/hand movement.

VTube Studio can receive and use mocopi data on Windows and macOS.

Example Model

The Sony mocopi team has provided an example Live2D model that is set up for VTube Studio and ready to use with all supported mocopi parameters. If you want to try out mocopi functionality or see an example model as reference for your own model, you can download it for free here:

https://www.sony.co.jp/en/Products/mocopi-dev/en/downloads/DownloadInfo.html#RAYNOS

The file to download is RAYNOS-chan-avatar_for_VTubeStudio_v1.0.0.zip​

Note: The model can be used freely on stream, but there may be some restrictions. Please read the Character License Agreement provided before downloading the model.

Setup

The first step is to set up your mocopi devices to send data to VTube Studio. The mocopi devices cannot directly connect to VTube Studio. You have to use the mocopi iOS/Android app or the Windows PC app.

• mocopi iOS/Android app: Runs on your smartphone and receives data from the mocopi trackers. Then sends that data to VTube Studio over your local network/Wifi.

  • Free.
  • More info: https://www.sony.co.jp/en/Products/mocopi-dev/en/documents/ReceiverPlugin/SendData.html

• mocopi desktop PC app: Runs directly on your PC and receives data from the mocopi trackers. The Sony mocopi sensor data receiver is required for this and has to be connected to your PC via USB. Also sends that data to VTube Studio over your local network (runs on same PC).

  • PC app requires monthly subscription to use USB sensor data receiver.
  • More info: https://www.sony.co.jp/en/Products/mocopi-dev/en/documents/mocopiPC/HowTo_mocopiPC.html

Generally, the way to use those apps is the same: Connect your mocopi devices, then set up the connection to VTube Studio. For the connection to VTube Studio, check your IP and selected port on the mocopi configuration page in VTube Studio. Then, put that IP and port into the 3rd party connection configuration in the mocopi app.

Then, click "start sending data" in the mocopi app and make sure the mocopi receiver is turned on in VTube Studio. You should now see that the connection has been established in VTube Studio and the mocopi tracking data should be available.

Supported Parameters

By default, VTube Studio hides the mocopi tracking parameters in the tracking parameter list when setting up your model. If you want to use mocopi, make sure you turn on "show tracking parameters" in the mocopi config in VTube Studio.

Important note about mocopi tracking parameter ranges: The parameter ranges noted in the list below are just the default min/max values used by VTube Studio for new parameter mappings. For some parameters like MocopiAngleX that have a listed range of [-30, 30] the parameter can go far beyond those values. For parameters with a [-180, 180] range, they cannot go beyond that range: once the parameter reaches one end of the range, it will jump to the other end (360°, centered around 0 with 180° in both directions).

For more details on how the parameters and motion ranges work, it is recommended to look at the mocopi Live2D model provided by Sony (linked above).

After turning on "show tracking parameters" in the VTube Studio mocopi settings, the following parameters will be available:

• MocopiConnected

  • Range: [0, 1]
  • Info: 0 when VTube Studio mocopi receiver is turned off or when no data is currently received. 1 when data is received from mocopi.

• MocopiHipAngleZ

  • Range: [-30, 30]
  • Info: Hip sway angle. - is left, + is right.

• MocopiAngleX

  • Range: [-30, 30]
  • Info: Head left/right rotation (head shake).

• MocopiAngleY

  • Range: [-30, 30]
  • Info: Head up/down rotation (head nod).

• MocopiAngleZ

  • Range: [-30, 30]
  • Info: Head left/right lean.

• MocopiBodyAngleX

  • Range: [-10, 10]
  • Info: Body left/right rotation.

• MocopiBodyAngleY

  • Range: [-10, 10]
  • Info: Body lean forward/backward rotation.

• MocopiBodyAngleZ

  • Range: [-10, 10]
  • Info: Body lean left/right rotation.

• MocopiBodyPositionX

  • Range: [-1, 1]
  • Info: Absolute body position: X.

• MocopiBodyPositionY

  • Range: [-1, 1]
  • Info: Absolute body position: Y.

• MocopiBodyPositionZ

  • Range: [-1, 1]
  • Info: Absolute body position: Z.

• MocopiUpperArmLeftAngleY / MocopiUpperArmRightAngleY

  • Range: [-90, 90]
  • Info: Upper arm forwards/backwards rotation.

• MocopiUpperArmLeftAngleZ / MocopiUpperArmRightAngleZ

  • Range: [-180, 180]
  • Info: Upper arm up/down rotation (360° rotation).

• MocopiLowerArmLeftAngleX / MocopiLowerArmRightAngleX

  • Range: [-180, 180]
  • Info: Wrist rotation (360° rotation).

• MocopiLowerArmLeftAngleY / MocopiLowerArmRightAngleY

  • Range: [-90, 90]
  • Info: Lower arm forwards/backwards rotation.

• MocopiLowerArmLeftAngleZ / MocopiLowerArmRightAngleZ

  • Range: [-180, 180]
  • Info: Lower arm bend (360° rotation). Can go negative when upper arm points inwards.

• MocopiUpperLegLeftAngleY / MocopiUpperLegRightAngleY

  • Range: [-30, 30]
  • Info: Upper leg forwards/backwards rotation.

• MocopiUpperLegLeftAngleZ / MocopiUpperLegRightAngleZ

  • Range: [-30, 30]
  • Info: Upper leg sideways rotation.

• MocopiLowerLegLeftAngleY / MocopiLowerLegRightAngleY

  • Range: [-30, 30]
  • Info: Lower leg forwards/backwards rotation.

• MocopiLowerLegLeftAngleZ / MocopiLowerLegRightAngle

  • Range: [-30, 30]
  • Info: Lower leg sideways rotation.

Combining with Webcam/iOS/Android Tracking

When using mocopi, all normal tracking parameters still work as usual. You can combine it with webcam- or smartphone-based face tracking.

If you want to use mocopi for head rotation tracking, you can use these parameters: MocopiAngleX, MocopiAngleY, MocopiAngleZ

While the mocopi receiver in VTube Studio is turned off or while no data is received from mocopi trackers, these parameters will automatically use any head tracking data available from the webcam or smartphone tracker instead.

All mocopi tracking parameters will fade back to 0 when VTube Studio stops receiving data from mocopi trackers.

Troubleshooting & FAQ

What if I have multiple mocopi trackers sending data at the same time?

VTube Studio will only receive data from one mocopi tracker at a time. Any additional trackers sending in data will be ignored.

When I put the mocopi smartphone app in the background, it stops sending data. What can I do?

On Android, depending on your device, apps will be prevented from accessing bluetooth or the local network when they are not in focus. This is done to save energy. This will cause the mocopi app running in the background to lose connection with either the mocopi trackers (bluetooth) or with VTube Studio (Wifi).

To fix this, make sure the mocopi app has all power saving options turned off (in your Android settings) so unrestricted battery usage in the background is granted.