Unity VFX Graph real-time point cloud streaming via Record3D demo
2021/07/28 Update: Added support for higher-quality LiDAR RGB streaming. Use this Unity demo with Record3D 1.6 or newer.
2021/03/15 Update: For playback of pre-recorded videos, visit the record3d_offline_unity_demo
2020/09/26 Update: Optimized the project for Unity 2019.4.11f1
2020/09/17 Update: Introduced LiDAR support
This demo shows how to use the Record3D app to stream Point Clouds from your iPhone/iPad with the TrueDepth camera into Unity VFX Graph in real-time.
Disclaimer: the demo is in pre-alpha state; in case of wider interest I can continue developing this project into a Unity Plugin.
What will you need?
- An iPhone or iPad with the TrueDepth Camera (FaceID)
- Record3D iOS app (live streaming requires purchase)
- Unity 2019.4.11 or newer
- Make sure you have installed iTunes (on Linux, you will need to install
libusbmuxd-devpackage — should be installed on Ubuntu by default, no action needed)
Before opening the Unity project
This demo depends on the C++ library
record3d_unity_streaming which you will either need to build yourself or download pre-built (available only for Windows and macOS, you will need to build it yourslef if you are running on Linux).
Pre-built library for Windows
dll file is needed:
record3d_unity_streaming.dll. Make sure it is in
PATH — the easiest way is to copy the file into
C:\Windows\System32 and/or into
Pre-built library for macOS
Similar to Windows, only one
dylib file is needed:
librecord3d_unity_streaming.dylib. Just copy it into
/usr/local/lib/. Alternatively, you can
export it into
After opening the Unity project
Open the LiveStreaming scene (
Connect your iPhone or iPad to your computer via USB cable. To make things easier, try having connected only single iOS device (if multiple iOS devices are connected, you can change the index of USB device in the "Device Index" property of the "Record 3D Stream" Script).
Open Record3D, go to the Settings tab and enable "USB Streaming Mode" (purchase required).
Go to the Record tab and press the circular red toggle button (you should see "Started, Waiting for Connection" message on your iPhone/iPad).
Press the "Play" button in Unity and wait.
Live stream should appear — you can now switch into the Scene tab to inspect the live stream.
You can contact me at email@example.com if you have any questions or encounter problems.