AR Foundation Replay

🚧 Work in progress 🏗️
PoC of Recording AR Foundation session into a single mp4 with a binary timeline track / Replaying it in Editor on AR Foundation without any extra settings.

2023-10-10-walking-geospatial.mp4

↓ Check out the complete project utilizing ARFoundationReplay.

editor-simulation.mp4

Tested platform

• Only iOS ARKit + macOS Editor for now. Android ARCore is planned.
• Tested with the latest Unity2022 LTS.
• Supported only URP. Built-in is not supported for now.

Implemented Subsystems

• XRSessionSubsystem
• XRCameraSubsystem
• XRPointCloudSubsystem
• XRPlaneSubsystem
• XRAnchorSubsystem
• XRRaycastSubsystem
• XRHumanBodySubsystem
• XREnvironmentProbeSubsystem
• XRInputSubsystem
• XRImageTrackingSubsystem
• XRObjectTrackingSubsystem
• XRFaceSubsystem
• XROcclusionSubsystem
• XRParticipantSubsystem
• XRMeshSubsystem
• ARCore Extensions
  • Earth Manager
  • Cloud Anchors
  • Streetscape Geometry

How to use

• Install MemoryPack from NuGet using NuGetForUnity

  • openupm add com.github-glitchenzo.nugetforunity
  • Open Window from NuGet -> Manage NuGet Packages, Search "MemoryPack" and Press Install.

• Add the following line to Packages/manifest.json to install AR Foundation Replay package via UPM:

  "dependencies": {
      "com.github.asus4.arfoundationreplay": "https://github.com/asus4/ARFoundationReplay.git?path=Packages/com.github.asus4.arfoundationreplay#v0.2.2",
      ... other dependencies
  }

• Simulation on the Unity Editor is supported only on iOS. Open Build Settings and switch the platform to iOS.

• Activate AR Foundation Replay for Unity Editor:

  1. Open Project Settings/XR Plug-in Management.
  2. Select the PC tab in the XR Plug-in Management.
  3. Activate AR Foundation Replay

• Put the ARRecordButton prefab into the Scene which is located at Packages/com.github.asus4.arfoundationreplay/Prefabs/ARRecordButton.prefab.

• Record AR on the device.

• You can change the replay file from Project Settings/XR Plug-in Management/AR Foundation Replay.

• Play the AR in the Editor.

ARCore Extensions Support

It supports replaying ARCore Geospatial. To test it, please use the forked version of the arcore-unity-extensions and refer to the ARFoundationReplayGeospatial example project.

Dependencies

• MemoryPack - MIT License
• XR SDK - Unity Companion License
• keijiro/Bibcam - Unlicense
• keijiro/Avfi - Unlicense: Metadata recording system is modified from Avfi.