Spatial Behavior Engine is an experimental Unity 6.4 project that turns audio-derived state into comfortable, expressive spatial motion.
audio / voice / music state
-> spatial behavior policy
-> trajectory + object parameters
-> Unity / renderer / future platform output
The repository includes a Windows authoring application and an isolated, phone-sized Android shell for SpatialMusic, plus a reusable engine intended to support both SpatialMusic and a future EmbodiedChat application. The shared engine stays independent from app scenes, platform file pickers, and demo presentation code.
- Rule-based spatial behavior for vocals, drums, bass, and other instruments.
- Song-section, melody, rhythm, and beat-aware motion control.
- Synchronized stem playback with per-component trajectories and centers.
- Beginner spatial ideas plus advanced trajectory and speed authoring.
- Custom 3D paths, performance recording, and behavior JSON export.
- Local, queued song import with four-stem separation and analysis.
- English, Chinese, German, and Japanese desktop UI.
- Interactive 3D stage visualization centered on the headphone listener.
Imported audio and generated stems remain on the user's computer or Android device. Both distributions bundle their processing runtime and HTDemucs model, so recipients do not need to install Python, use a separate Windows processor, or download a model after installation.
| Path | Purpose |
|---|---|
Assets/SpatialBehaviorEngine/ |
Reusable, app-independent engine core and Unity runtime adapters. |
Assets/SpatialMusicApp/Runtime/ |
Reusable SpatialMusic playback, authoring, trajectory, recording, and song-catalog features. |
Assets/Apps/SpatialMusicDesktop/ |
Windows desktop app shell, startup scene, runtime import UI, and editor build tooling. |
Assets/Apps/SpatialMusicAndroid/ |
Portrait Android app shell, safe-area UI, generated scene, and isolated Android build tooling. |
Assets/Demos/SpatialMusicDemo/ |
Development demo, scene builders, debug UI, and tests; not production app code. |
tools/spatial_music/ |
Local rule-based song analysis and processor worker. |
tools/windows_distribution/ |
Reproducible Windows installer and portable-package scripts. |
python/ |
Optional lightweight JSON and preference-analysis tools; not the bundled song processor runtime. |
docs/ |
Architecture, portability, validation, runtime, and distribution notes. |
The intended dependency direction is:
SpatialMusic.Desktop
-> SpatialMusic.App
-> SpatialBehaviorEngine.UnityRuntime
-> SpatialBehaviorEngine.Core
Requirements:
- Unity
6000.4.0f1(Unity 6.4). - Windows 10 or 11 for the current desktop shell and packaging workflow.
- Headphones or earphones for evaluating the spatial presentation.
Open the repository root as a Unity project, then open:
Assets/Apps/SpatialMusicDesktop/Scenes/DesktopAuthoringScene.unity
That scene is the only enabled player startup scene. Enter Play mode to use the desktop authoring experience. Do not add imported songs or generated stems to the repository.
Current Windows release: SpatialMusic 0.1.3 (August 6, 2026).
| Platform | Package | Size | SHA-256 |
|---|---|---|---|
| Windows 10/11 x64 | Download SpatialMusic 0.1.3 Store installer | approximately 376 MiB | ae9628d4d77fe671f23de6771670162db7e070c3a3ab20d8643cb39d4b88b823 |
| Android 8.0+ ARM64 | Download SpatialMusic Android 0.1.0 debug APK | approximately 229 MiB | 63d946c610eaf09a19772fd6483d7d4161715346438e3777bd68ed087ea42d52 |
These direct links bypass GitHub's large-file preview and download the packages
from Git LFS. The Windows link now targets the Store-ready, silent installer
SpatialMusicStoreSetup-0.1.3.exe. It is intended for Microsoft Partner Center
submission and can also be downloaded directly; when launched manually it
installs without an interactive setup window. The Android package is a
debug-signed ARM64 build; Android may ask you to allow installation from the
browser or file manager used to open it.
In Partner Center, use the same Store installer URL and set the EXE
Installer parameters field to /quiet; see the
Windows distribution guide for the exact
submission values and local validation command.
The installer contains the Unity player, private Python runtime, PyTorch, Demucs, and the HTDemucs model needed for local stem generation. It installs for the current Windows user and does not require administrator access. The installer is currently unsigned, so Windows SmartScreen may display a warning.
GitHub stores the installer with Git LFS. Use the download link above rather than a GitHub-generated repository source archive, which may contain only the LFS pointer depending on the repository archive settings.
The Android APK also contains the on-device ExecuTorch/HTDemucs processor. Song separation is performed locally, and processed stems are stored in the app's private song library. The first processing run can take substantial time and memory depending on the phone and song duration.
In Unity, run:
Spatial Behavior Engine > Build SpatialMusic for Windows
The player is generated under Builds/Windows/. The build includes the local
processor only when its Python runtime, dependency directory, and HTDemucs
model cache are available as described in the
Windows distribution guide.
After a successful player build, create the self-contained installer and portable ZIP from PowerShell:
powershell.exe -NoProfile -ExecutionPolicy Bypass -File `
.\tools\windows_distribution\Build-SpatialMusicDistribution.ps1Artifacts are written under Builds/Distribution/ and ignored by Git by
default. The versioned installer linked above is intentionally tracked through
Git LFS. The generated installer is currently unsigned; a public release
should code-sign the player and installer before distributing them.
Before opening a pull request:
- Open the project with Unity
6000.4.0f1and confirm there are no compile errors. - Run the relevant EditMode tests.
- Open
DesktopAuthoringSceneand smoke-test song selection, playback, spatial/original comparison, trajectory editing, and saving. - For distribution changes, build the Windows player and follow every check in the Windows distribution guide.
- Confirm the change does not add songs, generated stems, Unity caches, build output, or local processing dependencies to Git.
See the full validation checklist for engine and portability expectations.
- Runtime engine code must remain Unity 6.4 compatible and Android-portable.
- Keep
UnityEditorcode inside editor-only assemblies and folders. - Keep reusable engine code independent from app shells and demo scenes.
- Do not commit copyrighted audio, commercial stems, user recordings, generated song caches, packaged ML runtimes, or model files, except for an explicitly approved versioned distribution artifact tracked through Git LFS.
- Do not add cloud upload, accounts, social publishing, or ML training to the MVP.
- Do not claim Dolby Atmos export or compliance.
Only use audio that you own or are authorized to process. See the data policy for the complete repository rules.
- Architecture
- SpatialMusic runtime and development modules
- Rule-based spatial music player
- Song catalog and analysis
- Android portability
- Windows/Android package migration plan
- Windows distribution
Android remains a portability target for the shared runtime, and this repository now includes an Android application shell. See the Android application guide for its content and build boundaries.