Maverick XR Agent for Unity 1.1.0

Version 1.1.0 adds support for Maverick 1.1, the model update that learned to take corrections, follow what people look and point at, and tell who is speaking.

What is new

• Undo. A twelfth tool, undo, reverses the agent's last one to three commands exactly: objects, rooms, states, positions and what the player holds. It refuses, and changes nothing, when something else changed an object in the meantime, when a command failed or cannot be reversed, or when there is nothing to undo.
• Attention and people. The scene can carry what each person is looking at or pointing at, who is speaking and whose hands the agent acts with. SceneAgentGaze can report gaze and pointing for several people, and Command and Answer take the speaker.
• Short memory. The model sees its recent commands and any open question, and tool results now say what changed.
• Changes made outside the agent. SceneWorld.ApplyExternalChange and the syncFromTransforms option tell the model and undo about objects that a person, physics or your own code moved.
• Cancel. SceneAgentSession.Cancel() stops a running command cleanly.

All of this is opt-in. Set SceneAgent.schemaVersion to V1 to use it with Maverick 1.1. The default, V0, behaves exactly as in 1.0, and both Maverick 1.0 and Maverick 1.1 work with it.

Also fixed since 1.0

The output check now rejects missing required arguments, arguments of the wrong type and numbers that are not finite, and calls inside the model's reasoning never run. A command sent while another one is running is no longer started, and a cancelled run no longer keeps acting on the scene. The full list is in the changelog.

Demo

A demo of Maverick 1.1 (1 min 44 s) is attached to this release as maverick-1-1-demo.mp4. It follows a technician and a remote expert in a three-room house, and includes a correction, a fetch from another room, a two-step plan and an undo that is refused. It was recorded live on a laptop with a 4 GB GPU; the script was rehearsed and the expert's voice is text-to-speech.

Install

In Unity, choose Window > Package Manager > + > Add package from git URL and enter:

https://github.com/mcbu-xrlab/Maverick-Unity.git#v1.1.0

The model guide lists the Maverick 1.1 and 1.0 files with their checksums. The schema 1.1 guide explains the new format and how to migrate.

Worth knowing

• A remote person's request can move objects or press buttons in the on-site scene. The model does not check permissions, so enforce them in your application.
• Maverick 1.1 still tends to act instead of asking in situations it has not seen. Its model card lists this and the other limits.
• Tested on Windows with Unity 2022.3. Linux, macOS and standalone headsets have not been validated.

Maverick XR Agent for Unity 1.0.0

Maverick XR Agent for Unity 1.0.0

Maverick brings local natural-language scene control to Unity. Give it an English command such as “put the red mug on the counter” or “find the broom,” and it selects tools and objects from the scene description supplied by your application. When several objects fit, it can ask the user to choose.

This is the first GitHub distribution of the Unity package previously published on Hugging Face. It includes the runtime and editor source, an expanded installation guide, integration examples, model download instructions, and documentation of the published evaluation and current limitations.

View the 30-second demo (MP4, 7 MB).

Install

In Unity, open Window > Package Manager > + > Add package from git URL:

https://github.com/ErenAta16/Maverick-Unity.git#v1.0.0

Choose Tools > Scene Agent > Open Model Folder, then add a compatible Windows llama.cpp runtime and the GGUF model to Assets/StreamingAssets/SceneAgent/.

Download the Q4_K_M model · Installation guide · Model checksum

The model is approximately 2.50 GB and remains hosted on Hugging Face. The source archives for this release contain the Unity package and documentation; they do not include model weights or inference binaries.

Included in this release

• Local llama.cpp server startup and shutdown, with a GPU-first launch path and CPU fallback.
• Eleven tools for object interaction, room changes, search, and clarification.
• Output checking and scene-level action constraints.
• A command session interface for typed text or speech-recognition transcripts.
• Mouse or camera-center gaze targeting and a desktop chat interface.
• Editor tools for setup, object registration, and scene validation.
• A command script runner that records tool results, scene signatures, and timings.

The documented configuration targets Unity 2022.3 or later on Windows x64. The published measurements used an RTX 3050 Ti laptop GPU under Windows 11, with Unity rendering at 30 FPS. Standalone XR headsets, Linux, and macOS have not been validated.

Voice recognition, XR interface design, player rig movement, and scene-specific animations are supplied by your application. See the integration guide and known issues.

The runtime code matches the previously published Hugging Face package at revision 1bb8c83224fd470b1a25fac1ee74ce83ac489c44. The GitHub release adds documentation and repository metadata; it does not introduce new model weights or claim a new benchmark run.

The Unity package is available under the MIT License. The model is distributed separately under Apache-2.0.

Developed by Eren Ata at the Extended Reality Laboratory (XRLab), Manisa Celal Bayar University.