Maya-ACE is a reference implementation designed as a client for the ACE Audio2Face service, which leverages NVIDIA's cutting-edge Digital Human Technology to generate high-quality, audio-driven facial animation. With Maya-ACE, users can effortlessly access and utilize the Audio2Face service through a simple, streamlined interface, or dive into the source code to develop their own custom clients.
This repository includes a Maya plugin, gRPC client libraries, test assets, and a sample scene—everything you need to explore, learn, and innovate with the ACE Audio2Face service. Whether you're looking to integrate this technology into your own projects or simply experiment with its capabilities, Maya-ACE provides a robust foundation.
The project is released under the MIT license, granting you freedom to use and modify the code, with the responsibility to ensure its appropriate use. Please note that NVIDIA assumes no liability for any issues arising from the use of this software.
The ACE Audio2Face service is accessible through NVIDIA NIM , and all the information for the service can be found from the Document Portal.
- What can be done
- Requirements
- Getting Started
- User Interface
- Best Practices
- Troubleshooting Tips
- Build and Test
- Additional Knowledge
Maya-ACE allows users to send audio inputs and receive corresponding facial animations. These animations can be directly connected to a blendshape node, enabling you to animate any character in Maya seamlessly.
Maya-ACE provides all the necessary functionalities through a straightforward UI. It serves as an excellent tool for learning and experiencing the ACE Audio2Face service, helping users gain a deeper understanding of how it works.
For those looking to expand their workflow, users can elevate their projects by transitioning to the Kairos Unreal Engine integration or Tokkio: an interactive avatar virtual customer service assistant product SDK. This allows for the continued use of Audio2Face within other platforms, sharing the same parameters from Maya-ACE for a consistent experience.
Maya-ACE is designed to work seamlessly with standard Maya nodes, including the blendshape node, making it adaptable to drive any character in your scene.
The source code and scripts provided with Maya-ACE can be modified to create a custom pipeline within Maya or to develop a client for other platforms, giving users the flexibility to tailor the tool to their specific needs.
- Autodesk Maya 2023, 2024(recommended), or 2025
- Microsoft Windows 10 64bit or 11
- Internet connection
- API key or a private Audio2Face Service
Other Maya versions and platforms can be supported in the future.
- Set an environment variable
,
NVCF_API_KEY, with a valid API key - Download the
macepackage, unzip and copy contents to a Maya module path
- Download the sample maya scene and sample audio files: English(male) or Chinese(female)
- Launch Maya, load
maya_aceclientplugin, and open the sample scene.
- Adjust Time Slider preferences
: framerate=30, playback speed=30fps, looping=once

- Disable
Cached Playbackon the Time Slider
- Import an audio into the Maya scene and set it to the Time Slider sound

- Adjust Time Slider range to fit to the audio length
- Select
AceAnimationPlayer1node and open Attribute Editor - Click the option menu next to Audiofile attribute, and select the audio imported before.

- Click
Request New Animationbutton. Wait for the Received ### frames message. - Click Maya's play button. Watch how face is moving
- Get an API key and setup Maya-ACE for Maya. Please follow Quickstart 1. and 2.
- Download sample fbx files: Mark, Claire
- Launch maya with a new scene. Load maya plugins;
maya_aceclientandfbxmaya
- Create references of sample fbx files

- Adjust Time Slider preferences
: framerate=30, playback speed=30fps, looping=once

- Disable
Cached Playbackon the Time Slider
- Import a sample audio:
English or
Chinese

- Adjust timeslider range
to fit to the audio

- Select
c_headWatertight_midof Mark, and click menu->ACE->Attach a new Animation Player
- (Optional) Tips to optimize the viewport for faces
- Hide unused groups; such as c_mouth_grp_mid, r_eye_grp_mid, l_eye_grp_mid, root
- Setup a viewport camera with focal length between 75mm and 150mm looking at the face(s)
- Change viewport camera's Near Clip Plane to 10
- Assign standardSurface1 material to face(s)
- (Optional) Tips to optimize the viewport for faces
- Open Attribute Editor, Select
AceAnimationPlayer1node
- Enter
valid
Network Address,Api Key, andFunction IDon the Attribute Editor.
- To change audio, Select an option from the AudioFile, and also update Time Slider to use the same audio.

- Click
Request New Animationbutton, and wait for the Received ### frames message. - Click Maya's play button. Check animation on Mark's face.

- Select both
AceAnimationPlayer1andc_headWatertight_midof Claire in order, and click menu->ACE->Connect an existing Animation Player
- Click Maya's play button again. Check animation on both faces.

Information to connect ACE Audio2Face service, and audio file information to request animation.
- Network Address: A full url with protocol and port number. example: https://grpc.nvcf.nvidia.com:443
- Api Key: A valid api key acquired from the web page
- Function Id: A valid function id that is specific for the service and an AI model.
Find a proper Function ID from the web page
- Examples (as of August 21, 2024)
- Mark model: 945ed566-a023-4677-9a49-61ede107fd5a
- Claire model: 462f7853-60e8-474a-9728-7b598e58472c
- Examples (as of August 21, 2024)
- Audiofile: A path to an audio file to request animation from the service. It can be also selected from imported audios through the drop down option.
Parameters to control generated emotion and preferred(manual) emotion. Audio2Face generate animation with the emotion input which includes generated emotion and preferred emotion. Please watch this video to understand how it works.
- Emotion Strength: the strength of the overall emotion; the total of auto emotion and preferred(manual) emotion.
- emotion = emotion strength * (preferred weight * preferred emotion + (1.0 - preferred weight) * generated emotion)
- Preferred Emotion: Enable/disable and the ratio of the user driven emotion in the overall emotion (1.0 = 100% preferred emotion, 0.0 = 100% generated emotion).
- Auto Emotion: Parameters to control generated emotion.
Parameters to control overal face animation. Please check Audio2Face Microservice documents for the updated information.
Override specific expressions by multiplying the Audio2Face result.
Override specific expressions by adding constant values to the Audio2Face result;
- each output = (raw result * multipler) + offset
Create a new AceAnimationPlayer and connect to the selected blendshape node
Connect the select AceAnimationPlayer to the secondly selected blendshape node
Export a json file with parameters from a AceAnimationPlayer
- DISCLAIMER: This export may not fully reflect recent updates from the server or ACE services. The following limitations may result in potential discrepancies:
- Parameters not set and controlled by the Maya plugin will be exported with default values.
- Default values in the exported configuration files may differ if the A2F server is deployed with custom settings.
- To ensure consistency between the client and server, carefully compare the server's configuration with the exported parameters.
- For the most accurate and up-to-date information, please refer to the official website or the server configuration.
Generally, it requires multiple tries to find an optimal setting. Users are expected to work with an iterative process with using Maya-ACE in their projects. All animations should be downloaded to see the result through any changes; such as paramters. The typical workflow is,
- Adjust Parameters
- Update animation by clicking Request New Animation button
- Play the scene and check the animation with audio
- Repeat from 1 to 3 until satisfied with the result
- Bake animation of the blendshape weights
- Save the scene
Please check ACE Parameter Tuning Guide to understand individual parameters.
Maya provides
Bake Simulation
to freeze and save animation that is downloaded from the service.
- Select blendshape node and select all blendshape weights from the channel box
- Click Bake Simulation through menu; (Animation Layout) menu -> Keys -> Bake Simulation

Maya-ACE assumes to use ARKit compatible blendshape set , but it is still usable with a partial set of blendshapes through this process.
Once AceAnimationPlayer receives animation, it will update output weights with blendshape names that is received from the service,
and then Connect existing Animation Player menu will use the names to find the best possible blendshape targets based on the names.
Rules to match blendshape names
- The blendshape name must match with one of the arkit face location
- Case insensitive
Steps to use the name-based connection
- Finish receiving animation and update
Output Weightsof AceAnimationPlayer node with blendshape names - Select the AceAnimationPlayer node and the blendshape node to connect
- Click
Connect existing Animation Playerfrom the menu -> ACE
These are common errors when requesting new animation through Maya-ACE.
| Error Messages | Possible Reasons (not all) |
|---|---|
| Unauthenticated | wrong api key, using http for an https endpoint |
| no authorization was passed | missing api key |
| SSL_ERROR_SSL | mismatched SSL version |
| Deadline Exceeded | bad network connection, slow machine or service, long audio |
| Cloud credits expired | out of credits |
| DNS resolution failed | network error, wrong address |
| Connection refused | service is unavailable |
To attach AceAnimationPlayer, the selection should be a blendshape node or a mesh node with one blendshape connected.
To connect an existing AceAnimationPlayer to a new blendshape node, the selection should be one AceAnimationPlayer node and a blendshape node or a mesh with a blendshape node connected. There could be errors when,
- Selection is not in the correct order
- Selected multiple AceAnimationPlayer node or multiple blendshape node
- Selected a mesh connected with multiple blendshape nodes
Please make sure the selection is correct for each operation.
Please install the following build tools are accessible in your PATH environment variable.
- MSBuild (Visual Studio 2019+)
- python3
To build Maya plugins, Maya devkit is required and must be located at the correct path.
Download Maya devkit from Autodesk website
and locate the devkitBase directory under deps directory of the local repo.
./deps
└───devkitBase
├───cmake
├───devkit
├───include
├───lib
└───mkspecs
To run maya tests and use plugins, Maya application is required, and MAYA_LOCATION environment variable should be set to the Maya's install directory.
By default, Maya 2024 is expected, and the build script will copy plugin(s) to _build/windows-x86_64/release/plguins/mace/plug-ins/2024.
The script will download required dependencies and place in _build\target-deps
.\fetch_deps.batFor convenience, Premake5 will be downloaded to .\deps\premake. Please use this version for subsequent builds or add it to your PATH variable. If you already have Premake5 installed, it should also be compatible.
For a quick release build with Visual Studio 2022
build.batTo specify visual Studio version other than 2022, please change vs2022 to your installed Visual Studio version
- To run MSBuild.exe, it needs to use Developer Powershell or to set up the system path.
# build maya_acecleint plugin
premake5.exe vs2022 --solution-name=maya-ace --aceclient_log_level=1
MSBuild.exe /NoWarn:MSB8003,MSB8005 .\_compiler\maya-ace.sln /p:Configuration=release /p:Platform=x64 /verbosity:minimal /m /t:mace
# for debug log level
premake5.exe vs2022 --solution-name=maya-ace --aceclient_log_level=3Note: Please build first before running the test. The build process will also pull pip packages needed for unit tests.
# run unit test with mayapy
.\test_maya.bat
# run tests which match the given substring
.\test_maya.bat -k test_plugin$ENV:MAYA_LOCATION="C:\Program Files\Autodesk\Maya2025"
.\test_maya.batSET "MAYA_LOCATION=C:\Program Files\Autodesk\Maya2025"
.\test_maya.bat.\run_maya.bat.\venv\Scripts\activate
.\run_mock_server.batA Static library that handles communication with ACE; sends and receives data.
For a quick release build with Visual Studio 2022
build.batTo specify visual Studio version other than 2022, please change vs2022 to your installed Visual Studio version
- To run MSBuild.exe, it needs to use Developer Powershell or to set up the system path.
# build aceclient test
premake5.exe vs2022 --solution-name=tests-aceclient --aceclient_log_level=1
MSBuild.exe /NoWarn:MSB8003,MSB8005 .\_compiler\tests-aceclient.sln /p:Configuration=release /p:Platform=x64 /verbosity:minimal /m /t:tests-aceclient# run tests for aceclient
.\test_aceclient.batPlease read Generating the ACE gRPC module to know how to update the grpc generated files.
Maya-ACE uses AudioFile to read audio files, and supported formats may vary from Maya's supported audio file formats'.
We recommend using the formats below for the best compatiblity and efficiency.
- file format: wav
- data format: 16bit PCM or 32bit IEEE Float
- samplerate: 16kHz, 32kHz, or 48kHz
- channels: 1 channel (mono)
NVIDIA ACE is a suite of real-time AI solutions for end-to-end development of interactive avatars and digital human applications at-scale.
Additional information about ACE is available from NVIDIA Documentation Hub.
For ACE customers to get support, please contact through NVIDIA Enterprise Support











