Skip to content

3.41.1

Choose a tag to compare

View all tags
@fernandotonon fernandotonon released this 21 Sep 21:25
3.41.1
ea8ee48

Audio2Face lipsync over MCP

generate_lipsync completes the GUI + CLI + MCP parity for audio-driven lipsync (#1019). An agent can now drive it directly:

{"audio_path": "take.wav", "clip": "Speech", "emotion": {"joy": 0.6}}

Optional mesh (defaults to the selection), fps, and output_path. Advertised only on an ONNX build — the handler can't work without it, so promising the capability otherwise would be a guaranteed failing call.

One core, three surfaces

Adding a third surface would have meant a third copy of the solve-to-keyframe logic, which encodes two rules that are silently wrong if reimplemented. Both are now in one place (LipsyncApply), shared by the CLI, the Inspector and MCP:

  • A keyframe carries every matched pose, not just the ones that changed. ARKit targets on a submesh share one VAT_POSE track, and Ogre interpolates a pose missing from the next keyframe toward zero — so per-channel keying made steady channels dip to 0 and back.
  • An existing clip of the same name is replaced, not merged into. Track reuse only overwrites coincident times, and clip length only grows, so re-running left stale keys behind.

MCP and the CLI now produce identical keyframe counts on identical input, which is the check that the sharing is real.

Multi-submesh fix (affects face capture too)

A morph target on a multi-submesh character exists once per submesh — a face split into skin / teeth / eyelashes has three poses called jawOpen, each on its own track. writeWeightKeyOn resolved only the first, so the jaw opened while the teeth stayed frozen.

Fixed at the root, so MocapRecorder (video face capture) gets the fix too — it called the same function and had the identical latent bug. It survived this long because a single-submesh test mesh cannot exhibit it; the regression test now builds a deliberately two-submesh mesh.

Contextual feedback prompt

A one-time user who got what they needed and one who got stuck look identical in retention data, and call for opposite responses. The new prompt separates them by offering two affirmative answers — Got what I needed / Something didn't work / Not now — at the moments that actually distinguish the two: first successful export, an import/export failure, or a long session that never exported.

A successful first import deliberately does not prompt: it is the start of the workflow, before the user has had the chance to succeed or fail.

Full detail in #1068 and #1069.

Assets 15
Loading