MMUs in Python

Integration of existing synthesis approaches in Python

In order to integrate an existing motion synthesis approach in python to the MOSIM framework, you require an MMU wrapper implementing the MotionModelInterface. In order to send and receive information, the MOSIM datatypes as defined in thrift have to be utilized. To use the MMU, an example behavior must be defined in the target engine. To utilize the retargeting service, you require a retargeting configuration, to initialize the service.

Unlike the CSharp-based MMUs, there are no dlls to integrate. However, we recommend utilizing the MMIPython and MMIStandard pip libraries from the core repository in [Framework/LanguageSupport/python/](https://github.com/Daimler/MOSIM_Core/tree/develop/Framework/LanguageSupport/python). Unlike the csharp- and unity-based MMUs, the adapter will not search in the file system to load mmus. As different MMUs from different providers will have separate python environments, the MMUs should instantiate their own Adapter. Using the PythonAdapter.start_adapter function, allows to easily start an adapter for a number of MMUs.

For distributing MMUs written in python, please do not forget to provide your own Python environment with all required packages.

We provide an example MMU for streaming BVH file content to the MOSIM Framework in the repository under [BasicMMUs/Python-MMUs/ExampleBVH/](https://github.com/Daimler/MOSIM_Core/tree/develop/BasicMMus/Python-MMUs/ExampleBVH)

MMU Functions

The following MMU functions must be implemented. The MMU Interface is defined in the [MMU thrift file](https://github.com/Daimler/MOSIM_Core/blob/master/Framework/LanguageSupport/thrift/mmi/mmu.thrift). # Initialize Parameters: MAvatarDescription avatarDescription; Map<string, string> properties

Returns: MBoolResponse

This function gets called when the target engine is started and initializes a new target avatar. The target avatar is described with the avatarDescription. In this function, the MMU can initialize the avatar, adjust the environment based on the target scene and prepare everything for the MMU execution. If there is an error during initialization, the MMU can return a negative response, which will prevent the co-simulator to load. # AssignInstructions Parameters: MInstruction motionInstruction; MSimulationState simulationState

Returns: MBoolResponse

In this function, an instruction is assigned. The instruction should contain all required information for the MMU to be executed. In case of IDs, please consider using IDs that are referring to Constraints rather than to scene objects. The simulationState describes the current state of the simulation, containing the current avatar pose and its constraints. If the requirements for execution are not met, the MMU can return a negative result.

DoStep

Parameters: double time; MSimulationState simulationState

Returns: MSimulationResult

DoStep is the central method to simulate poses. It is compatible to the Update function of the Unity game engine and performs a single simulation step for the amount of time in seconds as provided. The simulationState contains the current and initial posture, as well as all active constraints. In this method, the MMU is supposed to calculate the next pose after time elapsed seconds to the previous pose. The previous frames merged posture is contained in simulationState.Initial. The resulting posture should be returned with the simulation result. In this result, you can as well return events, depending on the current state of the simulation. The most important constraint is the "end" constraint, denoting the end of simulation for this MMU, e.g. when the goal is reached. For all constraints, please consider [the MMI.thrift file](https://github.com/Daimler/MOSIM_Core/blob/562e43c36da24f6bf4c37a37bd8c4cba7744a43a/Framework/LanguageSupport/thrift/mmi/mmi.thrift#L23).

MSimulationState

The MSimulationState contains two avatar definitions. * Initial describes the initial MAvatarPosture before any MMU was executed. Usually, it is the last frames posture after merging and applying all constraints. * Current describes the current MAvatarPosture with all animations from other MMUs applied, which are running in parallel and are executed prior to the current MMU. If no MMU was executed before, it will be equal to the initial posture. * Constraints describes the list of constraints, which should be enforced currently, this field should be (at least) forwarded to the simulation result. * SceneManipulations contains all scene manipulations, which are requested by MMUs that are executed prior to this MMU for the current frame. This field should be (at least) forwarded to the simulation result. * Events contains a list of events, which are sent by MMUs that are executed prior to this MMU for the current frame. This field should be (at least) forwarded to the simulation result.

MSimulationResult

• Posture contains the avatar posture after applying this MMU in the internal skeleton format.

• Constraints this field contains all constraints combined: constraints from MMUs executed prior to this MMU and constraints enforced by this MMU.

• Events this field contains all events combined: events from MMUs executed prior to this MMU and events evoked by this MMU.

• SceneManipulations this field contains all scene manipulations combined: manipulations requested from MMUs executed prior to this MMU and manipulations requested by this MMU.

• DrawingCalls this field can contain drawing calls from this MMU

Example MMU: BVH Streamer

When developing MMUs in Python, there are a couple of things to consider.

• Interfaces are defined in the main packages (e.g. import MMIPython.PythonAdapter as PythonAdapter)

• Datatypes are defined in a sub-package .ttpyes (e.g. from MMIStandard.core.ttypes import MBoolResponse)

• Some functions require an "MBoolResponse" return. Returning a boolean here is not sufficient.

MotionModelInterface.init

• The init method provides scene and service access, which can be used to access further services (e.g. service_access.GetRetargetingService())

• The init method can be utilized to load data (e.g. load the neural network, the motion database, etc.)

MotionModelInterface.Initialize

• In the initialize, the avatar description of the target avatar is provided

• Initialize your own skeleton and a scaling function here. The retargeting service does not solve scaling issues

MotionModelInterface.AssignInstruction

• In the AssignInstruction method, an instruction object is provided

• Use the instruction to prepare motion synthesis, e.g. setting the current position, planning paths, etc.

MotionModelInterface.DoStep

• This is the core simulation loop

• create new frames and send them to the framework

• At least copy the simulation state to the result:

simres = MSimulationResult()
simres.Posture = simulationState.Current
simres.Events = simulationState.Events
simres.SceneManipulations = simulationState.SceneManipulations
simres.Constraints = simulationState.Constraints