Avatar
The focus of the MMI framework is the generation of motions and postures; therewith a format to exchange posture information is essential for the framework. We propose a novel format specifically addressing the requirements of the MMI framework. The bone hierarchy, joint placement and zero-posture of the intermediate representation is pre-defined and constant. However, the dimensions and body parameters of the target avatar can change between different simulations. The definition of the intermediate hierarchy can be found below. The so-called MAvatarDescription struct contains this static information, which have to be exchanged during simulation initialization.
| MAvatarDescription |
|---|
| +AvatarID: string +ZeroPosture: MAvatarPosture +Properties: map<string,string> |
Detailed description of available parameters:
| Parameter Name | Required | Description |
|---|---|---|
| AvatarID | x | The unique id of the corresponding avatar. |
| ZeroPosture | x | The zero posture describes the specific hierarchy of the target avatar. The list consists of multiple Joints (MJoint) which are described below. |
| Properties | Additional semantic information which can be set (e.g. weight of body parts). |
To describe a full posture the so-called MAvatarPosture class can be utilized. The struct contains a unique id, the hierarchy, rotations and positions of all joints. The format can be used for transferring a full posture. Moreover, it is planned to provide several utilizations and helper functions for the MAvatarPosture class. For instance, the global positions and rotations can be easily determined based on the class. Moreover, coordinate system specific conversions.
| MAvatarPosture |
|---|
| +AvatarID: string +Joints: list |
Detailed description of available parameters:
| Parameter Name | Required | Description |
|---|---|---|
| AvatarID | x | The unique ID of the avatar. |
| Joints | x | The joint hierarchy describing the posture. |
The MAvatarPosture describes the specific hierarchy of the target avatar. The list consists of multiple Joints (MJoint) which are described below.
| MJoint |
|---|
| +ID: string +Type: MJointType +Position: MVector3 +Rotation: MQuaternion +Channels: list +Parent: string |
Detailed description of available parameters:
| Parameter Name | Required | Description |
|---|---|---|
| ID | x | Unique id for the joint. |
| Type | ||
| Position | The position data represented in Cartesian coordinates (analogously to bvh offset). | |
| Rotation | x | The data for rotation represented as quaternion. |
| Channels | x | Channel information about the representation of the position/rotation (similar to bvh) which is necessary to interpret the transmitted posture values for each frame. |
| Parent | The id of the parent (if available). |
To represent the information of the transferred rotation/position data and its order, the so-called MChannel enum is utilized.
| MChannel (enum) |
|---|
| Xoffset Yoffset Zoffset Xrotation Yrotation Zrotation Wrotation |
Detailed description of available parameters:
| Parameter Name | Description |
|---|---|
| Xoffset | See bvh |
| Yoffset | See bvh |
| Zoffset | See bvh |
| Xrotation | See bvh |
| Yrotation | See bvh |
| Zrotation | See bvh |
| Wrotation | See bvh |
We assume a left-handed coordinate system with +y as the global up axis and +z as the global forward axis. The joint location is defined in the parent joints coordinate system. The bone orientation is explicitly defined by the offset rotation of the joint. We assume the direction of the bone to be aligned with y-axis of the local coordinate system (see figure 49). Using the channels, rotations around these local axes can be unlocked. Hence, any arbitrary number of channels is valid. In case of the knee-joint, for example, the Channels would be set to [Xrotation, Zrotation], where Xrotation defines the rotation of the knee itself and Zrotation the twist of the calf bone. The global transform matrix Mjoint of a single joint can be computed with
With Mparent being the parents global transform matrix, Ojoint the offset and ROjoint the rotational offset of the joint. MAjoint then describes the animation transform matrix built by subsequently applying the animation rotations and animation offsets in order of the defined channels.
The information above is only exchanged during simulation initiation. During the simulation itself, MMUs, Co-Simulator and Target Engine can reduce the information flow to only exchange joint rotation information using MAvatarPostureValues. MMUs can further reduce the network traffic by announcing to only transfer a limited amount of data. A grasping MMU, for example, might only want to share rotation information regarding the hands, but not the rest of the body. The joint hierarchy and channel information define the semantics of PostureData. In the appendix (6.2), an example listing is available, which defines the full body hierarchy as well as the joint hierarchy inside the individual hands. They were separated for visual purposes in this document. As a zero posture, a T-posture with parallel legs and feet flat on the ground is expected. The specific joint offsets are depending on the proportions and size of the target avatar.
The MAvatarPostureValues are a compact representation of a posture given the assumption that the hierarchy is already known. It is noteworthy that the MAvatarPostureValues class should be used as often as possible (and preferred over MAvatarPosture) in order to reduce traffic and latency during the communication.
| MAvatarPostureValues |
|---|
| +AvatarID: string +PostureData: list +PartialJointList: list |
Detailed description of available parameters:
| Parameter Name | Required | Description |
|---|---|---|
| AvatarID | x | Unique ID that references the avatar. |
| PostureData | x | A list representation of the transmitted posture data. The corresponding hierarchy is described as MAvatarPosture (MAvatarDescription) with the identical AvatarID. |
| PartialJointList | X |
The intermediate skeleton as well as skeleton helper functions are managed by the intermediate skeleton service (MSkeletonAccess).
| <<interface>> MSkeletonAccess |
|---|
| +InitializeAnthropometry(description: MAvatarDescription) +GetAvatarDescription(avatarID: string): MAvatarDescription +SetAnimatedJoints(avatarID: string, joints: list) +SetChannelData(values: MAvatarPostureValues) +GetCurrentGlobalPosture(avatarID: string): MAvatarPosture |
| +GetCurrentLocalPosture(avatarID: string): MAvatarPosture +GetCurrentPostureValues(avatarID: string): MAvatarPostureValues |
| +GetCurrentPostureValuesPartial(avatarID: string, joints: list): MAvatarPostureValues +GetCurrentJointPositions(avatarID: string): list +GetRootPosition(avatarID: string): MVector3 +GetRootRotation(avatarID: string): MQuaternion +GetGlobalJointPosition(avatarID: string, joint: MJointType): MVector3 +GetGlobalJointRotation(avatarID: string, joint: MJointType): MQuaternion +GetLocalJointPosition(avatarID: string, joint: MJointType): MVector3 +GetLocalJointRotation(avatarID: string, joint: MJointType): MQuaternion +SetRootPosition(avatarID: string, position: MVector3) +SetRootRotation(avatarID: string, rotation: MQuaternion) +SetGlobalJointPosition(avatarID: string, joint: MJointType position: MVector3) +SetGlobalJointRotation(avatarID: string, joint: MJointType rotation: MQuaternion) +SetLocalJointPosition(avatarID: string, joint: MJointType position: MVector3) +SetLocalJointRotation(avatarID: string, joint: MJointType rotation: MQuaternion) +RecomputeCurrentPostureValues(avatarID: string): MAvatarPostureValues |
Detailed description of available parameters:
| Function/ Parameter Name | Description |
|---|---|
| InitializeAnthropometry | Initializes the internal representation based on the given MAvatarDescription. |
| GetAvatarDescription | Returns the avatar description (if available). |
| SetAnimatedJoints | Sets the animated joints. |
| SetChannelData | Sets the current posture (channel data) based on the given MAvatarPostureValues. |
| GetCurrentGlobalPosture | |
| GetCurrentLocalPosture | |
| GetCurrentPostureValues | |
| GetCurrentPostureValuesPartial | |
| GetCurrentJointPositions | |
| GetRootPosition | |
| GetRootRotation | |
| GetGlobalJointPosition | |
| GetGlobalJointRotation | |
| GetLocalJointPosition | |
| GetLocalJointRotation | |
| SetRootPosition | |
| SetRootRotation | |
| SetGlobalJointPosition | |
| SetGlobalJointRotation | |
| SetLocalJointPosition | |
| SetLocalJointRotation | |
| RecomputeCurrentPostureValues |
To simplify utilization an enum for describing the available joint types (MJointType) is available. The enum comprises in total 61 different bone types.
| MJointType (enum) |
|---|
| Undefined, |
| LeftBallTip, |
| LeftBall, |
| LeftAnkle, |
| LeftKnee, |
| LeftHip, |
| RightBallTip, |
| RightBall, |
| RightAnkle, |
| RightKnee, |
| RightHip, |
| PelvisCentre, |
| S1L5Joint, |
| T12L1Joint, |
| T1T2Joint, |
| C4C5Joint, |
| HeadJoint, |
| HeadTip, |
| MidEye, |
| LeftShoulder, |
| LeftElbow, |
| LeftWrist, |
| RightShoulder, |
| RightElbow, |
| RightWrist, |
| LeftThumbMid, |
| LeftThumbMeta, |
| LeftThumbCarpal, |
| LeftThumbTip, |
| LeftIndexMeta, |
| LeftIndexProximal, |
| LeftIndexDistal, |
| LeftIndexTip, |
| LeftMiddleMeta, |
| LeftMiddleProximal, |
| LeftMiddleDistal, |
| LeftMiddleTip, |
| LeftRingMeta, |
| LeftRingProximal, |
| LeftRingDistal, |
| LeftRingTip, |
| LeftLittleMeta, |
| LeftLittleProximal, |
| LeftLittleDistal, |
| LeftLittleTip, |
| RightThumbMid, |
| RightThumbMeta, |
| RightThumbCarpal, |
| RightThumbTip, |
| RightIndexMeta, |
| RightIndexProximal, |
| RightIndexDistal, |
| RightIndexTip, |
| RightMiddleMeta, |
| RightMiddleProximal, |
| RightMiddleDistal, |
| RightMiddleTip, |
| RightRingMeta, |
| RightRingProximal, |
| RightRingDistal, |
| RightRingTip, |
| RightLittleMeta, |
| RightLittleProximal, |
| RightLittleDistal, |
| RightLittleTip, |
| Root |