New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Mocap leg fixes #9074
Merged
Merged
Mocap leg fixes #9074
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Improve hand tracking through screen landmarks
Belated approval |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
🤖 Generated by Copilot at aeccf4c
The pull request implements various improvements and refactors for the avatar, mocap, and XR features of the engine. It enhances the accuracy, performance, and flexibility of the avatar animation, footstep, and IK systems, and supports VRM models and lower body tracking for mocap. It also simplifies and organizes the code by removing unused or unnecessary imports, variables, and actions, and by using hookstate, action queues, state mutations, and entity UUIDs. It also fixes some errors and bugs related to the XR session and the avatar thumbnail.
References
closes #insert number here
Explanation
🤖 Generated by Copilot at aeccf4c
AvatarNetworkAction.setAvatarXrTracking
action to synchronize the XR tracking state of an avatar entity with the network (link, link, link, link, link)setIkFootTarget
function to make the footstep transition more precise and avoid jittering (link)AvatarIKTargetSystem
to thespawnLocalAvatarInWorld
andcreateIkTargetsForLocalAvatar
functions inreceiveJoinWorld.ts
to avoid unnecessary loops and queries and to centralize the logic for spawning the local avatar and its IK targets (link, link, link)ikDataByName
object and added checks for the blend weight value before calling the IK solver in theAvatarAnimationSystem
(link, link)execute
function of theAvatarAnimationSystem
to theif
block that checks for themotionCaptureRigComponent
or theXRRigComponent
to avoid unnecessary calculations for non-XR or non-mocap avatars (link)newScreenlandmarks
instead of thelandmarks
for the wrist, pinky, and index positions in thesolveMotionCapturePose
function (link)solveMotionCapturePose
function to the beginning, before the check for thenewScreenlandmarks
, to ensure that the head IK is always solved regardless of the hand landmarks availability (link)solvingLowerBody
flag and thehipRotation
from theMotionCaptureRigComponent
instead of theMotionCaptureState
and added a fallback shoulder quaternion when the left and right hips are not visible in thesolveSpine
function (link, link)onInit
,hipRotation
, andsolvingLowerBody
properties to theMotionCaptureRigComponent
definition and schema and set thesolvingLowerBody
flag to1
in theuseEffect
hook in the component (link, link, link)XRRigComponent
to indicate that an entity has an XR rig and enable the IK calculations for the XR avatars in theAvatarAnimationSystem
(link)$cache
property of theAvatarNetworkAction.setAvatarAnimation
action fromremovePrevious: true
toremovePrevious: false
to allow multiple animation actions to be applied to the same avatar entity without removing the previous ones (link)MotionCaptureRigComponent
and commented it out with atodo
comment to indicate that it causes issues with the IK solves (link)solveMotionCapturePose
function and theAvatarAnimationSystem
and commented it out with atodo
comment to indicate that there are issues with the foot quaternions that need to be fixed (link, link)XRAction.sessionChanged
action when the XR session ends fromXRSessionFunctions.ts
because it was not needed and caused errors when the session was already ended (link)getState(EngineState).deltaSeconds
with a local variabledelta
in thesetIkFootTarget
function to improve performance (link)orthogonalVector
andshoulderPositionAlongPlane
in thesolveMotionCapturePose
andsolveSpine
functions with atodo
comment to indicate that they are not used (link, link)avatarDetails.thumbnailResource.url
to use optional chaining inHeader/index.tsx
to avoid errors when theavatarDetails
or thethumbnailResource
are undefined or null (link)🤖 Generated by Copilot at aeccf4c
QA Steps
List any additional steps required to QA the changes of this PR, as well as any supplemental images or videos.
Checklist