Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Animating on MacOS but not on iOS, maybe still "Armature/" related #22

Closed
enzyme69 opened this issue Sep 1, 2018 · 35 comments

Comments

Projects
None yet
4 participants
@enzyme69
Copy link

commented Sep 1, 2018

I thought this one was already solved and I tried deleting all "Armature/" manually using USDA, but somewhat the USDZ is not working. Strange...

Example animation: (made using Blender Grease Pencil, exported as GLTF)
greasepencil_quicky.zip

@enzyme69

This comment has been minimized.

Copy link
Author

commented Sep 1, 2018

screen shot 2018-09-01 at 11 33 21 pm

Aha... I got it, I am an idiot! 👻

@enzyme69 enzyme69 closed this Sep 1, 2018

@wave-electron

This comment has been minimized.

Copy link

commented Sep 1, 2018

@enzyme69

I was having a look at this myself... and while the animation runs in xcode beta 6, it wouldn't start in safari ios12 (beta 12). Did it run for you after making those changes to Armature/?

@enzyme69

This comment has been minimized.

Copy link
Author

commented Sep 3, 2018

@wave-electron Yes it works! Ensure the "first name of the joint or bone" matches those for root. That's another gotcha, but @kcoley your Python GLTF 2 USD converter is so handy!

@wave-electron

This comment has been minimized.

Copy link

commented Sep 5, 2018

@enzyme69 Jimmy I'm not following what you mean by this. I've tried replacing "Armature/" references to say "Armature_Bone/" in the same spots as your screenshot above... but the animation still does not run in IOS12 beta 12. It works however in xcode 10 beta 6. Can you be more explicit with what you mean by.... ensure the first name of the joint or bone matches those for root?

Yes it works! Ensure the "first name of the joint or bone" matches those for root. That's another gotcha, but @kcoley your Python GLTF 2 USD converter is so handy!

@wave-electron

This comment has been minimized.

Copy link

commented Sep 5, 2018

When I make those changes the animation will work in xcode beta6, but it previews in IOS12 Beta 12 but the animation does not run. I've uploaded the files for you to check if we're both on the same page. Note: the usdz may work on MAC, I'm still running on High Sierra OS, so cannot check that.
quicky.zip

@wave-electron

This comment has been minimized.

Copy link

commented Sep 5, 2018

@enzyme69 Jimmy did you get a chance to look at this? I'd love some feedback.

@enzyme69

This comment has been minimized.

Copy link
Author

commented Sep 5, 2018

@wave-electron You cannot have same name bone within the hierarchy. On MacOS Mojave, the USDZ animation works fine, but on iOS, it's more strict.

Let me try to fix yours, I've been using Sublime to quickly replace same name bones ... one sec..

Believe it or not, I have been asked the same questions on this weird hierarchy name issue, no "Armature/" ... :D

screen shot 2018-09-05 at 11 40 11 pm

screen shot 2018-09-05 at 11 39 47 pm

screen shot 2018-09-05 at 11 36 52 pm

screen shot 2018-09-05 at 11 22 50 pm

@wave-electron

This comment has been minimized.

Copy link

commented Sep 5, 2018

ok. Thats what I thought it was... but I've been finding it hard to fix that hierachy from within the usda file. Do you fix that inside Blender? If so could you screen shot how it looks. Thanks

@enzyme69

This comment has been minimized.

Copy link
Author

commented Sep 5, 2018

I am fixing it using Sublime Text.

Basically "Find All" : "Armature/", whilst all highlighted, I deleted it.

screen shot 2018-09-05 at 11 47 40 pm

quicky_fixed.zip

@wave-electron

This comment has been minimized.

Copy link

commented Sep 5, 2018

Can you reload that zip again. I don't think you gave it enough time to upload

@wave-electron

This comment has been minimized.

Copy link

commented Sep 5, 2018

Jimmy got it. It works for me now on ios 12 as well. Appreciate that. I think that will help me fix many other bone animations I've been having problems with. :)

@enzyme69

This comment has been minimized.

Copy link
Author

commented Sep 5, 2018

screen shot 2018-09-05 at 11 52 36 pm

It works, yay!! Great, alright can start making some animation now :D

@wave-electron

This comment has been minimized.

Copy link

commented Sep 7, 2018

@enzyme69 @ox @kcoley Shouldn't this issue still be open?

Unfortunately, haven't had time myself to explore a coding solution, but it really requires one.

The only way I know to currently build a bone animation using gltf2usd tool required me to manually edit the usda file to make the animation run on IOS12 beta 12. Whilst the animation does work in Xcode without the edit.... I would think most folks would ultimately require it just work on IOS as well.

I've also found nearly every bone animation I've tried suffers from this joint name problem, except a couple of models I found in KhronosGroup Samples glTF

@kcoley

This comment has been minimized.

Copy link
Owner

commented Sep 7, 2018

Yeah I am not sure how this was closed (might have been auto-closed by one of the other merged PRs). I'll reopen this.

@kcoley kcoley reopened this Sep 7, 2018

@ox

This comment has been minimized.

Copy link
Contributor

commented Sep 7, 2018

@wave-electron yes this issue should still open. I'm digging through it right now. So far, the only real problem I see in @enzyme69's model is the lack of a root skeletal node (without '/' in it's path) in the "wrong" screenshot. Looking at the difference between the "quicky" models, I noticed that in quickyX, the "Armature_Bone" skeleton has a root joint "Armature_Bone" (not to be confused with the skeleton name), whereas the broken one has an invalid root joint "Armature/Armature_Bone".

I think the solution is to remove the root skeleton name from the hierarchy building code. I made the following changes in _convert_skin_to_usd and it did give a root node, which is a start:

-        skeleton = UsdSkel.Skeleton.Define(self.stage, '{0}/{1}'.format(parent_path, gltf_root_node_name))
+        skeleton_path = '{0}/skel'.format(parent_path)
+        skeleton = UsdSkel.Skeleton.Define(self.stage, skeleton_path)
+        print("skeleton path %s, root %s" % (skeleton_path, gltf_root_node_name))
         skel_binding_api_skel_root.CreateSkeletonRel().AddTarget(skeleton.GetPath())
         if len(bind_matrices) > 0:
             skeleton.CreateBindTransformsAttr().Set(bind_matrices)
 
-        joint_paths = []
+        joint_paths = [Sdf.Path(gltf_root_node_name)]
@wave-electron

This comment has been minimized.

Copy link

commented Sep 8, 2018

@ox I have now successfully changed several doing @enzyme69 suggestion. The animation after the edit in all cases work in IO12. I'll try out your patch above and see what results I get. Cheers.

I created another animation this morning which I posted on my fusionar.app. gallery. Tt could benefit from the Blend shape support. How are you progressing on that front?
bananaman

@ox

This comment has been minimized.

Copy link
Contributor

commented Sep 8, 2018

@wave-electron BlendShape support was merged however I haven’t gotten animations to play. To be honest I haven’t seen any BlendShape USD models out on the Internet. I think that part of the schema is very new so we might be on the frontier on this one. I have successfully generated USD models from gltf models which has an animation channel with “path”: “weights”, but iOS refuses to render it.

What exactly are you doing to get the skinning to work?

@wave-electron

This comment has been minimized.

Copy link

commented Sep 8, 2018

We might be able to request example from Pixar in the usd-interest group. Those guys seem fairly accommodating to requests. In respect to my animations with weights. I’m creating the animation in Cheetah 3D, exporting as a collada, importing into Blender to export as glTF 2. Then running the gltf2usd command line tool. Finally manually editing the file to fix the joint name. Then using usd_converter tool for final conversion. That’s it!

@wave-electron

This comment has been minimized.

Copy link

commented Sep 8, 2018

@ox I tried out your patch above on the bananaMan.usdz above. I had to make the following edits manually to get it to work. If i didn't do this the animation was all distorted and broken!

Are you able to re-jig a patch to replace Armature/Armature_Joint with skel?

The edits in red marker

usdz_wrong

Edited usda - animates in IOS 12 correctly

edited_usda

@enzyme69

This comment has been minimized.

Copy link
Author

commented Sep 8, 2018

Exciting development in regards to blendshapes!!

But you are right, it does not work as USDZ yet, maybe on Apple side. I don't know how Maya/Houdini/Katana handles this USD however, does it simply load and can render for Renderman or something?

Is there example by Apple, a USDZ with blendshapes that work, apart from Memoji?

OUTPUT.zip

facecap_bs_failed

@enzyme69

This comment has been minimized.

Copy link
Author

commented Sep 8, 2018

Much simpler animation to test this "shapekeys" or "blendshapes".
simplevirus.zip

The GLTF with blendshapes seems to work, I tested:
https://gltf-viewer.donmccurdy.com

@wave-electron

This comment has been minimized.

Copy link

commented Sep 8, 2018

@ox @enzyme69 @kcoley From Pixar via usd-interest group 1st August 2018

On Blend Shapes
UsdSkel provides a fully featured encoding for Skeletons, geometry bindings, and Joint Animations for USD (with blend shape support anticipated in the coming year)

From Pixar via usd-interest group 8th Sept 2018

I'm not sure what iOS 12 supports, but I can tell you we haven't implemented Maya blendshape import/export yet. The schema has been written as part of UsdSkel, and we did some verification that the behavior matches Maya's conception of blendshapes, but it's still on our to-do list for get import/export working. Also, UsdImaging doesn't visualize blenshapes yet, but that's true for UsdSkel as well at the moment. We're hoping to work on that soon.

@ox

This comment has been minimized.

Copy link
Contributor

commented Sep 8, 2018

@wave-electron I’m away from my computers this weekend but I’m glad that we are narrowing down the issue. I have so far avoided wading into the skinning and skeleton building code in gltf2usd, as it’s the most intricate, so my patches are still just exploratory. Would it be possible for you to share the bananaman gltf model so I can build a fix against it?

It seems that first it’s important that the number of elements in all timesamples for each skeleton animation is the same as the number of tokens in that animstion’s joints array. Then that there is a root node and that it is the skeleton node? That last part doesn’t quite make sense since the UsdSkel docs have skeleton animations where the joints array doesn’t make any mention of the skeleton as the root node. Maybe extra intermediate joints are being computed and added? I think @kcoley is way more familiar with the Skelton code at this point.

@wave-electron

This comment has been minimized.

Copy link

commented Sep 8, 2018

When you do have chance later... take a look at this example by Apple on Skel animation with weights.

soccerBall.usda.txt

@ox

This comment has been minimized.

Copy link
Contributor

commented Sep 8, 2018

@wave-electron thanks for that model. I came across it earlier but I didn’t know USD then. Kind of bummed that blendshapes are in limbo. I was hoping to use it in place of collada files to play with face meshes, but I guess we have to wait for iOS 12 Gold?

@wave-electron

This comment has been minimized.

Copy link

commented Sep 8, 2018

Yeah its a pity. Anyway here's that BananaMan - with all the files you need to explore.
BananaMan.zip

@kcoley

This comment has been minimized.

Copy link
Owner

commented Sep 8, 2018

@ox

It seems that first it’s important that the number of elements in all timesamples for each skeleton animation is the same as the number of tokens in that animstion’s joints array.

I made some progress on fixing this in the refactorskeleton branch. I added logic to interpolate animation data based on a given time sample, which should fill in the missing frames. I am getting way smoother results, though some models are stuttery. I will double check this logic to make sure I did not miss something.

Then that there is a root node and that it is the skeleton node?

The skeleton root seems to be a parent to the joints in the joints array. This may be counted as the "true" root node of the joints, since it is possible to have multiple joints defined at the root level without a top level parent.

I

@ox

This comment has been minimized.

Copy link
Contributor

commented Sep 8, 2018

@kcoley I interpreted the presence of the SkelRoot to just mean that there is a skeleton somewhere further down this stack. I’m still confused about why the skeleton name needs to be specified as the root joint even though the joints array is just to establish hierarchy using arbitrary but consistent naming. I really think it’s a mismatch of array lengths between the joints tokens and the timesample arrays. That should fix bone animation, but now I’m not so sure.

@kcoley

This comment has been minimized.

Copy link
Owner

commented Sep 8, 2018

@ox ah, I learned at SIGGRAPH that this is a bug in the iOS implementation of USD. I believe the convention is based on Maya's USD exporter, which names the skeleton root the same name as the root joint. Renaming the joints is a workaround to get it to animate on iOS until the implementation is updated.

@kcoley

This comment has been minimized.

Copy link
Owner

commented Sep 8, 2018

@ox

I really think it’s a mismatch of array lengths between the joints tokens and the timesample arrays. That should fix bone animation, but now I’m not so sure

Yes, this is correct. With the skeleton refactoring branch, the array lengths are now the same for all the joint tokens since the animation samples are now being interpolated based on sampling with the fps parameter. This seems to be working with the models from Sketchfab, though I am chasing down an animation hierarchy issue, which causes some parts of an animated model to be detached and floating in the wrong location. I plan to make a PR this weekend with the update so others can try it out,

@ox

This comment has been minimized.

Copy link
Contributor

commented Sep 8, 2018

@kcoley awesome, can’t wait to see it. Thank you for putting in the time to refactor and clean!

@kcoley

This comment has been minimized.

Copy link
Owner

commented Sep 10, 2018

@ox no problem! I spent a good chunk of time working on this over the weekend, though I will probably need one more day or two to finish integrating all my changes. It's definitely a big refactor so it's probably safer to test out the PR once I create it to make sure everything is ironed out.

@kcoley

This comment has been minimized.

Copy link
Owner

commented Sep 26, 2018

I made a recent update on iOS to the next Beta (12.1) and it seems to resolve the root name issue. Is anyone else able to confirm?

@wave-electron

This comment has been minimized.

Copy link

commented Oct 8, 2018

@kcoley Yes, that seems to be the case.

@kcoley

This comment has been minimized.

Copy link
Owner

commented Nov 4, 2018

Just merged a fix for this (#94)

@kcoley kcoley closed this Nov 4, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.