Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
Untangling the ThreeJS animation structure #6881
Been looking at modernizing/formalizing the ThreeJS animation framework to be more similar to Unity/Unreal Engine 4. There is a lot of great code in ThreeJS but there is a bit of missing top level organization of animation structures, rather there is a lot of duplication.
Here are the classes I've been studying today:
The AnimationHandler/Animation pair:
The following is not compatible with the Animation/AnimationHandler framework:
The following combine scheduling of animations with meshes/mesh hierarchies:
I'm trying to think of a way of reducing all of these classes into a clearer structure.
I think a structure similar to this might be the way forward:
I would get rid of AnimationHandler, Animation (replaced by Clip/Track), KeyframeAnimation (replaced by Track), MorphAnimation (replaced by Track), BlendCharacter (replaced by Mixer/Clip/Track), MorphAnimMesh (replaced by Mixer/Clip/Track), MorphBlendMesh (replaced by Animator/Mixer/Clip/Track), USCCharacter, MD2Character, MD2CharacterComplex (replaced by Animator/Mixer/Clip/Track), or at least deprecate them in favor for the Track/Clip/Action/Mixer/Animator design, and this design could drive any objects in ThreeJS, including Materials, Lights and Cameras, if you so desired.
Basically this is a more flexible and actually simpler design.
The above is very similar to Unity/Unreal Engine.
I just wanted to suggest we consider to import or maybe natively adopt the http://alembic.io/ format (way of thinking) at this time.
It seems FBX is the best but not officially documented. Here I found something possibly useful...
@MasterJames, I think I'll stay as much as possible with the existing JSONLoader format just because there is a lot of content and a lot of existing converters. I want to change the classes that load it though once it is in ThreeJS and how one uses the animation content.
The other format that may be of interest to load into the same structures would be @fabrobinet's glTF's format.
Both the JSONLoader and glTF are designed for the web, so I'd favor them.
FBX is sort of problematic as it is a proprietary format that changes a couple times a year and it is very hard to support -- most 3D content creation tools do not support it properly. But this design should be generally compatible with the internal organization of the FBX file format. :)
But no matter what format you choose to use as the data format, it doesn't really change the main design of the run-time structures.
Yes, please. It's improvable at places - but it's actually capable of development. There's no such freedom with any other format.
glTF is an interesting specification with good ideas and a 200% complete data model. Something to learn from and hopefully stop near the hundred. What is a super flexible format without tools that can handle its flexibility? Other than for glTF we have workable content pipelines and complete and efficient loaders for three.json today.
Great first pass at laying down the broad structure. Can you share more about the Track class and what other types of Tracks you expect to be used? For example, ConstantTrack to me is the same as a KeyframeTrack with one key at 0. If there aren't many other Tracks that you have planned, then I would propose simply having Track being synonymous with KeyframeTrack (i.e. merging the concept).
So I have it working with multiple different types of clips.
The latest is here:
It can drive morphTarget animations, visibility, material properties (colors, etc.), position, rotation, scale, and it can drive subcomponents, like position[x], rather than the whole position value. Just have to get it driving bones and the core of the system is complete.
Examples of how to create animations (morph, color, visible, position, rotation, scale) are here:
I think this can replace a lot of stuff in the Character animations, but also a lot of the Collada and @fabrobinet's glTF loader -- right now those loaders are sort of very special case in terms of how they handle loading and playing back animations.
@getzelone I've added a lot of this already.
You can set the weight of all Actions and it will blend between them properly and between the original value on the objects.
I've also started to convert the standard ThreeJS animation examples to use the new system. Here is a skeleton keyframe animation (240 tracks) and a morph target animation (4 morph targets) running together through the mixer at 60fps:
Nice work! I have been thinking about an animation redesign myself for some time, so here are some random related thoughts. I might have mentioned some of this already in another issue, so apologies if I'm repeating myself.
If you define a track as a pure function that maps time values to weights/positions/rotations/scales, you can support non-uniform keyframe tracks (array of time/value pairs), uniform keyframe tracks (array of values), and parametric curves.
Note: Evaluating a non-uniform keyframe track at a given time involves finding the two neighboring keyframes. As a full search might be too slow, such tracks may want to cache the last found position. This does not work if you want to play back the same track on multiple scene objects - if you want to render an army of marching characters, you want to store the marching animation only once.
Note: Looks like your
Note: I have no experience with this whatsoever, but wouldn't it pay off to uniformely resample animations into what I have called uniform keyframe tracks above? You'd have a guaranteed O(1) access and no headaches with finding the right keyframes. When storing sparse keyframes, you have to know the interpolation method used (or you get large deviations), so in practice sparse keyframes are only useful for animations authored with the same interpolation method as the three.js runtime animation system uses.
Do actions need to know about their weight? Shouldn't this be controlled by the mixer?
Mixer / Animator
As far as I understand it, the Unity 5.0 Mecanim system has three levels of animation mixing:
Animation blend trees
Animation blend trees are very flexible. In additional to blending between similar animations (walk/run), blend tree nodes can be used to transition between two unrelated animations. If the transition is short, if often looks ok. Unreal engine for example has a "blend by int" node where you specify an array of animations and an index of the desired animation, and the node will smoothly blend between the current and the desired animation.
When designing a blend tree, I would consider separating the following concepts:
I have implemented my own proof of concept for animation blend trees in this project. The code completely replaces the three.js animation code in a very hacky way, but is capable of applying a blend tree to a character with 50 bones at several hundred fps. It is a very early prototype, but feel free to use any ideas if you like. The blending code is based on the primitives sampleAnimation (sample an animation clip at a given time, store as a "pose"), blendPose (linear blend between "poses"), and exportPose (flatten a "pose" into a bone matrix texture). In this design, blend tree nodes would ask their children to compute their poses and then call
@crobi thanks for the input!
Regarding your ideas for tracks, I've implemented most of what you have suggested. Although I am using an array/object representation for the track data. I think that someone can create a BufferKeyframeTrack to optimize that part fairly easily - and it should be a drop in replacement that will still be compatible with the rest of the mixer.
Regarding the mixer/animator. I've only created the first layer, but I believe you need to start somewhere. I also think that the next layers are needed, but hopefully can be done as further improvements to this system.
Although the clips are sparse with regards to bones. Thus you can specify which bones you want to drive using a clip, and it doesn't have to be all of them. Thus there is sort of layer support. We can add further features there without too much difficulty I think.
The blend trees are very interesting-- I didn't know about them before. I think that one could create a blend tree using the Clip/Action and putting it into a tree form. Not too much work I think, but I would prefer to do that as a separate PR rather than further expanding this PR -- it is already huge.
Many game engines and animation middlewares (including Unity and Unreal Engine) use blend trees. Here's some links:
Even though many engines use blend trees, finding good articles about them is not easy. I guess that's because everyone uses a slightly different design. I've heard of projects using many small blend trees and state transitions between them (see below), and of other projects using a single huge blend tree that includes all the different states and transitions.
The animation state machine of Unity is an advanced system that I've seen less often than the blend trees. Consider the following example of a super mario game:
The reverse transition is even more obvious - you cannot start blending into the running animation until the character has landed (the jumping animation has finished). You can see this in action in this video.
This is a complex system for defining when animations may transition, and may be even used for a two-way interaction with the application state (by firing scripting events on transitions). This is in contrast to blend trees, which simply define how animations blend.
I'm not currently using animation in three.js, but plan to eventually. Both
On 1 August 2015 at 19:47, Ben Houston email@example.com wrote:
Happy to look it over, though I'm not familiar with the existing animation
On 1 August 2015 at 20:21, Ben Houston firstname.lastname@example.org wrote: