Skeletal Animation

[guidocalvano]

Dear mr. Doob,

Great library, very inspiring. Would like to build a game in it I've been thinking about for a long long time. But, I need character animation... Are you going to put skeletal animation based on some file format I could get out of maya into Three?

Thanks,

Guido

[mrdoob]

empaempa was working on it recently, but I don't think he managed to get something solid yet.

[guidocalvano]

Any ideas on when I can expect it?

[mrdoob]

Nope...

[geoffreak]

Would you please mention this on the main page/readme, mrdoob? I have been trying to find examples of skeletal animation and wasn't aware that it wasn't available yet. Being a complete newbie to 3D animation, it would be nice to know which common 3D features are available or not.

Also, it would be really awesome if you could provide tutorials or links to tutorials for the export scripts to three.js format :)

[mrdoob]

Would you please mention this on the main page/readme, mrdoob? I have been trying to find examples of skeletal animation and wasn't aware that it wasn't available yet.

All in good time. Rather spend 5 minutes implementing the skeletal animation than writing in the readme that is not working yet ;)

[guidocalvano]

TTF based text rendering and skeletal animation are really the only still things missing in Three

[empaempa]

Skins (SkinnedMesh.js) and bones (Bone.js) and animation of bones (Animation.js) should work, just that I haven't got any models/animation that I can share in an example right now. I'll try to put together something as soon as I get some time.

Generally speaking, in the model.js that you load, there should be (V=vertex, B=bone)...

skinIndices = [ V1B1index, V1B2index, V2B1index, V2B2index... ],
skinWeights = [ V1B1influence, V1B2influence, V2B1influence, V2B2influence... ],
bones=[ { B1Setup }, { B2Setup }... ]

The BXSetup

{

  parent: BXindex,  // -1 for root bone
  pos:[x,y,z],
  rotq[x,y,z,w], // quaternion
  scl[x,y,z]

}

The animation format is quite straight forward (used for both skinned models and objects in a hierarchy):

animation: {

name: "nameOfAnimation",
length: 1201,
fps: 0.03,  // fpms actually
hierarchy: [

{ parent: -1,
keys: [

{ time: 0,
pos: [ x, y, z ],
rot: [ x, y, z, w ],
scl: [ x, y, z ] },

{ time: 620,
rot: [ x, y, z, w ] },

{ time: 1201,
pos: [ x, y, z ],
rot: [ x, y, z, w ],
scl: [ x, y, z ] } ] },

{ parent: 0,
keys: [ ...

] }

“length” is the animation length in milliseconds. “time” is a key’s time in milliseconds. “pos”, “rot” and “scl” are local coordinates (relative its parent). “rot” is a quaternion. “parent” is the parent’s index in “hierarchy”, and the root’s parent is -1.

Note that the first and last key need to match in a looping animation. First and last keys always need “pos/rot/scl”. The last key’s “time” must be equal to “length”. Keys in the middle of the animation only need one of “pos/rot/scl” to work.

The order in “hierarchy” is either dictated by the “bones” array in a rigged model (see above) or a recursed object hierarchy…

hierarchy = [];
recurse( root, hierarchy );

recurse = function( root, hierarchy ) {

  hierarchy.push( root );
  for( var c = 0; c < root.children.length; c++ ) recurse( root.children[ c ], hierarchy );

}

At this point there's no Blender-exporter for this, so somehow you need to manage this yourself. Hope that helps!

[guidocalvano]

mmmmmm

I don't use Blender... and using the editor I do use it would probably take me about a month to write the exporter. I'd have to learn the format of the editor, I'd have to learn the inner mechanics of Three... And then I'd have to build the actual exporter. I am already on a rush to finish my master project in time...

Are you planning on building some loader for some common 3D file format?

[guidocalvano]

I mean, it seems like a piece of cake for someone who already knows three and say 3DS files.

[empaempa]

Yeah, I know it's a lot of work to get this into a pipeline - sorry!

As far as I'm concerned, 3DS doesn't support skinning but I know Collada (DAE) does. I'm not sure, but there should be some kind of Collada-importer in JavaScript that you potentially could "port" to support Three.

[empaempa]

If it's a simple model, like a tube, you can probably calculate the skinWeights and skinIndices without any problem. I'll try to put together an example of this.

[guidocalvano]

I'm building an RTS. It's for the units... tubes aren't very sexy now are they... Three already reads collada files right? So the collada reader needs to be extended to read skeletal animations too. Collada is a pretty common format right? So all major editors should be able to save to collada...

Fact of the matter is, that if you guys have skeletal animation you will be the (one of the?) first open source 3D engine on webgl that can support a major 3D game.

[empaempa]

No, tubes aren't really sexy ;)

Most editors have some sort of DAE-support, so chances are that you can export that from your tool. But unfortunately, at this point Three doesn't support DAE (see https://github.com/mrdoob/three.js/issues#issue/63 for more details).

That said, there are other WebGL-libraries out there with support for DAE and I guess a read-through and "port" of one of these implementations shouldn't be that hard. You would make a lot of people very happy if you sat down and just did it :)

[mrdoob]

guidocalvano: Building a Collada Reader is a fairly hard task even for someone that knows his way around all this. It may happen that someone kindly gets on it and implements it from night to day, but it may also take a few months until it gets done. I wouldn't really depend on features that do not exist yet.

[et-nowis]

@empaempa If I read it correctly, you commented that the skinIndices and skinWeights accounts for two bones influence.
Is it possible to extend to unlimited number of influences?
Or may be at max 4 bones because some models look smoother.

[empaempa]

It's not that complicated to add more influences but you rarely see any difference above three influences - atleast that's my experience. I kept it to two as it's a good trade-off between speed, memory and looks. I don't have time right now to add more influences, but if you like to do it, SkinnedMesh.js, WebGLRenderer.js and WebGLShaders.js is the place to look and add another array of data.

Good luck! :)

[jicksta]

I see this thread is almost a year old now. Has there been any progress toward supporting skeletal animations exported from a standard tool?

[alteredq]

Not really :S.

One thing though that should somehow work is skeletal animations in Collada (these are baked into morph animations upon loading).

But still no full pipeline for GPU accelerated skinning. I should really look into this.

Any suggestions for some good models and animations? I will need something to work with (it doesn't make sense to have pipeline that works for a single model, we have been there, unfortunately most of WebGL skinning examples are like this).

For example MD2s have been real boon for our morph animation pipeline - there is plenty of models available, they all have the same structure, they are small and they are usually released with some permissive license.

Anything like that for skinned models? I remember somebody mentioning MD5s are not very good for GPU skinning. Maybe something from other game engines (Source, Unreal, Crysis, Gamebryo)? It should be something with active modding community.

[jicksta]

BlendSwap.com has a lot of CC models that have rigging:

http://www.blendswap.com/blends/category/rigs/

Here are a few that seem useful:

http://www.blendswap.com/blends/characters/base-mesh-with-simple-rig-and-animation/
http://www.blendswap.com/blends/animals/raptorian-hunter-fantasy-character-contest/
http://www.blendswap.com/blends/characters/matt-rig/
http://ivogrigull.com/blog/?page_id=127

[Digitoxin]

This seems to be the oldest unresolved issue at the moment at 3 years...

So, to update on the situation: Empaempa's skeletal animation system works quite well though it hasn't seen much love for a while, and there is a Blender exporter now that exports skeletal animations. The other model exporters probably support skeletal animation as well by now. All that's missing is blending between animations and GPU skinning.

[empaempa]

Hey!
Sure is the oldest issue ATM :)

If I where you I'd look into rewriting these parts from scratch. There's a lot in there which isn't up to par with the rest of the engine and giving it love isn't going to cut it, I think.

A good start would be changing the animation format into a proper property-curve-format (so you can animate any property you like, including transforms and materials and whatever you like). Then whip up an animation player that handles the new format, plus blending. From there I'd look into the shaders, as well, as I don't think they're optimal (don't remember the exact reason for going down the current path).

Unfortunately I've got very little time to pitch in right now but is hoping to get my soon-to-be-released Unity exporter to support any new format you come up with :)

[Digitoxin]

Unity exporter!? Excuse me while I hyperventilate for a moment. Sounds awesome!

Now, I don't think I can get a new animation system done, let alone better than what you've done already. I might try to figure out how to implement a short-term ugly hackish workaround to get something working quickly, though.

My idea would be to have Animation objects contain the current position/rotation of the skeleton in that animation, and then get the mesh's skeleton's position/rotation by somehow interpolating them. I've still got only vague notions of implementation and theory, but it seems doable.

[insominx]

I have blending working with the current system which you can a demo of here: http://www.realitymeltdown.com/WebGL/BlendExample/SkeletalAnim.html . I'll submit it as soon as I get a couple of basic animations for the character to allow for moving between idle, walk, and maybe run animations.

[mrdoob]

@insominx yay! :)

[insominx]

and here's a better example going between an idle and a walk cycle: http://webcabin.org/temp/Character%20Blending/character-blend-demo.html . I'll be gone for a week but plan on cleaning things up a bit and submitting.

[kumavis]

since #4658 has been merged, I think this can be closed