New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] AnimationUtils: Add .subclip(), clip.clone(), track.clone(). #13430

Open
wants to merge 2 commits into
base: dev
from

Conversation

Projects
None yet
9 participants
@donmccurdy
Copy link
Collaborator

donmccurdy commented Feb 25, 2018

See discussion in the forums, ClipAction select range to play. The original suggestion was to allow clamping a clip to a particular time span, but I think an API for splitting clips would be best — this makes it possible to play each section of the clip independently, or crossfade between them.

Usage:


Previous syntax
clips = THREE.AnimationUtils.splitClip( clips[ 0 ], [
  { name: 'Idle', startTime: 0 },
  { name: 'Run', startTime: 1 },
  { name: 'Samba', startTime: 2 },
] );

idleAction = mixer.clipAction( clips[ 0 ] );
runAction = mixer.clipAction( clips[ 1 ] );

idleAction.play();
setTimeout( () => {
  idleAction.crossFadeTo( runAction, 1 );
}, 1000 );

Updated syntax:

var idleClip = THREE.AnimationUtils.subclip( clip, 'idle', 0, 60 );
var runClip = THREE.AnimationUtils.subclip( clip, 'run', 60, 120 );
var sambaClip = THREE.AnimationUtils.subclip( clip, 'samba', 120, 180 );

idleAction = mixer.clipAction( idleClip );
runAction = mixer.clipAction( runClip );

idleAction.play();
setTimeout( () => {
  idleAction.crossFadeTo( runAction, 1 );
}, 1000 );

If this API seems OK, I'm glad to add docs and some unit tests. A few questions first:

  • Should the .splitClip() portion be kept out of core, e.g. AnimationClipCreator.splitClips(), instead?
  • An example model would be helpful; I suspect frames could be off by one here.
  • There's a warning in the KeyframeTrack source, IMPORTANT: We do not shift around keys to the start of the track time, because for interpolated keys this will change their values. I'm not sure what this means, or if it's relevant here. @bhouston do you recall?

/cc @looeee

@looeee

This comment has been minimized.

Copy link
Collaborator

looeee commented Feb 25, 2018

Wow, you are fast! 😁

This looks great, although I would rather a signature like:

THREE.AnimationUtils.splitClip( clip, name, from, to );

This would allow greater flexibility so, for example, if t=2 to t=3 contains a second walk animation that you don't like, you could skip it:

const walkClip = THREE.AnimationUtils.splitClip( clip, 'walk', 0, 1 );

// Note: decided not to use this as it looks silly
// const goofyWalk= THREE.AnimationUtils.splitClip( clip, 'goofyWalk', 1, 2 );

const idleClip = THREE.AnimationUtils.splitClip( clip, 'idle', 2, 3 );

Since you are using the startTime of successive stops as the implied stopTime of the preceding stop, there would be no way to skip the goofyWalk clip with your setup.

@looeee

This comment has been minimized.

Copy link
Collaborator

looeee commented Feb 25, 2018

Here is a combined animation file in JSON format:

idleWalkRun.zip

It will work with the Samba Dancing.fbx file used in the current FBXLoader example. To apply it to the model, change the loader function in the example as follows:

var loader = new THREE.FBXLoader();
loader.load( 'models/fbx/Samba Dancing.fbx', function ( object ) {

	object.mixer = new THREE.AnimationMixer( object );
	mixers.push( object.mixer );

	var action;

	var animLoader = new THREE.FileLoader();
	animLoader.setResponseType( 'json' );

	animLoader.load( 'anims.json', function( json ) {

		var clip = THREE.AnimationClip.parse( json );
		action = object.mixer.clipAction( clip );
		action.play();

	} );

	scene.add( object );

} );

Animations are at 60fps and are:

Idle: frames 0 to 198
Walk: frames 199 to 261
Run: 262 to 304

At least, that's the frames in 3DS Max, there seem to be a couple of garbage frames added at the end when I apply it to the model - these may be artefacts of my converting the animation from FBX to JSON.

@waverider404

This comment has been minimized.

Copy link

waverider404 commented Feb 25, 2018

Awesome!!! This will be a huge contribution to threejs and easy for the users!!

@mrdoob mrdoob added this to the rXX milestone Feb 27, 2018

@donmccurdy donmccurdy changed the title [WIP] AnimationUtils: Add .splitClip(), clip.clone(), track.clone(). [WIP] AnimationUtils: Add .subclip(), clip.clone(), track.clone(). Feb 27, 2018

@donmccurdy

This comment has been minimized.

Copy link
Collaborator

donmccurdy commented Feb 27, 2018

Thanks @looeee! I think I like that syntax structure better, yeah. Perhaps renaming to .subclip(...) similar to .subarray(...)?

The model is "working" (I can split the clip fine) but it seems like there are some issues: (1) garbage frames as you mention, and (2) the exported frames have been optimized I think, and there are clips that do not "contain" some of the frames that still apply to them:

-------------clip1-------------↓-------------clip2-------------↓-------------clip3-------------|
track1 ----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|---|
track2----|------------------------------------------------------------------------------------|
track3----|----|----|----|----|-------------------------------------|----|----|----|----|----|-|

Not exactly this, but gets the point across... do we consider this "bad input" and expect users to export all necessary frames for each range? Or try to preserve the last keyframe from the prior track, if the track has no keyframe at its own beginning? I've already commented out the .optimize() call in each track's constructor and that's not the source here, although it could be its own problem...

I'm confident I could create a working example in Blender, but then the point is to have a viable workflow from DCC tools that can't export multiple actions to three.js... 😕

@looeee

This comment has been minimized.

Copy link
Collaborator

looeee commented Feb 28, 2018

I've tested the FBX file that I used to generate the JSON anims - it has the same issue with garbage frames at the end, so that wasn't an artefact of conversion to JSON, however exporting the file with baked animation does remove them.

I've tested it with a couple of other tools (FBX review, Motion builder) and the unbaked animations play fine there, so I think that we should consider them "good input".

HOWEVER.... This is all probably an issue with the way that FBX stores animations (sparse by default), and is probably a bug in the FBXLoader rather than something you need to worry about here.

So here's the FBX file exported with baked animations. I would say if you can get it to work with this, that's good enough for now and I'll work on getting the FBXLoader to load the unbaked animation correctly.

idleWalkRunBAKED.zip

@takahirox

This comment has been minimized.

Copy link
Collaborator

takahirox commented Feb 28, 2018

Off the topic, but how can users/viewers know the ranges in animation data? For example, Ams - Bms for walk, Cms - Dms for run. Externally documented? This packing animation data style is new to me and I'm curious to know.

@donmccurdy

This comment has been minimized.

Copy link
Collaborator

donmccurdy commented Mar 1, 2018

Thanks @looeee I'll give that a try soon!

@takahirox — if all you have is the model, you really can't tell. Or at least not well enough to make a walk cycle loop cleanly. But in the modeling tool your artist could make a 60-frame walk cycle, 60-frame run cycle, and you'd just ask for that information when importing to Unity or something. Blender has Markers you could use to keep track of things like that, although they aren't exported I don't think.

Related: is it better to have the method take start/stop as a number of frames, rather than a time? not sure how this workflow typically is done..

@looeee

This comment has been minimized.

Copy link
Collaborator

looeee commented Mar 1, 2018

I would say splitting by frames makes more sense - that's the way animations are usually specified in asset creation tools.

@donmccurdy donmccurdy force-pushed the donmccurdy:feat-animationutils-splitclip branch from 13a20f7 to 258b462 Mar 1, 2018

@donmccurdy

This comment has been minimized.

Copy link
Collaborator

donmccurdy commented Mar 1, 2018

Updated to split by frames. Added a demo (probably not intended to submit) showing progress:

Demo: https://rawgit.com/donmccurdy/three.js/feat-animationutils-splitclip/examples/misc_animation_utils.html

^Note the demo resets at 5s, but each clip loops at least once in that time range.

Some notes:

  • Model comes in with 153 frames, rather than expected ~304. I divided all the expected start/stop frames by 2 and that was about right, but still took more tweaking, and the Run animation still has noticeable jank. Either I'm missing a frame, or there's an interpolation issue. It's like the first frames in each range are still blended with something else, maybe?
  • Had to disable .optimize() call in the KeyframeTrack constructor, or we can't split the clip properly. Optimizing after the split would be fine.
  • Not entirely sure how the fadeIn/fadeOut API works but tried to include that for good measure. 😅

So, getting better, but this still feels hard to use and I'm not really sure where the problem is yet.

@looeee

This comment has been minimized.

Copy link
Collaborator

looeee commented Mar 1, 2018

Model comes in with 153 frames, rather than expected ~304.

So something strange was happening - I exported the animations from Mixamo at 60fps, but when I loaded the files in 3DS Max they imported at 30fps. For the previous JSON example I had rescaled them back to 60fps, but perhaps I neglected to save after doing that. Sorry for confusion!

the Run animation still has noticeable jank

Again, probably an artefact of the conversion process from Mixamo - or perhaps even an issue with the clip itself.

Otherwise, looks great! If we decide to go with this model and setup for the example then I'll redo everything, get rid of the jank and make sure it's exported at 60fps.
It's a nice model, but the size is currently quite large (16mb) so perhaps we don't want to add it to the repo.

@looeee

This comment has been minimized.

Copy link
Collaborator

looeee commented Mar 1, 2018

Another thought - the model from the example I linked above (webgl_animation_skinning_blending) is really showing its age. Perhaps we could replace it with this one, or even combine the two examples into webgl_animation_skinning_subclip_blending?

@donmccurdy

This comment has been minimized.

Copy link
Collaborator

donmccurdy commented Mar 1, 2018

Ok thanks, that's more promising then 🙂

About the .optimize() thing, I don't really want to add an option to the KeyframeTrack constructor because we'd need to plumb it into the loader(s) somehow... maybe a global setting like this?

THREE.KeyframeTrack.autoOptimize = false;

// ...load clip from file...

var idleClip = THREE.AnimationUtils.subclip( clip, 'idle', 0, 60 ).optimize();
var walkClip = THREE.AnimationUtils.subclip( clip, 'walk', 0, 60 ).optimize();
@looeee

This comment has been minimized.

Copy link
Collaborator

looeee commented Mar 1, 2018

maybe a global setting

Yeah, I think that would work. Then inside the subClip method:

subclip: function ( sourceClip, name, startFrame, endFrame ) {
   
    var autoOptimiseValue = THREE.KeyframeTrack.autoOptimize;
    THREE.KeyframeTrack.autoOptimize = false;

    var clip = sourceClip.clone();

    ...

    clip.resetDuration();

    THREE.KeyframeTrack.autoOptimize = autoOptimiseValue;

    return clip;
}
   
@DenisKen

This comment has been minimized.

Copy link

DenisKen commented May 11, 2018

I've tried here, but AnimationUtils.subclip is not a function.

@donmccurdy

This comment has been minimized.

Copy link
Collaborator

donmccurdy commented May 11, 2018

@DenisKen note that this PR has not been merged.

@looeee

This comment has been minimized.

Copy link
Collaborator

looeee commented May 12, 2018

Perhaps the issue with this one is that it includes the example? @donmccurdy could you make the example a seperate PR? It would be nice to get these added.

@donmccurdy

This comment has been minimized.

Copy link
Collaborator

donmccurdy commented May 14, 2018

The main problem, I think, is that I don’t have a workflow where I can confirm this works well and solves a useful problem... sample models I’ve found that cycle through different animations are often “optimized” such that they can’t trivially be split and loop cleanly. If someone can produce such a workflow and model(s) to test I would be glad to clean this up. Otherwise it may be better to spend time on things like KhronosGroup/glTF-Blender-Exporter#166.

@looeee

This comment has been minimized.

Copy link
Collaborator

looeee commented May 14, 2018

Otherwise it may be better to spend time on things like KhronosGroup/glTF-Blender-Exporter#166.

Not everyone is using Blender. For people that use pretty much any commercial 3D software, I'm not aware of any exporters in the works. Just converters, which won't solve this issue.

@donmccurdy

This comment has been minimized.

Copy link
Collaborator

donmccurdy commented May 14, 2018

For people that use pretty much any commercial 3D software, I'm not aware of any exporters in the works...

Do you mean for glTF? Or that support multiple animation export generally? There are a couple third-party plugins for 3DS Max and Maya, although I don't know what level of quality they're at today.

In any case, I don't think we can justify merging this PR without finding at least one workflow where it works correctly with a clean loop on each clip.

@looeee

This comment has been minimized.

Copy link
Collaborator

looeee commented May 14, 2018

Do you mean for glTF?

Yes.

There are a couple third-party plugins for 3DS Max and Maya

The only glTF exporter I know of the 3DS Max is the Babylon.js exporter, which has completely failed for anything that I've tried exporting. Then again, things are changing so fast here that there could be ten more by now!

@abrakadobr

This comment has been minimized.

Copy link

abrakadobr commented Jul 14, 2018

i just want to say HUGE THANKS, and confirm this commit donmccurdy@2538285 as working (tested with cutting big clip to subclips and play them separatelly).
Thanks, @donmccurdy - I start touching animations in my project 1 hour ago, and your commit solved my "possible headache" just in time =)

@donmccurdy

This comment has been minimized.

Copy link
Collaborator

donmccurdy commented Jul 14, 2018

@abrakadobr good to hear this was helpful! Would you be willing to share a bit about your workflow for exporting animated models? The reason I haven't updated this PR lately is that I don't really have a good workflow to create animated models with animation on a single timeline... I think this PR needs at least one example model that can be split and will loop cleanly. I've been using glTF, which lets you split clips at export time, and so haven't had much need to work on this PR lately. But still happy to update it if it's useful to others.

@abrakadobr

This comment has been minimized.

Copy link

abrakadobr commented Jul 15, 2018

happy to share small peaces of code right now (peaces about animation clips), cuz it's a bit messy to see all together and code is still "dev-dirty", sorry =) and all project I hope to publish in few days, so whole code can be reviewed later =)

helper class methods:

    loadFbxAnimations(path)
    {
      return new Promise(acc=>{
        let loader = new THREE.FBXLoader()
        loader.load(path+'.fbx',(model)=>{
          acc(model.animations)
        },(progress)=>{},(err)=>{
          console.log(['FBXANIM ERR',err])
        })
      })
    }

    loadFbx(path,sc=1,shadows=true)
    {
      return new Promise(acc=>{
        let loader = new THREE.FBXLoader()
        loader.load(path+'.fbx',(model)=>{
          model.scale.set(sc,sc,sc)
          if (shadows)
            model.traverse( function ( child ) {
              if ( child.isMesh ) {
                child.castShadow = true
                child.receiveShadow = true
              }
            })
          model.mixer = new THREE.AnimationMixer( model )
          //model.a0 = model.mixer.clipAction( model.animations[0] )
          acc(model)
        },(progress)=>{},(err)=>{
          console.log(['FBX ERR',err])
        })
      })
    }

    cloneFbx(fbx)
    {
      const clone = fbx.clone(true)
      clone.animations = fbx.animations
      clone.skeleton = { bones: [], getBoneByName:(n)=>{} }

      const skinnedMeshes = {}

      fbx.traverse(node => {
        if (node.isSkinnedMesh)
          skinnedMeshes[node.name] = node
      })

      const cloneBones = {}
      const cloneSkinnedMeshes = {}

      clone.traverse(node => {
        if (node.isBone)
          cloneBones[node.name] = node
        if (node.isSkinnedMesh)
          cloneSkinnedMeshes[node.name] = node
      })

      for (let name in skinnedMeshes) {
        const skinnedMesh = skinnedMeshes[name]
        const skeleton = skinnedMesh.skeleton
        const cloneSkinnedMesh = cloneSkinnedMeshes[name]

        const orderedCloneBones = []

        for (let i=0; i<skeleton.bones.length; i++) {
          const cloneBone = cloneBones[skeleton.bones[i].name]
          orderedCloneBones.push(cloneBone)
        }

        cloneSkinnedMesh.bind(new THREE.Skeleton(orderedCloneBones, skeleton.boneInverses),cloneSkinnedMesh.matrixWorld)

        // For animation to work correctly:
        clone.skeleton.bones.push(cloneSkinnedMesh)
        clone.skeleton.bones.push(...orderedCloneBones)
      }

      return clone
    }

mixmao character class also has method to add separate loaded animation from separate files
animations object preloaded before this code and looks like { clipName: clipAnimation, otherClip: otherAnimation, ... }

    //add separate loaded animations for mixmao characters
    addSkinAnimations(animations)
    {
      if(!this.skin)
        return

      for(let clipName in animations)
      {
        let ai = this.skin.animations.length
        this.skin.animations.push(animations[clipName])
        this.actions[clipName] = this.skin.mixer.clipAction(this.skin.animations[ai])
      }
    }

i'm using this animateted model: https://www.turbosquid.com/FullPreview/Index.cfm/ID/1282563 and it has all animations together, and has txt file with description of subclips frames

somewhere, in loader object, model downloading:

      this.models.ship = await this.helper.loadFbx('3d/ship/sci_fi_aircraft',1,true)

, so, here is code of clips cutting in spaceship class:

      this.model = this.loader.helper.cloneFbx(this.loader.models.ship)
      this.mixer = new THREE.AnimationMixer( this.model )

      //.............

      //prepare spaceship animations
      let all = this.mixer.clipAction( this.model.animations[ 0 ] )
      this.subclips = {
        idle1: THREE.AnimationUtils.subclip( all._clip, 'idle1', 0, 140 ),
        idle2: THREE.AnimationUtils.subclip( all._clip, 'idle2', 145, 240 ),
        walk: THREE.AnimationUtils.subclip( all._clip, 'walk', 250, 285 ),
        //run: THREE.AnimationUtils.subclip( all._clip, 'run', 290, 305 ),
        //skillstart: THREE.AnimationUtils.subclip( all._clip, 'skillstart', 310, 320 ),
      }

      this.actions = {}
      for(let clip in this.subclips)
        this.actions[clip] = this.mixer.clipAction( this.subclips[clip] )

and small method of spaceship class to play clips

    play(action)
    {
      if (this.actions[action])
        this.actions[action].play()
    }

so how, i play clips just calling object method like

//.... let ship = new ....
ship.play('idle1')

everithing works fine, possible to load clips in searate files and push them to skinned model, and possible to load single big clip and cut it to peaces...
no idea if my code can help, hope it is =)

@donmccurdy

This comment has been minimized.

Copy link
Collaborator

donmccurdy commented Jul 16, 2018

Thanks! I'm wondering about the part where you're splitting imported animations:

{
idle1: THREE.AnimationUtils.subclip( all._clip, 'idle1', 0, 140 ),
idle2: THREE.AnimationUtils.subclip( all._clip, 'idle2', 145, 240 ),
walk: THREE.AnimationUtils.subclip( all._clip, 'walk', 250, 285 ),
...
}

Do you export FBX from Maya, 3DS Max, or another tool? With my workflow in Blender, animations often have to be "baked" to preserve IK constraints, and this inserts additional keyframes at export time.

Which, now that I'm thinking it through, is a basic flaw in this PR — the subclip ( sourceClip, name, startFrame, endFrame ) slices clips based on keyframes, not frames. An animation running from frames 0–140 does not necessarily have 140 keyframes in every track, or even the same number across tracks, unless tracks have been baked to each frame.

I will update subclip() to split clips based on frames, rather than keyframes, and then I think this will by ready to go.

@donmccurdy donmccurdy force-pushed the donmccurdy:feat-animationutils-splitclip branch 3 times, most recently from e503aec to cbd801f Jul 17, 2018

@donmccurdy donmccurdy force-pushed the donmccurdy:feat-animationutils-splitclip branch 2 times, most recently from bd80e08 to 4b5c6a2 Jul 17, 2018

@donmccurdy donmccurdy force-pushed the donmccurdy:feat-animationutils-splitclip branch from 4b5c6a2 to 9d9b1bc Jul 17, 2018

@abrakadobr

This comment has been minimized.

Copy link

abrakadobr commented Jul 17, 2018

i'm working with blender, but i'm not 3d disiner-animator. i can do some simple 3d works, but mainly i'm looking for ready models and not animating them by myself. that's why you code important for me =) - cutting big animations for small peaces (which i want to get) is super easy for me, comparing to "create animation by myself".
i don't really care - this is blender/maya/fbx or something else - if it's open to scene and keeps animations - it's all i need =) spaceship model has 3-4 formats, but only fbx loaded without any magic - with materials, textures and animations - this is ony the reason, why i use fbx.
same story about "code beauty" - if it's working, i don't care how it's cut - with frames or keyframes =))) i can use some nice method like clip.clip(), or can dig it out from privates like clip._clip..
but i understand you position, and it's very nice that you going to make all beauty =)

@mikebourbeauart

This comment has been minimized.

Copy link

mikebourbeauart commented Aug 25, 2018

I'm a tech artist who leans more towards the artistic end of things and I've been playing around with three.js for the past couple of weekends. I've also run into the problem of properly playing anim clips from an fbx.

Manually creating subclips from an entire animation is useful, especially if the animation hasn't been exported with clips. However, I'm used to game engine pipelines and they're able to read animation clips and play them back correctly without hard coding frame ranges. This got me thinking that the data must be within the file. I did some digging and found "LocalStart" and “LocalStop” parameters for each animation clip.

So, with the data already existing in the file it seems to me that it would be a much better workflow to just get the stored "LocalStart" and "LocalStop" values and convert those to frames so every clip has the proper range automatically. I found that 1 frame == 1924423250 (nanoseconds?). So, I can take any time value, divide it by 1924423250 and get the frame value.

Using this we can get the frame range. For example, my first clip has a LocalStart of 1924423250, so 1924423250/1924423250 = 1 frame. It's LocalEnd is 48110581250, so 48110581250/1924423250 = 25 frame. Giving me a frame range of 1-25 (at 24fps).

I'll do some more tests to see if things vary at different fps. I’ll also try to implement this into three.js, but I'm unfamiliar with three.js and how it parses fbx files, so hopefully someone here can beat me to it!

@mrdoob

This comment has been minimized.

Copy link
Owner

mrdoob commented Aug 25, 2018

@looeee have you see these LocalStart and LocalStop on your fbx files?

@looeee

This comment has been minimized.

Copy link
Collaborator

looeee commented Aug 25, 2018

@mrdoob I haven't personally come across those properties. Doesn't mean they don't exist though.

@mikebourbeauart the ratio of FBX time to seconds is 1 / 46186158000 for whatever reason. You are correct though that this means 1924423250 = 1/25seconds.

Do you have any FBX files with LocalStart and LocalStop that you can share?

@mikebourbeauart

This comment has been minimized.

Copy link

mikebourbeauart commented Aug 26, 2018

@mrdoob @looeee I can't link to my working repo due to character IP, but I don't care about the code because I'm just messing around. I'll make a new repo and put a temp .fbx in there.

@mikebourbeauart

This comment has been minimized.

Copy link

mikebourbeauart commented Aug 26, 2018

@mrdoob @looeee Here's the repo: https://github.com/mikebourbeauart/fbx-anim-tests

I'm using Maya's GameExporter plugin. I've exported binary and ASCII files for every FBX version possible. It looks like the parameter isn't available in FBX 2010 and lower.

If you go into the ball_anims_asc_2018.fbx file you'll see the LocalStart parameter with clip frame values beginning at line 474. These parameters are also present in the binary files.

clips:
"clip_vertical" = frames 1-10
"clip_horizontal = frames 11-20

Also, if you have any pointers for my code I'd appreciate it! Edits are welcome.

@mikebourbeauart

This comment has been minimized.

Copy link

mikebourbeauart commented Aug 27, 2018

@mrdoob I'm not sure what's up, but my pushes aren't working from my home pc. Here's a zip of the project.

fbx-anim-tests.zip

@looeee

This comment has been minimized.

Copy link
Collaborator

looeee commented Aug 27, 2018

@mikebourbeauart thanks for sharing. I can see the localStart and localStop, as well as referenceStart and referenceStart which I think we'll ignore for now.

2018-08-27_10-24-07

Here's what the SDK docs have to say about localStart:

This property stores the local time span "Start" time.

This "start" time should be seen as a time marker. Typically it would represent the whole animation starting time but its use (and update) is left to the calling application (with one exception occurring in the BakeLayers). The FBX SDK does not use this value internally and only guarantees that it will be stored to the FBX file and retrieved from it.

Default value is 0.

The issue here is that these properties are defined on the FBX.AnimationStack, and we are creating an AnimationClip from each stack. However, AnimationClip doesn't have any properties related to animation start or end time.

Really this information should sit alongside the AnimationClip so that people can use the data however they like at the application level - that is, we should be returning an array of animation objects like this:

fbx.animations = [
   {
       animationClip,
       name: 
       description,
       duration,
       localStart, 
       localStop, 
       referenceStart,
       referenceStop,    
   }
]

I think that would cover all the animation stack properties. Of course, it makes things a bit harder for people to deal with on the application side.

@mikebourbeauart

This comment has been minimized.

Copy link

mikebourbeauart commented Aug 27, 2018

@looeee It might be helpful to explain what lead me to this PR and hopefully you can lead me in the right direction because I'm not sure this feature is relevant for current situation. My issue might be due to my lack of experience with three.js and i'm doing something totally wrong so please let me know!

I'm currently running into a problem with clip playback. Only the first clip plays correctly. Any clip that comes after the first starts its animation with a pause that is the length of all previous clips.

For example, I have CLIP_A with a local range of 1-10 frames and CLIP_B with a local range of 11-20 frames. When I play CLIP_B it pauses for 1-10 frames (CLIP_A's range) then plays frames 11-20. The duration parameter of CLIP_B is also incorrect as it's set to 20 frames (1-20 local range) instead of the expected 10 frames (11-20 local range). You'll see this behavior in the zip I previously uploaded. It also contains my Maya file if you'd like to see that too.

With that said, it looks like this feature should JUST be for people who want to split a single source clip into multiple animations. Very useful if you don't have access to 3D software/are using an older FBX format without local range parameters. However, I'm expecting some sort of feature parity or equivalent to a game engine's ability to just play an animation at the clip's range. So if I set CLIP_B to play it should just play frames 11-20 by default. It would also be nice to be able to query those values per clip as well in case people want to edit those values. Something like Unity's clip.startFrame and clip.endFrame would be great.

@donmccurdy

This comment has been minimized.

Copy link
Collaborator

donmccurdy commented Aug 27, 2018

With that said, it looks like this feature should JUST be for people who want to split a single source clip into multiple animations. Very useful if you don't have access to 3D software/are using an older FBX format without local range parameters...

It seems like there are several different workflows for achieving multiple animations with FBX — cases (like Mixamo) where you might have each clip in a separate FBX file, as well as one or more mechanisms for putting multiple clips into a single FBX file. It looks like the GameExporter plugin also has an option for "Save Multiple Clip Files". Does the workflow you're describing require this Maya GameExporter plugin? And do you know if other tools (3DS Max, Blender, ...) have the ability to write clips this way?

This PR exists because exporting multiple animations to a single FBX file appears difficult, depending on your tooling. But I've only used Maya LT and Blender recently, and can't comment on best practice in game engine pipelines. If we can match those better, that's awesome. Maybe we should document this workflow for Maya users in the FBXLoader docs?

When I play CLIP_B it pauses for 1-10 frames (CLIP_A's range) then plays frames 11-20.

There is some code in this PR (shift all tracks such that clip begins at t=0) that might help with this, applied to each clip appropriately... not sure if that should be user-land code or built into FBXLoader.

we should be returning an array of animation objects like this...

@looeee maybe still an array of AnimationClip objects, with things like name and description appended to .userData?

@donmccurdy

This comment has been minimized.

Copy link
Collaborator

donmccurdy commented Aug 27, 2018

/cc @zellski does FBX2GLTF do anything particular with LocalStart / LocalStop, if you've run into them?

@mikebourbeauart

This comment has been minimized.

Copy link

mikebourbeauart commented Aug 27, 2018

Maybe we should document this workflow for Maya users in the FBXLoader docs?

I'd be happy to write up some best practices/tutorial for exporting from Maya FBX to three.js (multiple clips, single clip, using time editor to create clips, etc).

There is some code in this PR (shift all tracks such that clip begins at t=0) that might help with this, applied to each clip appropriately... not sure if that should be user-land code or built into FBXLoader.

Seems to be exactly what I'm looking for.

I can think of a few use cases where the shift shouldn't be applied to every clip coming in on load. It makes more sense to me if the shift is applied on a per-clip basis. Whether or not the shift should be applied by default is debatable though.

@zellski

This comment has been minimized.

Copy link

zellski commented Aug 28, 2018

Hi folks. I will add try to comment, with the caveats that I didn't have time to read through the above in great detail, that I'm a little vague on the finer points of animation support, and that FBX2glTF takes some liberties in this area.

With that out of the way, briefly, first, yes, those are precisely the parameters we use. For each FbxAnimStack, extract its associate FbxTakeInfo (if null, skip this animation). Then:

        FbxTime start = takeInfo->mLocalTimeSpan.GetStart();
        FbxTime end   = takeInfo->mLocalTimeSpan.GetStop();

However, because we bake animations at a fixed rate, we immediately lose the precision of that interval:

        FbxLongLong firstFrameIndex = start.GetFrameCount(eMode);
        FbxLongLong lastFrameIndex  = end.GetFrameCount(eMode);

where eMode is hardcoded to a fairly arbitrary FbxTime::eFrames24. This is a likely source of some animation errors I've seen, where the animation's beginning and end doesn't fall exactly on one of these new keyframes we're introducing for our own benefit.

Then we iterate through each frame, and for each frame, traverse the entire scene graph, letting the FBX SDK do all the heavy lifting to compute the local transform of each node at each moment in time. That local transform is what we append to the glTF animation associated with the current FbxAnimStack. We only create one animation within the exported glTF for each such stack.

For what it's worth, I found this is a pretty decent summary of FBX animation internals. As for practical workflows and the like, I do get the sense that the game exporter approach is common, as is separate FBX files for animations.

I apologise if this is a non-answer; I am happy to follow up tomorrow when I will also have more time to read through the in-depth digging you folks have been doing here.

@looeee

This comment has been minimized.

Copy link
Collaborator

looeee commented Aug 29, 2018

@zellski

where eMode is hardcoded to a fairly arbitrary FbxTime::eFrames24.

Currently in the FBXLoader we're ignoring the eMode (which is an enum that defines the frame rate) entirely. I did try using this at one point, but in the end it's just simpler to take the time from each frame, since that works well with the three.js animation system. This means that the frame rate is automatically correct.

Then we iterate through each frame, and for each frame, traverse the entire scene graph, letting the FBX SDK do all the heavy lifting to compute the local transform of each node at each moment in time.

We're doing something similar - that is, computing the local transform for each animated node for each defined time and then building up an animation track from that.

@donmccurdy, @mikebourbeauart

The duration parameter of CLIP_B is also incorrect as it's set to 20 frames (1-20 local range) instead of the expected 10 frames (11-20 local range).

There is some code in this PR (shift all tracks such that clip begins at t=0) that might help with this, applied to each clip appropriately... not sure if that should be user-land code or built into FBXLoader.

Yeah, this is what I'm thinking the issue is here - currently, the way that the second clip is created in the loader is to set the times as the are found in the FBX file. This means that the second clip starts at t=frame 11.

We previously did try shifting track start times to 0 in #13743, however it broke certain models so I reverted it in #13861. @mikebourbeauart here's a version of the loader with that PR added back in, could you test it and see if that works for you?

@mikebourbeauart

This comment has been minimized.

Copy link

mikebourbeauart commented Sep 9, 2018

@looeee yep, that's just what I was looking for!

@looeee

This comment has been minimized.

Copy link
Collaborator

looeee commented Sep 10, 2018

@mikebourbeauart great! Could you open a seperate issue for this, so that we can track it there? I'll investigate what if there's something I can do to make the loader work with your model and the other models that were broken by this fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment