Animation: Interpolants & extensibility overhaul. #7312

Merged
merged 19 commits into from Oct 25, 2015

Conversation

Projects
None yet
@tschw
Contributor

tschw commented Oct 8, 2015

What's new?

  • Smooth interpolation! More importantly, an extensible infrastructure for interpolation modes.
  • Well-defined interfaces within the system.
    This way, the internal components can easier be used in a stand-alone fashion and the system becomes easier to maintain and extend. The high level Mixer & Actions API remains 100% compatible.
  • Many bugs and inefficiencies removed (see list below). It's now several factors faster.
  • Caching for very fast .addAction / .removeAction.
  • Better readable source code in many places.

Design Issues Solved

AnimationClip is documented to be reusable. The track-cached index however did result in tons of (linear!) scans when running multiple instances of the same clip (not action) at the same time!

The data model was deeply nested (both for "standard Three" JSON and at run time). Indirections take their toll. Furthermore, the indirections were completely useless.

PropertyBindings had no clearly defined role and .getValue returned something reference or value, only usable with an extra call to a .clone utility function.

Bugs

PropertyBinding.apply: The original value was "copied" without AnimationUtils.clone (incorrect for non-reference properties), also contained a lengthy computation of a one-valued constant.

KeyframeTrack.trim: The loop index in KeyframeTrack.trim was counting into the wrong direction.

KeyframeTrack.getAt: Thenext_is_constant "optimization" did not update after an interval change and also in fact caused a slowdown (the polymorphism caused more overhead than the work saved, also it only works correctly for linear interpolation and requires a very irregular and inefficient data model).

@tschw

This comment has been minimized.

Show comment
Hide comment
@tschw

tschw Oct 8, 2015

Contributor

window-2015-10-08-18 38 41
window-2015-10-08-18 38 49

Contributor

tschw commented Oct 8, 2015

window-2015-10-08-18 38 41
window-2015-10-08-18 38 49

+
+ gui.add( API, guiNameAddRemove ).onChange( function() {
+
+ if ( API[ guiNameAddRemove ] ) {

This comment has been minimized.

@GGAlanSmithee

GGAlanSmithee Oct 8, 2015

Contributor

Can use ternary operator

@GGAlanSmithee

GGAlanSmithee Oct 8, 2015

Contributor

Can use ternary operator

This comment has been minimized.

@tschw

tschw Oct 8, 2015

Contributor

Probably having a temporary stupidity attack, but I can't figure it out. Please help!

@tschw

tschw Oct 8, 2015

Contributor

Probably having a temporary stupidity attack, but I can't figure it out. Please help!

This comment has been minimized.

@GGAlanSmithee

GGAlanSmithee Oct 8, 2015

Contributor

Its probably I who is the stupid one but:

API[ guiNameAddRemove ] ? mixer.addAction( action ) : mixer.removeAction( action )

Another approach would be to have a mixer.setAction and resolve the action (add/remove) inside it to make the consuming code cleaner.. Probably overkill though

@GGAlanSmithee

GGAlanSmithee Oct 8, 2015

Contributor

Its probably I who is the stupid one but:

API[ guiNameAddRemove ] ? mixer.addAction( action ) : mixer.removeAction( action )

Another approach would be to have a mixer.setAction and resolve the action (add/remove) inside it to make the consuming code cleaner.. Probably overkill though

@mrdoob

This comment has been minimized.

Show comment
Hide comment
@mrdoob

mrdoob Oct 8, 2015

Owner

The high level Mixer & Actions API remains 100% compatible.

I wouldn't worry about keeping it compatible to Mixer & Action if you're compromising. As this is still new code that no one relies on yet.

Owner

mrdoob commented Oct 8, 2015

The high level Mixer & Actions API remains 100% compatible.

I wouldn't worry about keeping it compatible to Mixer & Action if you're compromising. As this is still new code that no one relies on yet.

@mrdoob

This comment has been minimized.

Show comment
Hide comment
Owner

mrdoob commented Oct 8, 2015

/ping @bhouston

@mrdoob

This comment has been minimized.

Show comment
Hide comment
@mrdoob

mrdoob Oct 8, 2015

Owner

Btw, thanks a ton @tschw! 😊

Owner

mrdoob commented Oct 8, 2015

Btw, thanks a ton @tschw! 😊

@WestLangley

This comment has been minimized.

Show comment
Hide comment
@WestLangley

WestLangley Oct 8, 2015

Collaborator

@tschw Wow! And thank you for your well-written inline comments. So helpful.

@gero3 http://threejsworker.com/pullrequests.html Incredibly useful. Thanks.

Collaborator

WestLangley commented Oct 8, 2015

@tschw Wow! And thank you for your well-written inline comments. So helpful.

@gero3 http://threejsworker.com/pullrequests.html Incredibly useful. Thanks.

src/animation/AnimationClip.js
}
}
- if ( tracks.length === 0 ) {
+ if( tracks.length === 0 ) {

This comment has been minimized.

@tschw

tschw Oct 8, 2015

Contributor

This is actually code I didn't touch. The diff is against a tidier version of the code that I had to revert for pulling up...

@tschw

tschw Oct 8, 2015

Contributor

This is actually code I didn't touch. The diff is against a tidier version of the code that I had to revert for pulling up...

This comment has been minimized.

@mrdoob

mrdoob Oct 8, 2015

Owner

No worries! I'll tidy things up later 😊

@mrdoob

mrdoob Oct 8, 2015

Owner

No worries! I'll tidy things up later 😊

src/Three.js
+
+}
+
+if ( Object.assign === undefined ) {

This comment has been minimized.

@GGAlanSmithee

GGAlanSmithee Oct 8, 2015

Contributor

I guess the part below is a bit of copy paste, but there is a lot of formatting errors according to the style guidelines

@GGAlanSmithee

GGAlanSmithee Oct 8, 2015

Contributor

I guess the part below is a bit of copy paste, but there is a lot of formatting errors according to the style guidelines

This comment has been minimized.

@tschw

tschw Oct 8, 2015

Contributor

Yes, most of it is copypasta from MDN. I freestyled on .of and .EPSILON as there were none provided. These should be trivial enough.

@tschw

tschw Oct 8, 2015

Contributor

Yes, most of it is copypasta from MDN. I freestyled on .of and .EPSILON as there were none provided. These should be trivial enough.

This comment has been minimized.

@mrdoob

mrdoob Oct 8, 2015

Owner

@tschw thanks for these!

@mrdoob

mrdoob Oct 8, 2015

Owner

@tschw thanks for these!

- return this;
-
-};
+ InterpolantFactoryMethidSmooth: undefined // not yet implemented

This comment has been minimized.

@tschw

tschw Oct 8, 2015

Contributor

Bug.

@tschw

tschw Oct 8, 2015

Contributor

Bug.

This comment has been minimized.

@mrdoob

mrdoob Oct 8, 2015

Owner

Typo for sure 😜

@mrdoob

mrdoob Oct 8, 2015

Owner

Typo for sure 😜

src/animation/AnimationAction.js
+
+ var tracks = clip.tracks,
+ nTracks = tracks.length,
+ interpolants = new Array( nTracks );

This comment has been minimized.

@GGAlanSmithee

GGAlanSmithee Oct 8, 2015

Contributor

Not sure if you need (should) initalize an array with size. Probably better to just do = [] (same below)

see https://coderwall.com/p/h4xm0w/why-never-use-new-array-in-javascript for some thoughts ön this.

@GGAlanSmithee

GGAlanSmithee Oct 8, 2015

Contributor

Not sure if you need (should) initalize an array with size. Probably better to just do = [] (same below)

see https://coderwall.com/p/h4xm0w/why-never-use-new-array-in-javascript for some thoughts ön this.

This comment has been minimized.

@tschw

tschw Oct 8, 2015

Contributor

See also http://www.html5rocks.com/en/tutorials/speed/v8/

Preallocate small arrays (<64k) to correct size before using them

@tschw

tschw Oct 8, 2015

Contributor

See also http://www.html5rocks.com/en/tutorials/speed/v8/

Preallocate small arrays (<64k) to correct size before using them

This comment has been minimized.

@GGAlanSmithee

GGAlanSmithee Oct 8, 2015

Contributor

Interesting read!

Are you sure new Array(n) is actually pre-allocating sizeof n * n? The article I linked seems to suggest it only sets the length of the new array to n.

var a = new Array(10);
a[0] // returns undefined
a.length // returns 10, because of reasons.

@GGAlanSmithee

GGAlanSmithee Oct 8, 2015

Contributor

Interesting read!

Are you sure new Array(n) is actually pre-allocating sizeof n * n? The article I linked seems to suggest it only sets the length of the new array to n.

var a = new Array(10);
a[0] // returns undefined
a.length // returns 10, because of reasons.

This comment has been minimized.

@mrdoob

This comment has been minimized.

Show comment
Hide comment
@mrdoob

mrdoob Oct 8, 2015

Owner

@GGAlanSmithee lets leave the formatting bits out of the review by now.

Owner

mrdoob commented Oct 8, 2015

@GGAlanSmithee lets leave the formatting bits out of the review by now.

src/animation/AnimationAction.js
@@ -52,19 +80,20 @@ THREE.AnimationAction.prototype = {
this.actionTime = this.actionTime + clipDeltaTime;
- if ( this.loop === THREE.LoopOnce ) {
+ if( this.loop === THREE.LoopOnce ) {

This comment has been minimized.

@GGAlanSmithee

GGAlanSmithee Oct 8, 2015

Contributor

All if statements below needs formatting

@GGAlanSmithee

GGAlanSmithee Oct 8, 2015

Contributor

All if statements below needs formatting

This comment has been minimized.

@mrdoob

mrdoob Oct 8, 2015

Owner

Yes, I'll do it later.

@mrdoob

mrdoob Oct 8, 2015

Owner

Yes, I'll do it later.

@GGAlanSmithee

This comment has been minimized.

Show comment
Hide comment
@GGAlanSmithee

GGAlanSmithee Oct 8, 2015

Contributor

Sorry! No more comments about formatting.

Contributor

GGAlanSmithee commented Oct 8, 2015

Sorry! No more comments about formatting.

@tschw

This comment has been minimized.

Show comment
Hide comment
@tschw

tschw Oct 8, 2015

Contributor

@mrdoob

@GGAlanSmithee lets leave the formatting bits out of the review by now.

Awwww.... Too late. I used a separate commit for the code I didn't touch. Shall I reset --hard HEAD^..force-push it?

Contributor

tschw commented Oct 8, 2015

@mrdoob

@GGAlanSmithee lets leave the formatting bits out of the review by now.

Awwww.... Too late. I used a separate commit for the code I didn't touch. Shall I reset --hard HEAD^..force-push it?

@tschw

This comment has been minimized.

Show comment
Hide comment
@tschw

tschw Oct 8, 2015

Contributor

Shall I reset --hard HEAD^..force-push it?

Took that commit out for now. We can re-add it later. It's enough of a rewrite already...

Contributor

tschw commented Oct 8, 2015

Shall I reset --hard HEAD^..force-push it?

Took that commit out for now. We can re-add it later. It's enough of a rewrite already...

@tschw

This comment has been minimized.

Show comment
Hide comment
@tschw

tschw Oct 8, 2015

Contributor

Also squashed polyfill formatting into the corresponding commits. Once you start gitting...

Contributor

tschw commented Oct 8, 2015

Also squashed polyfill formatting into the corresponding commits. Once you start gitting...

@tschw

This comment has been minimized.

Show comment
Hide comment
@tschw

tschw Oct 8, 2015

Contributor

Still bugs me that I've lost some performance over an earlier iteration. It used to be a factor of 5 on Firefox and 3 on Chrome :-(.

Seems the measurements spike out quite a bit, so I probably just had a lucky streak there. Good enough. Enough! Once you start profiling... :-)

Contributor

tschw commented Oct 8, 2015

Still bugs me that I've lost some performance over an earlier iteration. It used to be a factor of 5 on Firefox and 3 on Chrome :-(.

Seems the measurements spike out quite a bit, so I probably just had a lucky streak there. Good enough. Enough! Once you start profiling... :-)

src/animation/AnimationMixer.js
+
+ // remove from array-based unordered set
+ actions[ index ] = actions[ actions.length - 1 ];
+ actions.pop();

This comment has been minimized.

@GGAlanSmithee

GGAlanSmithee Oct 8, 2015

Contributor

Can't these two lines become one? :
actions[ index ] = actions.pop();

@GGAlanSmithee

GGAlanSmithee Oct 8, 2015

Contributor

Can't these two lines become one? :
actions[ index ] = actions.pop();

This comment has been minimized.

@tschw

tschw Oct 8, 2015

Contributor

Another mean question for a job interview ;-).
I guess not, because it will re-add the element if it's the last in the array.

@tschw

tschw Oct 8, 2015

Contributor

Another mean question for a job interview ;-).
I guess not, because it will re-add the element if it's the last in the array.

This comment has been minimized.

@GGAlanSmithee

GGAlanSmithee Oct 8, 2015

Contributor

Guess I wouldn't have gotten the job then ;) Lucky me, I already have one :)

@GGAlanSmithee

GGAlanSmithee Oct 8, 2015

Contributor

Guess I wouldn't have gotten the job then ;) Lucky me, I already have one :)

This comment has been minimized.

@mrdoob

mrdoob Oct 8, 2015

Owner

Another mean question for a job interview ;-)

Haha, that made me laugh 😄

@mrdoob

mrdoob Oct 8, 2015

Owner

Another mean question for a job interview ;-)

Haha, that made me laugh 😄

This comment has been minimized.

@tschw

tschw Oct 8, 2015

Contributor

Was inspired by the article @GGAlanSmithee cited above, "suggesting" stuff like [] + {} for that :-).

@tschw

tschw Oct 8, 2015

Contributor

Was inspired by the article @GGAlanSmithee cited above, "suggesting" stuff like [] + {} for that :-).

src/animation/AnimationUtils.js
- return a;
- },
+ // fuzz-free, array-based Quaternion SLERP operation
+ slerp: function( dst, dstOffset, src, srcOffset0, srcOffset1, t ) {

This comment has been minimized.

@bhouston

bhouston Oct 8, 2015

Contributor

I would put an array-based interpolator functions into their parent class as static functions. This way when other people want to do fast interpolation they can easily reuse them. Thus this could be moved to Quaternion.slerpFlat() or somethig like this.

@bhouston

bhouston Oct 8, 2015

Contributor

I would put an array-based interpolator functions into their parent class as static functions. This way when other people want to do fast interpolation they can easily reuse them. Thus this could be moved to Quaternion.slerpFlat() or somethig like this.

This comment has been minimized.

@tschw

tschw Oct 9, 2015

Contributor

Yes, I like it! In fact, I wasn't all too happy with its location in the Utils bucket either.

@tschw

tschw Oct 9, 2015

Contributor

Yes, I like it! In fact, I wasn't all too happy with its location in the Utils bucket either.

This comment has been minimized.

@bhouston

bhouston Oct 9, 2015

Contributor

We should likely add this to the unit test framework as well.

@bhouston

bhouston Oct 9, 2015

Contributor

We should likely add this to the unit test framework as well.

This comment has been minimized.

@tschw

tschw Oct 9, 2015

Contributor

The factorization / generalization (it allows different source arrays now) is in.

We should likely add this to the unit test framework as well.

Will do once I know how to really test it:

The current unit test for .slerp is very weak, since it only tests the two boundary cases t e { 0, 1 } that are specifically addressed in Quaternion.prototype.slerp :-). Open question: How to make it better?

/ping @simonThiele is the testing expert IIRC.

For math stuff, I guess we'd actually have to properly specify expected error margins and have comparison functions for those...

@tschw

tschw Oct 9, 2015

Contributor

The factorization / generalization (it allows different source arrays now) is in.

We should likely add this to the unit test framework as well.

Will do once I know how to really test it:

The current unit test for .slerp is very weak, since it only tests the two boundary cases t e { 0, 1 } that are specifically addressed in Quaternion.prototype.slerp :-). Open question: How to make it better?

/ping @simonThiele is the testing expert IIRC.

For math stuff, I guess we'd actually have to properly specify expected error margins and have comparison functions for those...

This comment has been minimized.

@simonThiele

simonThiele Oct 9, 2015

Contributor

Could write tests for it if you want :) Please tell me to avoid working on the same stuff. (Or why you pinged me?)

@simonThiele

simonThiele Oct 9, 2015

Contributor

Could write tests for it if you want :) Please tell me to avoid working on the same stuff. (Or why you pinged me?)

src/animation/KeyframeTrack.js
// ensure we do not get a GarbageInGarbageOut situation, make sure tracks are at least minimally viable
// One could eventually ensure that all key.values in a track are all of the same type (otherwise interpolation makes no sense.)
validate: function() {
- var prevKey = null;
+ if ( this.getValueSize() % 1 !== 0 ) {

This comment has been minimized.

@bhouston

bhouston Oct 8, 2015

Contributor

This must be a trick of some sort? Maybe we should state in a comment above what it is checking for.

@bhouston

bhouston Oct 8, 2015

Contributor

This must be a trick of some sort? Maybe we should state in a comment above what it is checking for.

This comment has been minimized.

@tschw

tschw Oct 9, 2015

Contributor

This must be a trick of some sort?

Not really. Just the modulus operator working for floats, actually. Same as fract in GLSL.

Maybe we should state in a comment above what it is checking for.

Something like

// Note: The result of the modulus operator includes the partial remainder in JavaScript.

?

@tschw

tschw Oct 9, 2015

Contributor

This must be a trick of some sort?

Not really. Just the modulus operator working for floats, actually. Same as fract in GLSL.

Maybe we should state in a comment above what it is checking for.

Something like

// Note: The result of the modulus operator includes the partial remainder in JavaScript.

?

This comment has been minimized.

@bhouston

bhouston Oct 9, 2015

Contributor

Could you add fract method to THREE.Math and add a unit test or something?

@bhouston

bhouston Oct 9, 2015

Contributor

Could you add fract method to THREE.Math and add a unit test or something?

This comment has been minimized.

@bhouston

bhouston Oct 9, 2015

Contributor

The other way to be more obvious, as well as faster, would be ( this.values.length % this.times.length ) === 0.

@bhouston

bhouston Oct 9, 2015

Contributor

The other way to be more obvious, as well as faster, would be ( this.values.length % this.times.length ) === 0.

This comment has been minimized.

@tschw

tschw Oct 9, 2015

Contributor

Could you add fract method to THREE.Math and add a unit test or something?

Please, let's not do that!

I dislike convenience mumbo-jumbo that can be expressed directly in terms of the programming language: It takes already way too long for the JIT to get an app to full framerate. Fine-grained factorization, as appealing as it sometimes seems, makes this situation worse. Same goes for GLSL, BTW: I know examples where using many functions causes insane compilation / optimization times with some widely-used implementations - easily over a minute (!).

Could we just learn the relevant parts of the languages and use them directly, instead of trying to shape them into something foolproof? :-)

The behavior of the modulus operator is clearly defined for JavaScript. Since there is no fmod in Math, no JS implementation can have gotten around it. However, I rewrote the offending condition to valueSize - Math.floor( valueSize ) !== 0.

The other way to be more obvious, as well as faster, would be ( this.values.length % this.times.length ) === 0.

The assumption that values and times actually correspond was intended to be a default, not a requirement! I also gave .validate a proper cleanup, as it was not consistently following that scheme. Interpolant also only touches values, in methods that can be overridden separately, from getAt BTW.

See the extensibility note in BooleanKeyframeTrack for an example use case.

@tschw

tschw Oct 9, 2015

Contributor

Could you add fract method to THREE.Math and add a unit test or something?

Please, let's not do that!

I dislike convenience mumbo-jumbo that can be expressed directly in terms of the programming language: It takes already way too long for the JIT to get an app to full framerate. Fine-grained factorization, as appealing as it sometimes seems, makes this situation worse. Same goes for GLSL, BTW: I know examples where using many functions causes insane compilation / optimization times with some widely-used implementations - easily over a minute (!).

Could we just learn the relevant parts of the languages and use them directly, instead of trying to shape them into something foolproof? :-)

The behavior of the modulus operator is clearly defined for JavaScript. Since there is no fmod in Math, no JS implementation can have gotten around it. However, I rewrote the offending condition to valueSize - Math.floor( valueSize ) !== 0.

The other way to be more obvious, as well as faster, would be ( this.values.length % this.times.length ) === 0.

The assumption that values and times actually correspond was intended to be a default, not a requirement! I also gave .validate a proper cleanup, as it was not consistently following that scheme. Interpolant also only touches values, in methods that can be overridden separately, from getAt BTW.

See the extensibility note in BooleanKeyframeTrack for an example use case.

This comment has been minimized.

@bhouston

bhouston Oct 9, 2015

Contributor

See the extensibility note in BooleanKeyframeTrack for an example use case.

There is actually a huge amount of literature on compressed animation formats. Compressing the boolean tracks that way is nice, but it should be an option because they are not easily editable after that is done. I'd suggest having new classes to represent the compressed formats if their internal formats differ.

The optimize function for keyframe tracks can easily be extended to do much more useful optimizations, but again this isn't that useful in the context of playback because you really do not want to do it at load time.

In the past I've written linear optimizers with thresholds -- it just iteratively looks at keys that can be removed that can be reconstructed by linear interpolations of the resulting keyframes. There is more here:

https://docs.unrealengine.com/latest/INT/Engine/Animation/Sequences/index.html#compression

Some of the more advanced stuff applies compression on the data itself using size optimal representaions, rather than just removing keys:

https://engineering.riotgames.com/news/compressing-skeletal-animation-data

But all this should not be part of this PR.

@bhouston

bhouston Oct 9, 2015

Contributor

See the extensibility note in BooleanKeyframeTrack for an example use case.

There is actually a huge amount of literature on compressed animation formats. Compressing the boolean tracks that way is nice, but it should be an option because they are not easily editable after that is done. I'd suggest having new classes to represent the compressed formats if their internal formats differ.

The optimize function for keyframe tracks can easily be extended to do much more useful optimizations, but again this isn't that useful in the context of playback because you really do not want to do it at load time.

In the past I've written linear optimizers with thresholds -- it just iteratively looks at keys that can be removed that can be reconstructed by linear interpolations of the resulting keyframes. There is more here:

https://docs.unrealengine.com/latest/INT/Engine/Animation/Sequences/index.html#compression

Some of the more advanced stuff applies compression on the data itself using size optimal representaions, rather than just removing keys:

https://engineering.riotgames.com/news/compressing-skeletal-animation-data

But all this should not be part of this PR.

+ * @author tschw
+ */
+
+THREE.SmoothInterpolant = function( times, values, stride, result ) {

This comment has been minimized.

@bhouston

bhouston Oct 9, 2015

Contributor

I would prefer to call its by its technical name. There are a ton of different smooth interpolators. Maybe just CubicInterpolator would be better from my perspective.

@bhouston

bhouston Oct 9, 2015

Contributor

I would prefer to call its by its technical name. There are a ton of different smooth interpolators. Maybe just CubicInterpolator would be better from my perspective.

This comment has been minimized.

@tschw

tschw Oct 9, 2015

Contributor

Yes, much better. Not at least because it reads more encouraging to implement more sophisticated smooth interpolants.

@tschw

tschw Oct 9, 2015

Contributor

Yes, much better. Not at least because it reads more encouraging to implement more sophisticated smooth interpolants.

+
+ constructor: THREE.LinearInterpolant,
+
+ _interpolate: function( i1, t0, t, t1 ) {

This comment has been minimized.

@bhouston

bhouston Oct 9, 2015

Contributor

I'd prefer descriptive variable names. As the original author of the mixer, if I have to think about what these are for a bit, then it is likely non-obvious. I am assuming it is index1, time0, time, time1? I guess that time0 and time1 are the times of the surrounding keys and time is the current time? I'd suggest somethingl along the lines of: keyTime0, keyTime1, keyIndex1, currentTime.

@bhouston

bhouston Oct 9, 2015

Contributor

I'd prefer descriptive variable names. As the original author of the mixer, if I have to think about what these are for a bit, then it is likely non-obvious. I am assuming it is index1, time0, time, time1? I guess that time0 and time1 are the times of the surrounding keys and time is the current time? I'd suggest somethingl along the lines of: keyTime0, keyTime1, keyIndex1, currentTime.

This comment has been minimized.

@tschw

tschw Oct 9, 2015

Contributor

Well, in this (pretty much mathematical) context, the names are descriptive. You just proved it by guessing them right :-). The reason I used them is they are concise and common, and I don't really like very long lines in source files, as resulting from using long speaking names in mathematical formulae: They make files harder to read, edit, merge and debug.

I used keyTime and prevKeyTime in higher level code, making it clear what (just, "unsubscripted") index refers to...

The Interpolant family of classes does not depend on animation stuff. In fact, we're one layer below. Therefore we're not really talking about "times" and "keyframes" or anything "current", but about a parameter domain, commonly using the variable t in literature, subscripted for constants in respect to a line segment or an entire curve (depending on the context) and indices of samples, commonly i in literature, IOW "times" and "keyframes" are only the context-dependent actuals at the call site, not really the formal ones;l t may also be the parameter of a parametric curve that defines its tessellation and the samples are in fact "control points" in this context, other possible context include numeric methods (fully abstract), audio ("samples" on a time domain, its scale being the pitch)...

I think we should move the interpolants out of the animation folder. Maybe to math. Also seems to go well your suggestion to use them for curves.

Opinions anyone?

@tschw

tschw Oct 9, 2015

Contributor

Well, in this (pretty much mathematical) context, the names are descriptive. You just proved it by guessing them right :-). The reason I used them is they are concise and common, and I don't really like very long lines in source files, as resulting from using long speaking names in mathematical formulae: They make files harder to read, edit, merge and debug.

I used keyTime and prevKeyTime in higher level code, making it clear what (just, "unsubscripted") index refers to...

The Interpolant family of classes does not depend on animation stuff. In fact, we're one layer below. Therefore we're not really talking about "times" and "keyframes" or anything "current", but about a parameter domain, commonly using the variable t in literature, subscripted for constants in respect to a line segment or an entire curve (depending on the context) and indices of samples, commonly i in literature, IOW "times" and "keyframes" are only the context-dependent actuals at the call site, not really the formal ones;l t may also be the parameter of a parametric curve that defines its tessellation and the samples are in fact "control points" in this context, other possible context include numeric methods (fully abstract), audio ("samples" on a time domain, its scale being the pitch)...

I think we should move the interpolants out of the animation folder. Maybe to math. Also seems to go well your suggestion to use them for curves.

Opinions anyone?

+ * @author tschw
+ */
+
+THREE.SlerpInterpolant = function( times, values, stride, result ) {

This comment has been minimized.

@bhouston

bhouston Oct 9, 2015

Contributor

You could call this QuaternionLinearInterpolatant. Makes it clearer that it is akin to LinearInterpolatant. And then we can introduce a QuaternionCubicInterpolant/QuaternionSmoothInterpolant as the analogy to SmoothInterpolatant.

@bhouston

bhouston Oct 9, 2015

Contributor

You could call this QuaternionLinearInterpolatant. Makes it clearer that it is akin to LinearInterpolatant. And then we can introduce a QuaternionCubicInterpolant/QuaternionSmoothInterpolant as the analogy to SmoothInterpolatant.

This comment has been minimized.

@tschw

tschw Oct 9, 2015

Contributor

Yes, much better, again.

@tschw

tschw Oct 9, 2015

Contributor

Yes, much better, again.

+
+ constructor: THREE.SmoothInterpolant,
+
+ DefaultParameters: {

This comment has been minimized.

@bhouston

bhouston Oct 9, 2015

Contributor

Interesting to have the velocities. Usually I see people control this via just having more keys, rather than additional parameters.

I do know of Interpolators with a lot of parameters, but they are generally not used in real-time contexts -- like the bias/continuity/tensition smooth interpolator,

@bhouston

bhouston Oct 9, 2015

Contributor

Interesting to have the velocities. Usually I see people control this via just having more keys, rather than additional parameters.

I do know of Interpolators with a lot of parameters, but they are generally not used in real-time contexts -- like the bias/continuity/tensition smooth interpolator,

This comment has been minimized.

@bhouston

bhouston Oct 9, 2015

Contributor

If you get rid of these parameters, it will reduce the if's in this function and speed it up.

@bhouston

bhouston Oct 9, 2015

Contributor

If you get rid of these parameters, it will reduce the if's in this function and speed it up.

This comment has been minimized.

@tschw

tschw Oct 9, 2015

Contributor

I do know of Interpolators with a lot of parameters, but they are generally not used in real-time contexts -- like the bias/continuity/tensition smooth interpolator,

I don't remember the names of the inventors, but these are the Lightwave / Max splines :-).

Interesting to have the velocities.

Which velocities do you mean? The values you are referring to are Boolean ones, that specify how the spline behaves at the ends when running out of keyframes. These are needed to implement proper looping (which I haven't done yet, but .parameters is defined as an external structure, so it can be shared and we don't have to tell every single track that the loop count has reached zero and we're going to stop, for example).

Inside a loop, we want a "natural spline", IOW one that turns into a straight line (.zeroVelocityAtStart / End set to false). When starting and stopping, we want the spline to accelerate and decelerate, respectively (.zeroVelocityAtStart / End set to true).

@tschw

tschw Oct 9, 2015

Contributor

I do know of Interpolators with a lot of parameters, but they are generally not used in real-time contexts -- like the bias/continuity/tensition smooth interpolator,

I don't remember the names of the inventors, but these are the Lightwave / Max splines :-).

Interesting to have the velocities.

Which velocities do you mean? The values you are referring to are Boolean ones, that specify how the spline behaves at the ends when running out of keyframes. These are needed to implement proper looping (which I haven't done yet, but .parameters is defined as an external structure, so it can be shared and we don't have to tell every single track that the loop count has reached zero and we're going to stop, for example).

Inside a loop, we want a "natural spline", IOW one that turns into a straight line (.zeroVelocityAtStart / End set to false). When starting and stopping, we want the spline to accelerate and decelerate, respectively (.zeroVelocityAtStart / End set to true).

This comment has been minimized.

@bhouston

bhouston Oct 9, 2015

Contributor

I don't remember the names of the inventors, but these are the Lightwave / Max splines :-).

Both Lightwave and Max support TBC splines as defined here:

https://en.wikipedia.org/wiki/Kochanek%E2%80%93Bartels_spline

http://knowledge.autodesk.com/support/3ds-max/learn-explore/caas/CloudHelp/cloudhelp/2015/ENU/3DSMax/files/GUID-40C759C5-80AA-458B-BB57-145A78D8907F-htm.html

The values you are referring to are Boolean ones, that specify how the spline behaves at the ends when running out of keyframes.

I thought you could do all that with just additional keys -- you can add extra keys at the beginning and the end. I always though artists wanted to control all this through setting keys. I guess this is an implementation choice? Is there a reference design you are following?

@bhouston

bhouston Oct 9, 2015

Contributor

I don't remember the names of the inventors, but these are the Lightwave / Max splines :-).

Both Lightwave and Max support TBC splines as defined here:

https://en.wikipedia.org/wiki/Kochanek%E2%80%93Bartels_spline

http://knowledge.autodesk.com/support/3ds-max/learn-explore/caas/CloudHelp/cloudhelp/2015/ENU/3DSMax/files/GUID-40C759C5-80AA-458B-BB57-145A78D8907F-htm.html

The values you are referring to are Boolean ones, that specify how the spline behaves at the ends when running out of keyframes.

I thought you could do all that with just additional keys -- you can add extra keys at the beginning and the end. I always though artists wanted to control all this through setting keys. I guess this is an implementation choice? Is there a reference design you are following?

This comment has been minimized.

@bhouston

bhouston Oct 9, 2015

Contributor

For another reference here is Unity's AnimationCurve, which is analogous:

http://docs.unity3d.com/ScriptReference/AnimationCurve.html

They have a mode for pre and post, which is more flexible than the loop, pingpong, etc, flag, but I believe it doesn't change the behavior of the curve between the start and end times.

@bhouston

bhouston Oct 9, 2015

Contributor

For another reference here is Unity's AnimationCurve, which is analogous:

http://docs.unity3d.com/ScriptReference/AnimationCurve.html

They have a mode for pre and post, which is more flexible than the loop, pingpong, etc, flag, but I believe it doesn't change the behavior of the curve between the start and end times.

This comment has been minimized.

@tschw

tschw Oct 9, 2015

Contributor

The values you are referring to are Boolean ones, that specify how the spline behaves at the ends when running out of keyframes.

you can add extra keys at the beginning and the end.

You can. But once you add keys for a smooth start / end, the sequence won't loop smoothly anymore. Conversely, once you prep a clip for looping, it wont have a smooth start / stop.

Is there a reference design you are following?

Yes, I am following your design: Since the current system defines the loop mode as part of the Action and not the Clip, we must be able to smoothly start / stop and loop, right? Ideally without duplicating all the data.

For another reference here is Unity's AnimationCurve, which is analogous:
http://docs.unity3d.com/ScriptReference/AnimationCurve.html

These are weighted, taking an extra scalar per keyframe as input.

I guess this is an implementation choice?

Seeing their preWrapMode / postWrapMode (just being the loop mode) for the parameterization, I am tempted to call my boolean modes a half-baked quick shot. :-) With this information we can actually use the correct data value and it will work without imposing constraints on the data for smoothness. I'll adopt them.

We still have to switch that mode when starting / stopping and I would very much appreciate your help on that part.

@tschw

tschw Oct 9, 2015

Contributor

The values you are referring to are Boolean ones, that specify how the spline behaves at the ends when running out of keyframes.

you can add extra keys at the beginning and the end.

You can. But once you add keys for a smooth start / end, the sequence won't loop smoothly anymore. Conversely, once you prep a clip for looping, it wont have a smooth start / stop.

Is there a reference design you are following?

Yes, I am following your design: Since the current system defines the loop mode as part of the Action and not the Clip, we must be able to smoothly start / stop and loop, right? Ideally without duplicating all the data.

For another reference here is Unity's AnimationCurve, which is analogous:
http://docs.unity3d.com/ScriptReference/AnimationCurve.html

These are weighted, taking an extra scalar per keyframe as input.

I guess this is an implementation choice?

Seeing their preWrapMode / postWrapMode (just being the loop mode) for the parameterization, I am tempted to call my boolean modes a half-baked quick shot. :-) With this information we can actually use the correct data value and it will work without imposing constraints on the data for smoothness. I'll adopt them.

We still have to switch that mode when starting / stopping and I would very much appreciate your help on that part.

+ pp = p * p,
+ ppp = pp * p;
+
+ // evaluate polynomials

This comment has been minimized.

@bhouston

bhouston Oct 9, 2015

Contributor

I wonder if it is possible to pull out the cubic interpolator weigh calculation into a utility function that can be shared in a few different places. We already have cubic curves in ThreeJS. One could pass in an Float32Array to store sP, s0, s1, sN results.

@bhouston

bhouston Oct 9, 2015

Contributor

I wonder if it is possible to pull out the cubic interpolator weigh calculation into a utility function that can be shared in a few different places. We already have cubic curves in ThreeJS. One could pass in an Float32Array to store sP, s0, s1, sN results.

This comment has been minimized.

@bhouston

bhouston Oct 9, 2015

Contributor

I wonder if it is possible to pull out some of the interpolator code from Curve Utils and coodinate it with this? @zz85 The code I am referring to is here: https://github.com/mrdoob/three.js/blob/master/src/extras/core/Curve.js#L273

Probably best to do that as a separate PR though.

@bhouston

bhouston Oct 9, 2015

Contributor

I wonder if it is possible to pull out some of the interpolator code from Curve Utils and coodinate it with this? @zz85 The code I am referring to is here: https://github.com/mrdoob/three.js/blob/master/src/extras/core/Curve.js#L273

Probably best to do that as a separate PR though.

This comment has been minimized.

@tschw

tschw Oct 9, 2015

Contributor

I wonder if it is possible to pull out some of the interpolator code from Curve Utils and coodinate it with this?

For curves, t can be chosen to be uniform, while for interpolation over time, t changes its scale all the time. There are certain curve types that are constructed by changing the scale of t, in general however, interpolation over time is more constrained than curve rendering.

Since I don't have tangents or weights as input, I multiplied the tangent computation directly into the matrix of polynomial coefficients, transforming its base to refer to the control points directly. The matrix depends on the scale of the intervals.

When the matrix is multiplied with the powers of the parameter, it spits out a vector of linear weights for the four control points.

@tschw

tschw Oct 9, 2015

Contributor

I wonder if it is possible to pull out some of the interpolator code from Curve Utils and coodinate it with this?

For curves, t can be chosen to be uniform, while for interpolation over time, t changes its scale all the time. There are certain curve types that are constructed by changing the scale of t, in general however, interpolation over time is more constrained than curve rendering.

Since I don't have tangents or weights as input, I multiplied the tangent computation directly into the matrix of polynomial coefficients, transforming its base to refer to the control points directly. The matrix depends on the scale of the intervals.

When the matrix is multiplied with the powers of the parameter, it spits out a vector of linear weights for the four control points.

src/animation/AnimationUtils.js
+
+ if ( value[ 'splice' ] !== undefined ) {
+
+ values.push.apply( values, value );

This comment has been minimized.

@bhouston

bhouston Oct 9, 2015

Contributor

What is the difference between this line and the values.push( value ); expression in the "else" block? Maybe add a comment to address the thing that I am obviously missing.

@bhouston

bhouston Oct 9, 2015

Contributor

What is the difference between this line and the values.push( value ); expression in the "else" block? Maybe add a comment to address the thing that I am obviously missing.

This comment has been minimized.

@tschw

tschw Oct 9, 2015

Contributor

Function.prototype.apply takes ( this, arguments ) and e.g. array.push( 1 ,2, 3 ) appends three values to array.

@tschw

tschw Oct 9, 2015

Contributor

Function.prototype.apply takes ( this, arguments ) and e.g. array.push( 1 ,2, 3 ) appends three values to array.

This comment has been minimized.

@bhouston

bhouston Oct 9, 2015

Contributor

Right. I confused it with Function.prototype.call. Maybe add a comment to explain why a lack of value[ 'splice' ] implies this? I guess the 'splice' check is a fast way to check if it is an Array? Is there a more obvious type checking method?

@bhouston

bhouston Oct 9, 2015

Contributor

Right. I confused it with Function.prototype.call. Maybe add a comment to explain why a lack of value[ 'splice' ] implies this? I guess the 'splice' check is a fast way to check if it is an Array? Is there a more obvious type checking method?

This comment has been minimized.

@bhouston

bhouston Oct 9, 2015

Contributor

I wonder if the above function can be split so that it does the value check once per array rather than once per value -- would likely be clearer and faster. Might be more reusable too.

@bhouston

bhouston Oct 9, 2015

Contributor

I wonder if the above function can be split so that it does the value check once per array rather than once per value -- would likely be clearer and faster. Might be more reusable too.

This comment has been minimized.

@tschw

tschw Oct 9, 2015

Contributor

I wonder if the above function can be split so that it does the value check once per array rather than once per value -- would likely be clearer and faster. Might be more reusable too.

You tell me. It's a factorization of your code (last function in that source file). Used by convertTrack and for the old JSON.

@tschw

tschw Oct 9, 2015

Contributor

I wonder if the above function can be split so that it does the value check once per array rather than once per value -- would likely be clearer and faster. Might be more reusable too.

You tell me. It's a factorization of your code (last function in that source file). Used by convertTrack and for the old JSON.

@bhouston

This comment has been minimized.

Show comment
Hide comment
@bhouston

bhouston Oct 9, 2015

Contributor

Sweet! The flat array usage is great and a needed improvement. I always prefer to favor correctness first which is why I started there.

The trim command index direction bug is hilarous -- good catch!

Contributor

bhouston commented Oct 9, 2015

Sweet! The flat array usage is great and a needed improvement. I always prefer to favor correctness first which is why I started there.

The trim command index direction bug is hilarous -- good catch!

@bhouston

This comment has been minimized.

Show comment
Hide comment
@bhouston

bhouston Oct 9, 2015

Contributor

Trashing of CacheIndex in Interpolant

One real issue with the current design you may want to address is that you are still using the cachedIndex optimization pattern in the Interpolatant. It is actually suboptimal because if I am reading your code correctly, the Interpolatant is part of the KeyframeTrack, which as part of a Clip may be shared between multiple different Actions being played different times. This will cause trashing of the cachedIndex because well the cache isn't useful in this case. Although it will cause the Interpolant to rely more on the binary search method, somewhat unnecesarily.

I was going to move the cachedIndex into something that is Action specific if you are going to use it. If you can tie it with an Action that has high time coherence, it should be very effective. In fact, nearly all animations are played forward at a rate where there is less than one keyframe per rendered animation frame (because more than that would wasteful) -- thus local searches is nearly optimal as long as the cacheIndex isn't thrashing.

Contributor

bhouston commented Oct 9, 2015

Trashing of CacheIndex in Interpolant

One real issue with the current design you may want to address is that you are still using the cachedIndex optimization pattern in the Interpolatant. It is actually suboptimal because if I am reading your code correctly, the Interpolatant is part of the KeyframeTrack, which as part of a Clip may be shared between multiple different Actions being played different times. This will cause trashing of the cachedIndex because well the cache isn't useful in this case. Although it will cause the Interpolant to rely more on the binary search method, somewhat unnecesarily.

I was going to move the cachedIndex into something that is Action specific if you are going to use it. If you can tie it with an Action that has high time coherence, it should be very effective. In fact, nearly all animations are played forward at a rate where there is less than one keyframe per rendered animation frame (because more than that would wasteful) -- thus local searches is nearly optimal as long as the cacheIndex isn't thrashing.

src/animation/Interpolant.js
+
+ constructor: THREE.Intepolant,
+
+ getAt: function( time ) {

This comment has been minimized.

@bhouston

bhouston Oct 9, 2015

Contributor

I'd suggest unit tests or something similar to ensure correctness of this function. It is involved and there area lot of edge cases that we need to ensure stay covered. It may be correct now, and it is important to keep it that way.

@bhouston

bhouston Oct 9, 2015

Contributor

I'd suggest unit tests or something similar to ensure correctness of this function. It is involved and there area lot of edge cases that we need to ensure stay covered. It may be correct now, and it is important to keep it that way.

+
+ getInterpolation: function() {
+
+ switch ( this.createInterpolant ) {

This comment has been minimized.

@bhouston

bhouston Oct 9, 2015

Contributor

Why don't we just remember the THREE define of the interpolant that was set? Or alternatively, as the Interpolant what its type is? It could return a THREE define of the interpolator type.

@bhouston

bhouston Oct 9, 2015

Contributor

Why don't we just remember the THREE define of the interpolant that was set? Or alternatively, as the Interpolant what its type is? It could return a THREE define of the interpolator type.

This comment has been minimized.

@bhouston

bhouston Oct 9, 2015

Contributor

Why not just KeyframeTrack.interpolantType, a variable that returns the THREE define.

@bhouston

bhouston Oct 9, 2015

Contributor

Why not just KeyframeTrack.interpolantType, a variable that returns the THREE define.

This comment has been minimized.

@tschw

tschw Oct 9, 2015

Contributor

Because that's the kind of mess that breaks static analysis as used by tools that would allow aggressive JS optimization / compression / minification, e.g. switching on optimizations of the closure compiler in the build. We really should get rid of stuff like this, even if it takes more thinking and typing.

@tschw

tschw Oct 9, 2015

Contributor

Because that's the kind of mess that breaks static analysis as used by tools that would allow aggressive JS optimization / compression / minification, e.g. switching on optimizations of the closure compiler in the build. We really should get rid of stuff like this, even if it takes more thinking and typing.

This comment has been minimized.

@bhouston

bhouston Oct 9, 2015

Contributor

You mean recomputing this value via a switch statement is faster and also less code than returning a stored variable? I think we are misunderstanding each other.

@bhouston

bhouston Oct 9, 2015

Contributor

You mean recomputing this value via a switch statement is faster and also less code than returning a stored variable? I think we are misunderstanding each other.

This comment has been minimized.

@tschw

tschw Oct 10, 2015

Contributor

No.

I completely misread you here. Now I get what "THREE define" means... I was thinking of the THREE[ variable ] stuff, but it's not what you meant...

Why don't we just remember the THREE define of the interpolant that was set?

Could do. It would be redundant information and it's not needed in time critical code. There are many tracks. That's why I wrote it the way I did.

Or alternatively, as the Interpolant what its type is?

No interpolant to ask. Just a factory method that holds it... Could put the constants on these methods, and I guess it's more elegant, however, it would be a little odd to get the constants on the methods...

You mean recomputing this value via a switch statement is faster and also less code than returning a stored variable? I think we are misunderstanding each other.

No. It's slower and therefore not redundant. Used by .toJSON (a new feature I forgot to mention in the PR description BTW).

@tschw

tschw Oct 10, 2015

Contributor

No.

I completely misread you here. Now I get what "THREE define" means... I was thinking of the THREE[ variable ] stuff, but it's not what you meant...

Why don't we just remember the THREE define of the interpolant that was set?

Could do. It would be redundant information and it's not needed in time critical code. There are many tracks. That's why I wrote it the way I did.

Or alternatively, as the Interpolant what its type is?

No interpolant to ask. Just a factory method that holds it... Could put the constants on these methods, and I guess it's more elegant, however, it would be a little odd to get the constants on the methods...

You mean recomputing this value via a switch statement is faster and also less code than returning a stored variable? I think we are misunderstanding each other.

No. It's slower and therefore not redundant. Used by .toJSON (a new feature I forgot to mention in the PR description BTW).

src/animation/AnimationMixer.js
- for ( var i = 0; i < this.actions.length; i ++ ) {
+ var time = this.time += mixerDeltaTime;
+ var accuIndex = this.accuIndex ^= 1;

This comment has been minimized.

@bhouston

bhouston Oct 9, 2015

Contributor

I'm not understanding why it is necessary to have accuIndex? Can you explain what I am missing?

@bhouston

bhouston Oct 9, 2015

Contributor

I'm not understanding why it is necessary to have accuIndex? Can you explain what I am missing?

This comment has been minimized.

@tschw

tschw Oct 9, 2015

Contributor

The concept is called "double buffering": Prominently known from graphics, actually: You display one screenful of video memory while rendering to the other. When done, the framebuffer base address is switched and the rendered frame is displayed while rendering proceeds on the previously displayed memory. This way, no copy is needed (that's also why the GL ES specification warns about asking low-end hardware to keep the framebuffer contents).

Same thing happens here: By using the accumulation regions in an interleaved fashion, I can compare the "frames" without having to copy the value of the previous frame (which can then take an early exit for vectored data).

There's a shorter explanation in the comment about the buffer layout in the constructor.

@tschw

tschw Oct 9, 2015

Contributor

The concept is called "double buffering": Prominently known from graphics, actually: You display one screenful of video memory while rendering to the other. When done, the framebuffer base address is switched and the rendered frame is displayed while rendering proceeds on the previously displayed memory. This way, no copy is needed (that's also why the GL ES specification warns about asking low-end hardware to keep the framebuffer contents).

Same thing happens here: By using the accumulation regions in an interleaved fashion, I can compare the "frames" without having to copy the value of the previous frame (which can then take an early exit for vectored data).

There's a shorter explanation in the comment about the buffer layout in the constructor.

@tschw

This comment has been minimized.

Show comment
Hide comment
@tschw

tschw Oct 9, 2015

Contributor

@bhouston

if I am reading your code correctly, the Interpolatant is part of the KeyframeTrack,

KeyframeTrack only has factory methods to create as many interpolants as you like. The instances returned from the factory method are then owned by Action, as it's supposed to be.

Contributor

tschw commented Oct 9, 2015

@bhouston

if I am reading your code correctly, the Interpolatant is part of the KeyframeTrack,

KeyframeTrack only has factory methods to create as many interpolants as you like. The instances returned from the factory method are then owned by Action, as it's supposed to be.

src/animation/AnimationMixer.js
- action.propertyBindings[ j ].accumulate( actionResults[j], weight );
+ interpolants[ j ].getAt( actionTime );

This comment has been minimized.

@bhouston

bhouston Oct 9, 2015

Contributor

Is the value being passed from interpolants to propertyMixer via shared state? Where is that setup?

@bhouston

bhouston Oct 9, 2015

Contributor

Is the value being passed from interpolants to propertyMixer via shared state? Where is that setup?

This comment has been minimized.

@tschw

tschw Oct 9, 2015

Contributor

The constructor of Action calls track.createInterpolant( null ), where null is the .result that is attached later in mixer.addAction. The track type decides which exact interpolant to use.

The interpolant's .result is assigned to propertyMixer.buffer. Was missing a reset to null in mixer.removeAction (for historic reasons, I used to cache stuff on the actions - but got rid of it), now fixed.

@tschw

tschw Oct 9, 2015

Contributor

The constructor of Action calls track.createInterpolant( null ), where null is the .result that is attached later in mixer.addAction. The track type decides which exact interpolant to use.

The interpolant's .result is assigned to propertyMixer.buffer. Was missing a reset to null in mixer.removeAction (for historic reasons, I used to cache stuff on the actions - but got rid of it), now fixed.

src/animation/AnimationMixer.js
+ },
+
+ _Up: Float64Array.of( 0, 1 ),
+ _Down: Float64Array.of( 1, 0 ),

This comment has been minimized.

@bhouston

bhouston Oct 9, 2015

Contributor

I'd maybe add the term "Keys" or "Values" to these so it is a little more obvious what they are. Also Float32 should be sufficient for everything dealing with animation.

@bhouston

bhouston Oct 9, 2015

Contributor

I'd maybe add the term "Keys" or "Values" to these so it is a little more obvious what they are. Also Float32 should be sufficient for everything dealing with animation.

src/animation/KeyframeTrack.js
@@ -26,61 +35,124 @@ THREE.KeyframeTrack.prototype = {
constructor: THREE.KeyframeTrack,
- getAt: function( time ) {
+ TimeBufferType: Float64Array,
+ ValueBufferType: Float64Array,

This comment has been minimized.

@bhouston

bhouston Oct 9, 2015

Contributor

Float32 should be sufficient for all animation stuff. Will make it even speedier and save memory.

@bhouston

bhouston Oct 9, 2015

Contributor

Float32 should be sufficient for all animation stuff. Will make it even speedier and save memory.

src/animation/PropertyMixer.js
+ this.binding = new THREE.PropertyBinding( rootNode, path );
+ this.valueSize = valueSize;
+
+ var bufferType = Float64Array,

This comment has been minimized.

@bhouston

bhouston Oct 9, 2015

Contributor

Recommend Float32.

@bhouston

bhouston Oct 9, 2015

Contributor

Recommend Float32.

src/animation/AnimationMixer.js
@@ -141,12 +262,10 @@ THREE.AnimationMixer.prototype = {
fadeOut: function( action, duration ) {
- var keys = [];
+ var time = this.time,
+ times = Float64Array.of( time, time + duration );

This comment has been minimized.

@bhouston

bhouston Oct 9, 2015

Contributor

Float32 diddo.

@bhouston

bhouston Oct 9, 2015

Contributor

Float32 diddo.

src/animation/AnimationMixer.js
- var keys = [];
+ var time = this.time,
+ times = Float64Array.of( time, time + duration ),
+ values = Float64Array.of( startTimeScale, endTimeScale );

This comment has been minimized.

@bhouston

bhouston Oct 9, 2015

Contributor

Float32 diddo.

@bhouston

bhouston Oct 9, 2015

Contributor

Float32 diddo.

src/animation/AnimationMixer.js
- keys.push( { time: this.time, value: 0 } );
- keys.push( { time: this.time + duration, value: 1 } );
+ var time = this.time,
+ times = Float64Array.of( time, time + duration );

This comment has been minimized.

@bhouston

bhouston Oct 9, 2015

Contributor

Float32 diddo.

@bhouston

bhouston Oct 9, 2015

Contributor

Float32 diddo.

@bhouston

This comment has been minimized.

Show comment
Hide comment
@bhouston

bhouston Oct 9, 2015

Contributor

@tschw I guess that works, but having to allocate new interpolants for each mixer, just to cache a single integer seems excessive. I'd replace the per-Mixer interpolants with a single float array for the times and maybe another array for their results and figure out a way to share the Interpolator classes to reduce memory usage -- either per KeyframeTrack or even share single instances of the Interpolator classes.

Contributor

bhouston commented Oct 9, 2015

@tschw I guess that works, but having to allocate new interpolants for each mixer, just to cache a single integer seems excessive. I'd replace the per-Mixer interpolants with a single float array for the times and maybe another array for their results and figure out a way to share the Interpolator classes to reduce memory usage -- either per KeyframeTrack or even share single instances of the Interpolator classes.

@tschw

This comment has been minimized.

Show comment
Hide comment
@tschw

tschw Oct 22, 2015

Contributor

Damn! Wait, the last commit is missing, still.

Contributor

tschw commented Oct 22, 2015

Damn! Wait, the last commit is missing, still.

@tschw

This comment has been minimized.

Show comment
Hide comment
@tschw

tschw Oct 22, 2015

Contributor

/ping @mrdoob
OK, rebase complete. Can it get merged?

Contributor

tschw commented Oct 22, 2015

/ping @mrdoob
OK, rebase complete. Can it get merged?

@mrdoob

This comment has been minimized.

Show comment
Hide comment
@mrdoob

mrdoob Oct 22, 2015

Owner

Checking...

Owner

mrdoob commented Oct 22, 2015

Checking...

@tschw

This comment has been minimized.

Show comment
Hide comment
@tschw

tschw Oct 24, 2015

Contributor

Anything else missing, here?

Contributor

tschw commented Oct 24, 2015

Anything else missing, here?

@mrdoob

This comment has been minimized.

Show comment
Hide comment
@mrdoob

mrdoob Oct 24, 2015

Owner

I'm blocking Monday for reviewing this. I think I'll have API questions/suggestions.

Owner

mrdoob commented Oct 24, 2015

I'm blocking Monday for reviewing this. I think I'll have API questions/suggestions.

@tschw

This comment has been minimized.

Show comment
Hide comment
@tschw

tschw Oct 24, 2015

Contributor

OK, I'm fastening my seatbelt already. Can't we just merge it and refine / simplify the API from there? I have to deploy dependent code and I don't mind subsequent changes, but I do mind the delay - it's already been more of an odyssey than I had planned for...

Contributor

tschw commented Oct 24, 2015

OK, I'm fastening my seatbelt already. Can't we just merge it and refine / simplify the API from there? I have to deploy dependent code and I don't mind subsequent changes, but I do mind the delay - it's already been more of an odyssey than I had planned for...

@mrdoob

This comment has been minimized.

Show comment
Hide comment
@mrdoob

mrdoob Oct 25, 2015

Owner

Haha. Fair point!

Owner

mrdoob commented Oct 25, 2015

Haha. Fair point!

mrdoob added a commit that referenced this pull request Oct 25, 2015

Merge pull request #7312 from tschw/Animation
Animation: Interpolants & extensibility overhaul.

@mrdoob mrdoob merged commit 2142b89 into mrdoob:dev Oct 25, 2015

@mrdoob

This comment has been minimized.

Show comment
Hide comment
@mrdoob

mrdoob Oct 25, 2015

Owner

Thaaanks!

Owner

mrdoob commented Oct 25, 2015

Thaaanks!

@mrdoob

This comment has been minimized.

Show comment
Hide comment
@mrdoob

mrdoob Oct 25, 2015

Owner

Can we make webgl_animation_skinning_morph animate once it loads as it used to?

Owner

mrdoob commented Oct 25, 2015

Can we make webgl_animation_skinning_morph animate once it loads as it used to?

@tschw

This comment has been minimized.

Show comment
Hide comment
@tschw

tschw Oct 25, 2015

Contributor

@mrdoob

Can we make webgl_animation_skinning_morph animate once it loads as it used to?

See #7443

Contributor

tschw commented Oct 25, 2015

@mrdoob

Can we make webgl_animation_skinning_morph animate once it loads as it used to?

See #7443

@anvaka

This comment has been minimized.

Show comment
Hide comment
@anvaka

anvaka Dec 25, 2015

Contributor

GitHub doesn't show changes for AnimationMixer.js but it looks this file got a new variable: THREE.LoopOnceClamp, however it is not defined anywhere else.

Currently it will be equal to undefined, and it doesn't seem right. What was the intended behavior?

Contributor

anvaka commented on c32f31a Dec 25, 2015

GitHub doesn't show changes for AnimationMixer.js but it looks this file got a new variable: THREE.LoopOnceClamp, however it is not defined anywhere else.

Currently it will be equal to undefined, and it doesn't seem right. What was the intended behavior?

This comment has been minimized.

Show comment
Hide comment
@mrdoob

mrdoob Dec 25, 2015

Owner

/ping @tschw

Owner

mrdoob replied Dec 25, 2015

/ping @tschw

This comment has been minimized.

Show comment
Hide comment
@tschw

tschw Jan 1, 2016

Contributor

Leftover from changing things back and forth. Currently the clamp functionality is implemented by the Action's boolean property .clamp. THREE.LoopOnceClamp does not exist.

Contributor

tschw replied Jan 1, 2016

Leftover from changing things back and forth. Currently the clamp functionality is implemented by the Action's boolean property .clamp. THREE.LoopOnceClamp does not exist.

@anvaka

This comment has been minimized.

Show comment
Hide comment
@anvaka

anvaka Dec 25, 2015

Contributor

There is no keys in the function argument list. What should this be?

There is no keys in the function argument list. What should this be?

This comment has been minimized.

Show comment
Hide comment
@mrdoob

mrdoob Dec 25, 2015

Owner

/ping @tschw

Owner

mrdoob replied Dec 25, 2015

/ping @tschw

This comment has been minimized.

Show comment
Hide comment
@tschw

tschw Apr 12, 2016

Contributor

AOS -> SOA conversion, IOW keys were split into two arrays, one with the times and one with the values.

Contributor

tschw replied Apr 12, 2016

AOS -> SOA conversion, IOW keys were split into two arrays, one with the times and one with the values.

@anvaka

This comment has been minimized.

Show comment
Hide comment
@anvaka

anvaka Dec 25, 2015

Contributor

GitHub doesn't show changes for AnimationMixer.js but it looks this file got a new variable: THREE.LoopOnceClamp, however it is not defined anywhere else.

Currently it will be equal to undefined, and it doesn't seem right. What was the intended behavior?

Contributor

anvaka commented on c32f31a Dec 25, 2015

GitHub doesn't show changes for AnimationMixer.js but it looks this file got a new variable: THREE.LoopOnceClamp, however it is not defined anywhere else.

Currently it will be equal to undefined, and it doesn't seem right. What was the intended behavior?

This comment has been minimized.

Show comment
Hide comment
@mrdoob

mrdoob Dec 25, 2015

Owner

/ping @tschw

Owner

mrdoob replied Dec 25, 2015

/ping @tschw

This comment has been minimized.

Show comment
Hide comment
@tschw

tschw Jan 1, 2016

Contributor

Leftover from changing things back and forth. Currently the clamp functionality is implemented by the Action's boolean property .clamp. THREE.LoopOnceClamp does not exist.

Contributor

tschw replied Jan 1, 2016

Leftover from changing things back and forth. Currently the clamp functionality is implemented by the Action's boolean property .clamp. THREE.LoopOnceClamp does not exist.

@AndrewRayCode

This comment has been minimized.

Show comment
Hide comment
@AndrewRayCode

AndrewRayCode Apr 12, 2016

Contributor

I'm late to the party, but this new API is overly constricting to one specific use case where the user doesn't manage elapsed time. The only methods exposed to update animations take a delta, not an absolute time or an absolute percent. That means if I want to manually step through an animation I can't do it easily with this API. I can only tell this how much time has elapsed. If I want to jump to an animation that's 25% in, for example, this API doesn't allow that.

Contributor

AndrewRayCode commented Apr 12, 2016

I'm late to the party, but this new API is overly constricting to one specific use case where the user doesn't manage elapsed time. The only methods exposed to update animations take a delta, not an absolute time or an absolute percent. That means if I want to manually step through an animation I can't do it easily with this API. I can only tell this how much time has elapsed. If I want to jump to an animation that's 25% in, for example, this API doesn't allow that.

@tschw

This comment has been minimized.

Show comment
Hide comment
@tschw

tschw Apr 12, 2016

Contributor

Ooops. Damn! How could I overlook these many pings.

Contributor

tschw commented Apr 12, 2016

Ooops. Damn! How could I overlook these many pings.

@tschw

This comment has been minimized.

Show comment
Hide comment
@tschw

tschw Apr 12, 2016

Contributor

@delvarworld
The API was introduced in r73, this PR just made some changes to it.

And AnimationMixer exposes absolute .time, so I don't really get what you are talking about.

Contributor

tschw commented Apr 12, 2016

@delvarworld
The API was introduced in r73, this PR just made some changes to it.

And AnimationMixer exposes absolute .time, so I don't really get what you are talking about.

@bhouston

This comment has been minimized.

Show comment
Hide comment
@bhouston

bhouston Apr 12, 2016

Contributor

@delvarworld can you suggest the API changes you need more specifically? Maybe as an issue. I would suggest that you even write out the functions or parameters you would like added, especially if you could reference Unity3D or UE4's API docs for comparison.

Contributor

bhouston commented Apr 12, 2016

@delvarworld can you suggest the API changes you need more specifically? Maybe as an issue. I would suggest that you even write out the functions or parameters you would like added, especially if you could reference Unity3D or UE4's API docs for comparison.

@tschw

This comment has been minimized.

Show comment
Hide comment
@tschw

tschw Apr 12, 2016

Contributor

@delvarworld

That means if I want to manually step through an animation I can't do it easily with this API.

Depends on how easy is easy enough for you. Here is what changes to public .time do:

mixer.time = value; // sets the global time, without changing the local time of the actions
action.time = value; // sets the local time of a specific action
mixer.update( - mixer.time + value ); // sets the time of mixer and all running actions
mixer.update( 0 ); // just updates 

Action start, mixing / warping are global. Playback is local.

Contributor

tschw commented Apr 12, 2016

@delvarworld

That means if I want to manually step through an animation I can't do it easily with this API.

Depends on how easy is easy enough for you. Here is what changes to public .time do:

mixer.time = value; // sets the global time, without changing the local time of the actions
action.time = value; // sets the local time of a specific action
mixer.update( - mixer.time + value ); // sets the time of mixer and all running actions
mixer.update( 0 ); // just updates 

Action start, mixing / warping are global. Playback is local.

@AndrewRayCode

This comment has been minimized.

Show comment
Hide comment
@AndrewRayCode

AndrewRayCode Apr 12, 2016

Contributor

Sorry to prolong this thread, I'm still very new to animating meshes in Three so I'm not sure if I have enough information to make a new ticket yet.

The fundamental problem with this API is it's imperative. It says "go to this next point in the animation, which entirely depends on internal variables you can't see." What I want is a declarative API, to say "go exactly to this point in the animation."

For example, I have a mesh, with an animation of its hand going from down to up. I want to programmatically set the state of this animation. Maybe I want to tie it directly to the rotation of my character. If the character is rotated right, the hand should be up, if the character is rotated forward, the hand should be down. I already know how far rotated my character is and can easily calculate the percent along that rotation. I should be able to pass that same percent to the animation and have it set that current visible frame.

I've found a workaround for now which is similar to what you showed.

mixer.time = 0;
const action = mixer._actions[ 0 ];
const { duration } = action._clip;
action._loopCount = -1;
action.time = 0;
action.startTime = 0;
mixer.update(
    Math.min( duration * 0.999, duration * percentAlongAnimation )
);

This allows me to manually set percentAlongAnimation to whatever programatic value I choose and regain control. Obviously this isn't ideal because I'm mutating variables inside an API I don't own.

As I move on to animation blending and multiple animations, I imagine this will only get more complicated and difficult to achieve.

Contributor

AndrewRayCode commented Apr 12, 2016

Sorry to prolong this thread, I'm still very new to animating meshes in Three so I'm not sure if I have enough information to make a new ticket yet.

The fundamental problem with this API is it's imperative. It says "go to this next point in the animation, which entirely depends on internal variables you can't see." What I want is a declarative API, to say "go exactly to this point in the animation."

For example, I have a mesh, with an animation of its hand going from down to up. I want to programmatically set the state of this animation. Maybe I want to tie it directly to the rotation of my character. If the character is rotated right, the hand should be up, if the character is rotated forward, the hand should be down. I already know how far rotated my character is and can easily calculate the percent along that rotation. I should be able to pass that same percent to the animation and have it set that current visible frame.

I've found a workaround for now which is similar to what you showed.

mixer.time = 0;
const action = mixer._actions[ 0 ];
const { duration } = action._clip;
action._loopCount = -1;
action.time = 0;
action.startTime = 0;
mixer.update(
    Math.min( duration * 0.999, duration * percentAlongAnimation )
);

This allows me to manually set percentAlongAnimation to whatever programatic value I choose and regain control. Obviously this isn't ideal because I'm mutating variables inside an API I don't own.

As I move on to animation blending and multiple animations, I imagine this will only get more complicated and difficult to achieve.

@bhouston

This comment has been minimized.

Show comment
Hide comment
@bhouston

bhouston Apr 12, 2016

Contributor

@delvarworld would something like a manual mode be useful? Some actions can be set in manual mode and then you have to set their time and weight explicitly?

Contributor

bhouston commented Apr 12, 2016

@delvarworld would something like a manual mode be useful? Some actions can be set in manual mode and then you have to set their time and weight explicitly?

@tschw

This comment has been minimized.

Show comment
Hide comment
@tschw

tschw Apr 13, 2016

Contributor

@delvarworld

For example, I have a mesh, with an animation of its hand going from down to up. I want to programmatically set the state of this animation. Maybe I want to tie it directly to the rotation of my character. If the character is rotated right, the hand should be up, if the character is rotated forward, the hand should be down. I already know how far rotated my character is and can easily calculate the percent along that rotation. I should be able to pass that same percent to the animation and have it set that current visible frame.

Sounds more like setting the time of a particular action would be appropriate in this case.

https://github.com/mrdoob/three.js/blob/master/examples/js/TimelinerController.js#L57-L64

is how I did it in

http://threejs.org/examples/misc_animation_authoring.html

I've found a workaround for now which is similar to what you showed.
This allows me to manually set percentAlongAnimation to whatever programatic value I choose and regain control. Obviously this isn't ideal because I'm mutating variables inside an API I don't own.

Hmm... Honestly, this code does not look healthy or advisable in any way, not even for a workaround. I don't understand why you are having to mess with all these values. Also, why use mixer._actions[ 0 ] where there's clipAction to get the action?

Then there's the question, whether someone really should move the global time around. After all, AnimationMixer does not implement a full high level sequencer for the actions, so there really is no persistent global timeline, just a delayed start.

Contributor

tschw commented Apr 13, 2016

@delvarworld

For example, I have a mesh, with an animation of its hand going from down to up. I want to programmatically set the state of this animation. Maybe I want to tie it directly to the rotation of my character. If the character is rotated right, the hand should be up, if the character is rotated forward, the hand should be down. I already know how far rotated my character is and can easily calculate the percent along that rotation. I should be able to pass that same percent to the animation and have it set that current visible frame.

Sounds more like setting the time of a particular action would be appropriate in this case.

https://github.com/mrdoob/three.js/blob/master/examples/js/TimelinerController.js#L57-L64

is how I did it in

http://threejs.org/examples/misc_animation_authoring.html

I've found a workaround for now which is similar to what you showed.
This allows me to manually set percentAlongAnimation to whatever programatic value I choose and regain control. Obviously this isn't ideal because I'm mutating variables inside an API I don't own.

Hmm... Honestly, this code does not look healthy or advisable in any way, not even for a workaround. I don't understand why you are having to mess with all these values. Also, why use mixer._actions[ 0 ] where there's clipAction to get the action?

Then there's the question, whether someone really should move the global time around. After all, AnimationMixer does not implement a full high level sequencer for the actions, so there really is no persistent global timeline, just a delayed start.

@AndrewRayCode

This comment has been minimized.

Show comment
Hide comment
@AndrewRayCode

AndrewRayCode Apr 13, 2016

Contributor

Hmm... Honestly, this code does not look healthy or advisable in any way,

Your example worked for me with setting the time prop. I don't think either of our solutions are ideal though, they are both brittle and require knowledge of the internals of the class.

whether someone really should move the global time around

Yes, someone should be able to pass in an exact set of variables to a scene, including time, and get an exact, reproducible output. There's a lot that comes with that, but I don't think it would be productive to drag this thread in that direction.

A good solution might be to separate the time handling logic from the actual animation API. That way end users who want their time to be managed by a third party can use the time API. Similar to the distinction between TweenJS controlling time and easing, and my easing-utils library simply taking in a value and returning the eased value, with no concept of time.

re:

would something like a manual mode be useful?

and

Sounds more like setting the time of a particular action would be appropriate in this case.

There are two things I'm looking for in an animation API:

  1. The ability to manually control animation (with an api for percent based control and absolute time based control, for convenience)
  2. Have the same API work for blended animations. I'm currently working towards multiple animations on my character and don't know how the suggested method by @tschw will work with blending(?). As in I'd like to programmatically set the weights of each animation and the percent of each animation and still have blending work behind the scenes.
Contributor

AndrewRayCode commented Apr 13, 2016

Hmm... Honestly, this code does not look healthy or advisable in any way,

Your example worked for me with setting the time prop. I don't think either of our solutions are ideal though, they are both brittle and require knowledge of the internals of the class.

whether someone really should move the global time around

Yes, someone should be able to pass in an exact set of variables to a scene, including time, and get an exact, reproducible output. There's a lot that comes with that, but I don't think it would be productive to drag this thread in that direction.

A good solution might be to separate the time handling logic from the actual animation API. That way end users who want their time to be managed by a third party can use the time API. Similar to the distinction between TweenJS controlling time and easing, and my easing-utils library simply taking in a value and returning the eased value, with no concept of time.

re:

would something like a manual mode be useful?

and

Sounds more like setting the time of a particular action would be appropriate in this case.

There are two things I'm looking for in an animation API:

  1. The ability to manually control animation (with an api for percent based control and absolute time based control, for convenience)
  2. Have the same API work for blended animations. I'm currently working towards multiple animations on my character and don't know how the suggested method by @tschw will work with blending(?). As in I'd like to programmatically set the weights of each animation and the percent of each animation and still have blending work behind the scenes.
@MasterJames

This comment has been minimized.

Show comment
Hide comment
@MasterJames

MasterJames Apr 13, 2016

Contributor

I sometimes resort to using the greensock TweenMax and/or TweenLite for that. It doesn't need start for animation and other interesting lessons learned over the years as it was originally done in flash but now has a js version.
I used it in my animated spotlights example because the internal Tween didn't like setters/getters (no longer being used that way in the example) and then I realized it's simpler (no need to call start) Anyway I guess I'm suggesting to look at it for ideas and maybe as a solution for setting an animation to a point in between by percentage of 'progress'.

Contributor

MasterJames commented Apr 13, 2016

I sometimes resort to using the greensock TweenMax and/or TweenLite for that. It doesn't need start for animation and other interesting lessons learned over the years as it was originally done in flash but now has a js version.
I used it in my animated spotlights example because the internal Tween didn't like setters/getters (no longer being used that way in the example) and then I realized it's simpler (no need to call start) Anyway I guess I'm suggesting to look at it for ideas and maybe as a solution for setting an animation to a point in between by percentage of 'progress'.

@tschw

This comment has been minimized.

Show comment
Hide comment
@tschw

tschw Apr 13, 2016

Contributor

@delvarworld

Yes, someone should be able to pass in an exact set of variables to a scene, including time, and get an exact, reproducible output. There's a lot that comes with that, but I don't think it would be productive to drag this thread in that direction.

A good solution might be to separate the time handling logic from the actual animation API. That way end users who want their time to be managed by a third party can use the time API. Similar to the distinction between TweenJS controlling time and easing, and my easing-utils library simply taking in a value and returning the eased value, with no concept of time.

No, the tweening stuff wouldn't help, because there isn't really a problem on that end.

The animation system was designed to be a "live" system. I pushed the envelope a bit by adding more scheduling functionality. However, and this is the important part, there is currently no sequencer with a a global timeline data structure implemented in Three.js. AnimationMixer only goes as far as it goes, and pretending it was a sequencer would currently be misuse. In particular, an action added to the mixer takes up resources, so it can quickly be started and stopped, and it currently wouldn't scale well to building timelines.

The question that arises is, "can we push the envelope a little further to make AnimationMixer a sequencer?" I think this might be possible, but I'd have to implement some more resource reuse to make it advisable.

Although I have it on my list, it's currently not on the top of it in terms of priorities, so unless we can get some funding for the PR, it would need to wait until I need it / get around to it.

Your example worked for me with setting the time prop. I don't think either of our solutions are ideal though, they are both brittle and require knowledge of the internals of the class.

I think you are misreading my code. The code I referred to only changes internal properties on this :).

Contributor

tschw commented Apr 13, 2016

@delvarworld

Yes, someone should be able to pass in an exact set of variables to a scene, including time, and get an exact, reproducible output. There's a lot that comes with that, but I don't think it would be productive to drag this thread in that direction.

A good solution might be to separate the time handling logic from the actual animation API. That way end users who want their time to be managed by a third party can use the time API. Similar to the distinction between TweenJS controlling time and easing, and my easing-utils library simply taking in a value and returning the eased value, with no concept of time.

No, the tweening stuff wouldn't help, because there isn't really a problem on that end.

The animation system was designed to be a "live" system. I pushed the envelope a bit by adding more scheduling functionality. However, and this is the important part, there is currently no sequencer with a a global timeline data structure implemented in Three.js. AnimationMixer only goes as far as it goes, and pretending it was a sequencer would currently be misuse. In particular, an action added to the mixer takes up resources, so it can quickly be started and stopped, and it currently wouldn't scale well to building timelines.

The question that arises is, "can we push the envelope a little further to make AnimationMixer a sequencer?" I think this might be possible, but I'd have to implement some more resource reuse to make it advisable.

Although I have it on my list, it's currently not on the top of it in terms of priorities, so unless we can get some funding for the PR, it would need to wait until I need it / get around to it.

Your example worked for me with setting the time prop. I don't think either of our solutions are ideal though, they are both brittle and require knowledge of the internals of the class.

I think you are misreading my code. The code I referred to only changes internal properties on this :).

@tschw tschw referenced this pull request Apr 16, 2016

Closed

Non Linear Animation #7913

@AndrewRayCode

This comment has been minimized.

Show comment
Hide comment
@AndrewRayCode

AndrewRayCode Apr 20, 2016

Contributor

I'm still struggling with this, any advice on the correct way to use the API would be appreciated.

My mesh has two animations of different lengths (different number of frames). I need to control the time of each animation independently. In my case it's a walk cycle which always loops (setting its own time independently) and a jump animation (setting its own time independently). I'm controlling the blend between the two using weights.

I've tried doing basically this for each action. I've checked that for each animation, weight is set correctly, and percent is a value from 0 to 1.

action.setEffectiveWeight( weight );
const { duration } = action._clip;
action.time = duration * percent;

and then

mixer.update( 0 );

However my animations don't seem to reach completion - visually it appears as if each animation is still receiving influence from the others, even though I set weights explicitly to 0 (walking) and 1 (jumping). For example, my jump animation in Blender ends with the legs straight up and down, but in my running code, the animation ends with the legs about 80% to that goal. It might not be a weight issue, that's just a hunch.

I've tried debugging the internals of AnimationMixer but it's hard to follow because many functions, like setEffectiveWeight(), have deeply nested side effects.

Contributor

AndrewRayCode commented Apr 20, 2016

I'm still struggling with this, any advice on the correct way to use the API would be appreciated.

My mesh has two animations of different lengths (different number of frames). I need to control the time of each animation independently. In my case it's a walk cycle which always loops (setting its own time independently) and a jump animation (setting its own time independently). I'm controlling the blend between the two using weights.

I've tried doing basically this for each action. I've checked that for each animation, weight is set correctly, and percent is a value from 0 to 1.

action.setEffectiveWeight( weight );
const { duration } = action._clip;
action.time = duration * percent;

and then

mixer.update( 0 );

However my animations don't seem to reach completion - visually it appears as if each animation is still receiving influence from the others, even though I set weights explicitly to 0 (walking) and 1 (jumping). For example, my jump animation in Blender ends with the legs straight up and down, but in my running code, the animation ends with the legs about 80% to that goal. It might not be a weight issue, that's just a hunch.

I've tried debugging the internals of AnimationMixer but it's hard to follow because many functions, like setEffectiveWeight(), have deeply nested side effects.

@tschw

This comment has been minimized.

Show comment
Hide comment
@tschw

tschw Apr 20, 2016

Contributor

@delvarworld

const { duration } = action._clip;

The loader probably sets a time scale and you're ignoring it.

It might not be a weight issue, that's just a hunch.

Please file a bug report with a simple test that reproduces the "issue". Thanks.

Contributor

tschw commented Apr 20, 2016

@delvarworld

const { duration } = action._clip;

The loader probably sets a time scale and you're ignoring it.

It might not be a weight issue, that's just a hunch.

Please file a bug report with a simple test that reproduces the "issue". Thanks.

@AndrewRayCode

This comment has been minimized.

Show comment
Hide comment
@AndrewRayCode

AndrewRayCode Apr 20, 2016

Contributor

Woohoo! Finally found my final problem, which as I suspected, was specific to me. I'm using a custom shader where I computed the gl_Position wrong in the end. Thanks for helping me check my assumptions.

To come full circle, here's the API that I can now use: https://gist.github.com/DelvarWorld/586e9fb3102a39333f130836c416c87b
Under the hood I can do this by calling action.setEffectiveWeight, action.time = ..., and then mixer.update( 0 );. @bhouston While this still requires specific knowledge of the class internals (update( 0 ) especially, since that parameter is supposed to be a delta), I think it's simple enough for me that I don't think I need any API changes made.

Contributor

AndrewRayCode commented Apr 20, 2016

Woohoo! Finally found my final problem, which as I suspected, was specific to me. I'm using a custom shader where I computed the gl_Position wrong in the end. Thanks for helping me check my assumptions.

To come full circle, here's the API that I can now use: https://gist.github.com/DelvarWorld/586e9fb3102a39333f130836c416c87b
Under the hood I can do this by calling action.setEffectiveWeight, action.time = ..., and then mixer.update( 0 );. @bhouston While this still requires specific knowledge of the class internals (update( 0 ) especially, since that parameter is supposed to be a delta), I think it's simple enough for me that I don't think I need any API changes made.

@AndrewRayCode

This comment has been minimized.

Show comment
Hide comment
@AndrewRayCode

AndrewRayCode Apr 23, 2016

Contributor

@tschw me again. I'm trying to do this same process with morph targets. I'm trying to control them individually by hand. It doesn't seem quite as straightforward as the rigging blending.

1. I'm trying to rig an eyelid so I can close the upper and lower lid independently, using morph targets. With rigging, as shown above, I can set the weight of each rig animation to blend between them. I thought I could work with the individual upper and lower morph target animations somehow like

clipUpper = THREE.AnimationClip.CreateFromMorphTargetSequence( 'upper', [
        geometry.morphTargets[ 0 ], // open
        geometry.morphTargets[ 1 ], // upper close
], 3 );
actionUpper = mixer.clipAction( clipUpper ).play();

clipLower = THREE.AnimationClip.CreateFromMorphTargetSequence( 'lower', [
        geometry.morphTargets[ 0 ], // open
        geometry.morphTargets[ 3 ], // lower close
], 3 );
actionLower = mixer.clipAction( clipLower ).play();

...

clipUpper.time = ...;
clipLower.time = ...;
mixer.update( 0 );

However in this case only one of the morph animations ever seems to play, which is the first one created. Is it possible to control multiple morph targets independently with the AnimationMixer class? This is necessary for things like programatically setting character expressions.

2. I'm trying to tween the eyelid closing using two morph targets so that it properly tweens around the eye. I have "open", "half closed" and "full closed" targets. However, you can see when I tween between them using

clip = THREE.AnimationClip.CreateFromMorphTargetSequence( 'blink', geometry.morphTargets, 3 );
action = mixer.clipAction( clip ).play();

You can see the problem that occurs in this gif. When closing, the lid successfully tweens through half closed to full closed, which is required so the lid doesn't intersect the eye. However, as it opens again, it doesn't go back through the half way target, so there's an intersection:

blink-no

I tried manually inserting the half close morph target into the sequence again:

clipUpper = THREE.AnimationClip.CreateFromMorphTargetSequence( 'upper', [
        geometry.morphTargets[ 0 ], // open
        geometry.morphTargets[ 1 ], // half close
        geometry.morphTargets[ 2 ], // upper close
        geometry.morphTargets[ 3 ], // half close
], 3 );

However this destroys the animation and the vertices are mangled. Do you have a suggestion for how to reuse morph targets in the same animation without having to duplicate them in Blender? These two questions are related, I tried to go the first route so I could manually set the tween for each morph target, but when I couldn't figure that out, I tried to set a specific order of targets. That didn't work either.

Contributor

AndrewRayCode commented Apr 23, 2016

@tschw me again. I'm trying to do this same process with morph targets. I'm trying to control them individually by hand. It doesn't seem quite as straightforward as the rigging blending.

1. I'm trying to rig an eyelid so I can close the upper and lower lid independently, using morph targets. With rigging, as shown above, I can set the weight of each rig animation to blend between them. I thought I could work with the individual upper and lower morph target animations somehow like

clipUpper = THREE.AnimationClip.CreateFromMorphTargetSequence( 'upper', [
        geometry.morphTargets[ 0 ], // open
        geometry.morphTargets[ 1 ], // upper close
], 3 );
actionUpper = mixer.clipAction( clipUpper ).play();

clipLower = THREE.AnimationClip.CreateFromMorphTargetSequence( 'lower', [
        geometry.morphTargets[ 0 ], // open
        geometry.morphTargets[ 3 ], // lower close
], 3 );
actionLower = mixer.clipAction( clipLower ).play();

...

clipUpper.time = ...;
clipLower.time = ...;
mixer.update( 0 );

However in this case only one of the morph animations ever seems to play, which is the first one created. Is it possible to control multiple morph targets independently with the AnimationMixer class? This is necessary for things like programatically setting character expressions.

2. I'm trying to tween the eyelid closing using two morph targets so that it properly tweens around the eye. I have "open", "half closed" and "full closed" targets. However, you can see when I tween between them using

clip = THREE.AnimationClip.CreateFromMorphTargetSequence( 'blink', geometry.morphTargets, 3 );
action = mixer.clipAction( clip ).play();

You can see the problem that occurs in this gif. When closing, the lid successfully tweens through half closed to full closed, which is required so the lid doesn't intersect the eye. However, as it opens again, it doesn't go back through the half way target, so there's an intersection:

blink-no

I tried manually inserting the half close morph target into the sequence again:

clipUpper = THREE.AnimationClip.CreateFromMorphTargetSequence( 'upper', [
        geometry.morphTargets[ 0 ], // open
        geometry.morphTargets[ 1 ], // half close
        geometry.morphTargets[ 2 ], // upper close
        geometry.morphTargets[ 3 ], // half close
], 3 );

However this destroys the animation and the vertices are mangled. Do you have a suggestion for how to reuse morph targets in the same animation without having to duplicate them in Blender? These two questions are related, I tried to go the first route so I could manually set the tween for each morph target, but when I couldn't figure that out, I tried to set a specific order of targets. That didn't work either.

@tschw

This comment has been minimized.

Show comment
Hide comment
@tschw

tschw Apr 23, 2016

Contributor

@delvarworld
First of, your issues has nothing to do with AnimationMixer or this PR. Therefore you should open a new ticket. There's no magic in what animation mixer does: It interpolates the keyframes in all active actions' clips and applies a weighted state of the results to object properties. At that level, there's no difference between morph targets or bones.

Now, to answer both of your questions:

  1. When you want to do a circular motion with morph targets, you need many of them. So you were going into the right direction, just not far enough.
  2. When you want to apply two different morph target animations to the same object at the same time, these are mixed together. You have to make sure they do or do not overlap in terms of the tracks, depending on whether that is the desired effect or not. Using a common base state for independent morph targets that should run at the same time can't work.

HTH

Contributor

tschw commented Apr 23, 2016

@delvarworld
First of, your issues has nothing to do with AnimationMixer or this PR. Therefore you should open a new ticket. There's no magic in what animation mixer does: It interpolates the keyframes in all active actions' clips and applies a weighted state of the results to object properties. At that level, there's no difference between morph targets or bones.

Now, to answer both of your questions:

  1. When you want to do a circular motion with morph targets, you need many of them. So you were going into the right direction, just not far enough.
  2. When you want to apply two different morph target animations to the same object at the same time, these are mixed together. You have to make sure they do or do not overlap in terms of the tracks, depending on whether that is the desired effect or not. Using a common base state for independent morph targets that should run at the same time can't work.

HTH

@bhouston

This comment has been minimized.

Show comment
Hide comment
@bhouston

bhouston Apr 23, 2016

Contributor
  1. When you want to apply two different morph target animations to the same object at the same time, these are mixed together. You have to make sure they do or do not overlap in terms of the tracks, depending on whether that is the desired effect or not. Using a common base state for independent morph targets that should run at the same time can't work.

If Three.JS were to use deltas (e.g.: morph - base)_morphWeight + base) when applying for morph targets rather than blending to the exact set of morph vertices (morph1_morphWeight 1 + morph2*morphWeight2, etc.. ), you can apply multiple at the same time even if they are overlapping. Although the way that Three.JS uses morphs to do full animations like walking may conflict with the idea of using morph targets for layering facial animation.

This is how we implemented our JavaScript-based moprh target code Clara.io's with great effect and it was necessary in order to replicate Maya, 3DS Max, Blender morph target behavior.

Contributor

bhouston commented Apr 23, 2016

  1. When you want to apply two different morph target animations to the same object at the same time, these are mixed together. You have to make sure they do or do not overlap in terms of the tracks, depending on whether that is the desired effect or not. Using a common base state for independent morph targets that should run at the same time can't work.

If Three.JS were to use deltas (e.g.: morph - base)_morphWeight + base) when applying for morph targets rather than blending to the exact set of morph vertices (morph1_morphWeight 1 + morph2*morphWeight2, etc.. ), you can apply multiple at the same time even if they are overlapping. Although the way that Three.JS uses morphs to do full animations like walking may conflict with the idea of using morph targets for layering facial animation.

This is how we implemented our JavaScript-based moprh target code Clara.io's with great effect and it was necessary in order to replicate Maya, 3DS Max, Blender morph target behavior.

@tschw

This comment has been minimized.

Show comment
Hide comment
@tschw

tschw Apr 23, 2016

Contributor

@bhouston
Where/how to best implement it?

I'm wondering whether doing it right would break legacy content and need to become switchable or could just work.

There's still the state += (1 - sumOfWeights) * stateOnAttach logic in the mixer, so requiring stateOnAtach == morphBaseState might be an option for a shortcut, but maybe to nowhere really worth going...

Can it be done generally - also for bones (see #7913)?

Contributor

tschw commented Apr 23, 2016

@bhouston
Where/how to best implement it?

I'm wondering whether doing it right would break legacy content and need to become switchable or could just work.

There's still the state += (1 - sumOfWeights) * stateOnAttach logic in the mixer, so requiring stateOnAtach == morphBaseState might be an option for a shortcut, but maybe to nowhere really worth going...

Can it be done generally - also for bones (see #7913)?

@AndrewRayCode

This comment has been minimized.

Show comment
Hide comment
@AndrewRayCode

AndrewRayCode Apr 23, 2016

Contributor

Oh, duh! All I wanted was mesh.morphTargetInfluences. That's the only thing I need to use. I can declare morph target percents individually using that, and they automatically blend fine and multiple can run at once.

Contributor

AndrewRayCode commented Apr 23, 2016

Oh, duh! All I wanted was mesh.morphTargetInfluences. That's the only thing I need to use. I can declare morph target percents individually using that, and they automatically blend fine and multiple can run at once.

@AndrewRayCode

This comment has been minimized.

Show comment
Hide comment
@AndrewRayCode

AndrewRayCode Apr 24, 2016

Contributor

Charisma lives :)

welcome-charisma
https://twitter.com/andrewray/status/724081650242445312

Thanks for your help everyone!

Contributor

AndrewRayCode commented Apr 24, 2016

Charisma lives :)

welcome-charisma
https://twitter.com/andrewray/status/724081650242445312

Thanks for your help everyone!

@mrdoob

This comment has been minimized.

Show comment
Hide comment
@mrdoob

mrdoob Apr 24, 2016

Owner

Cuteness!

Owner

mrdoob commented Apr 24, 2016

Cuteness!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment