glMatrix v4.0 - Request for feedback

[toji]

glMatrix was a project I started 12 years ago(!), because at the time there weren't too many good options for doing the type of matrix math realtime 3D apps required in JavaScript. I never thought that anyone outside of myself and a few early WebGL developers would use it, and certainly didn't anticipate it becoming as popular as it did.

Fortunately for everyone, the landscape for 3D on the web looks a lot different today than it did over a decade ago! There's a lot of great libraries that offer comprehensive tools for creating realtime 3D web apps, usually with their own very competent vector and matrix capabilities built in. Many of these offer features or syntax niceties that glMatrix hasn't been able to match due to it's history and design ethos.

The web itself has also evolved in that time. When I published the first version of glMatrix Chrome was on version 5, Firefox was at version 3, and Internet Explorer was still a thing developers cared about. Node.js and TypeScript hadn't even been released! We've made astronomical strides in terms of the capabilities of the Web platform since then. For example, none of the following existed (or at the very least were widespread) at the time glMatrix was first developed:

• let and const
• Arrow functions ((x) => { return x; })
• JavaScript classes
  • which includes Getters and Setters
• JavaScript Modules
• Template literals
• Spread syntax (someFunction(...args);)
• Even Float32Array wasn't around until shortly after!
• Oh, and WebGL itself was still in development

Over the years glMatrix has been updated in small ways to take advantage of some of these things, it hasn't strayed too far from it's original design. But these days we can do so much better, and despite the excellent competition that I strongly believe there's still a need for a solid, standalone vector and matrix math library.

I've had a bunch of ideas floating around in my head for how glMatrix could be updated to take advantage of the modern web platform, but haven't had an opportunity to work on it till recently. (Let's be honest, the library has needed some maintenence for a while now.) Now that I've had a chance to try some of it out, though, I feel pretty confident that it's a viable direction for the future of the API.

So let me walk you through my plans for a glMatrix 4.0! Feedback highly appreciated!

Backwards compatibility first and foremost

glMatrix has a lot of users, and they have a lot of carefully written algorithms using the library. It would be unrealistic to expect them to do complete re-writes of their code base just to take advantage of a nicer code pattern. So the first principle of any update is that backwards compatibility is always priority number one.

This doesn't mean that EVERYTHING is carried forward, mind you. I think it's appropriate for some functionality to be labeled as deprecated and for some lesser used, fairly awkward bit of the library to be dropped. (I'm looking at you, weird forEach experiment that never quite worked the way I wanted.)

But the majority of the library should be able to be used with minimal or no changes to existing code, and new features should cleanly layer on top of that existing code rather than requiring developers to make a wholesale switch from the "old way" to the "new way".

Lots of ease-of-use improvements

glMatrix was designed for efficiency, but that left a lot to be desired in terms of ease-of-use. There's only so much that JavaScript allows in terms of cleaning up the syntax, but with some slightly unorthodox tricks and taking advantage of modern language features we can improve things quite a bit.

Constructors

The current syntax for creating vector and matrix objects isn't ideal (I'll be using vec3 for examples, but everything here applied to each type in the library):

let v1 = vec3.create();
let v2 = vec3.fromValues(1, 2, 3);

We'd much rather use the familiar JavaScript new operator. Turns out we can without losing any backwards compatibility simply by declaring our class to extend Float32Array!

export class Vec3 extends Float32Array {
  constructor(...values) {
    switch(values.length) {
      case 3: super(values); break;
      case 2:
      case 1: super(values[0], value[1] ?? 0, 3); break;
      default: super(3); break;
    }
  }
}

This allows us to use a few variants of a typical constructor.

let v1 = new Vec3(); // Creates a vector with value (0, 0, 0)
let v2 = new Vec3(1, 2, 3); // Creates a vector with value (1, 2, 3)
let arrayBuffer = new ArrayBuffer(32);
let v3 = new Vec3(arrayBuffer); // Creates a vector mapped to offset 0 of arrayBuffer
let v4 = new Vec3(arrayBuffer, 16); // Creates a vector mapped to offset 16 of arrayBuffer
let v5 = new Vec3(v2); // Creates a copy of v2

It's pretty flexible, and not that complex to implement! And of course because Vec3 is still a Float32Array under the hood, you can pass it into WebGL/WebGPU (or any other API that expects Typed arrays or array-like objects) with no conversion.

gl.uniform3fv(lightPosition, v5);

Static methods

For backwards compatibility we'll keep around vec3.create() and friends, but have them return instances of the new Vec3 class instead of raw typed arrays. In order to keep everything together, they'll become static methods on the Vec3 class. Same goes for every other existing method for a given type.

export class Vec3 extends Float32Array {
  static create() { return new Vec3(); }
  static fromValues(x, y, z) { return new Vec3(x, y, z); }

  static add(out, a, b) {
    out[0] = a[0] + b[0];
    out[1] = a[1] + b[1];
    out[2] = a[2] + b[2];
    return out;
  }

  static sub(out, a, b) {
    out[0] = a[0] - b[0];
    out[1] = a[1] - b[1];
    out[2] = a[2] - b[2];
    return out;
  }
}

Used as:

let v1 = new Vec3(1, 2, 3);
let v2 = new Vec3(4, 5, 6);

Vec3.add(v1, v1, v2); // v1 is now Vec3(5, 7, 9);
Vec3.sub(v1, v1, [1, 1, 1]); // v1 is now Vec3(4, 6, 8);

As a minor design aside, I felt pretty strongly that as a class the type names should begin with an uppercase, but that does break backwards compat since in the original library all the "namespaces" were lowercase. This can be resolved by having the library defined a simple alias:

export const vec3 = Vec3;

Which then allows you to import whichever casing you need for your app, and even mix and match.

import { Vec3, vec3 } from './gl-matrix/vec3.js';

// This is all fine.
let v1 = new Vec3(1, 2, 3);
let v2 = new vec3(4, 5, 6);
Vec3.add(v1, v1, v2);
vec3.sub(v1, v1, [1, 1, 1]);

I would probably encourage migration to the uppercase variant over time, though.

Instance methods

Once we have a proper class backing out vectors and matrices, we can make many of the methods for those types instance methods, which operate explicitly on the this object.

export class Vec3 extends Float32Array {
  add(b) {
    this[0] += b[0];
    this[1] += b[1];
    this[2] += b[2];
    return this;
  }

  sub(b) {
    this[0] -= b[0];
    this[1] -= b[1];
    this[2] -= b[2];
    return this;
  }

  // etc...
}

Turns out that this doesn't conflict with the static methods of the same name on the same class! And it makes the syntax for common operations much easier to type and read:

let v1 = new Vec3(1, 2, 3);
let v2 = new Vec3(4, 5, 6);

v1.add(v2).sub([1, 1, 1]); // v1 now equals Vec3(4, 6, 8);

Actually there's two ways of going about this. One is that you implicitly make every operation on a vector apply to the vector itself, as shown above. The other is that you have each operation return a new instance of the vector with the result, leaving the operands unchanged. I feel pretty strongly that the former fits the ethos of glMatrix better by not creating constantly creating new objects unless it's necessary.

If you don't want to alter the values of the original object, there's still reasonably easy options that make it more explicit what you're doing and where new memory is being allocated.

let v3 = new Vec3(v1).add(v2); // v3 is now v1 + v2, v1 and v2 are unchanged.

And, of course you can mix and match with the older function style too, which is still handy for applying the result of two different operands to a third value or simply for migrating code piecemeal over time.

v1.add(v2);
Vec3.sub(v3, v1, [1, 1, 1]);

Attributes

Vectors being a real class means we can also offer a better way to access the components, because lets face it: typing v[0] instead of v.x is really annoying. Getters and setters to the rescue!

export class Vec3 extends Float32Array {
  get x() { return this[0]; }
  set x(value) { this[0] = value; }

  get y() { return this[1]; }
  set y(value) { this[1] = value; }

  get z() { return this[2]; }
  set z(value) { this[2] = value; }
}

Now we can choose to reference components by either index or name:

let v = new Vec3(1, 2, 3);
// Now this...
let len = Math.sqrt((v.x * v.x) + (v.y * v.y) + (v.z * v.z));
// Is equivalent to this...
let len2 = Math.sqrt((v[0] * v[0]) + (v[1] * v[1]) + (v[2] * v[2]));

All of method implementations internally will continue to lookup components by index both because it's a bit faster and because it allows for raw arrays to be passed in as temporary vectors and matrices, which is convenient.

Swizzles!

And hey, while we're adding accessors, why not borrow one of my favorite bits of shader syntax and add swizzle operators too!

export class Vec3 extends Float32Array {
  get xxx() { return new Vec3(this[0], this[0], this[0]); }
  get xxy() { return new Vec3(this[0], this[0], this[1]); }
  get xxz() { return new Vec3(this[0], this[0], this[2]); }
  get xyx() { return new Vec3(this[0], this[1], this[0]); }
  get xyy() { return new Vec3(this[0], this[1], this[1]); }
  // etc...
  get zzx() { return new Vec3(this[2], this[2], this[0]); }
  get zzy() { return new Vec3(this[2], this[2], this[1]); }
  get zzz() { return new Vec3(this[2], this[2], this[2]); }
}

Swizzles can operate between vector sizes as well. In practice it looks like this:

let v1 = Vec3(1, 2, 3);
let v2 = v1.zyx; // Vec3(3, 2, 1);
let v3 = v2.zz; // Vec2(1, 1);
let v4 = v1.xyzz; // Vec4(1, 2, 3, 3);

(These do break the "don't allocate lots of new objects rule a bit, but as a convenience I think it's worth it.)

Operator overloading

v1 = v2 + v3;

... is what I WISH I could implement here. Got your hopes up for a second, didn't I?

But no, JavaScript still stubbornly refuses to give us access to this particular tool because some people shot themselves in the foot with C++ once I guess? Check back in another decade, maybe?

TypeScript

I'm sure this will be mildly controversial, but I'm also leaning very strongly towards implementing the next version of glMatrix in TypeScript. There's a few reasons for this, not the least of which is that I've been using it much more myself lately as part of my job, and my understanding is that it's becoming far more common across the industry. It also helps spot implementation bugs, offers better autocomplete functionality in various IDEs, and I feel like the tooling that I've seen around things like documentation generation is a bit better.

As a result, having the library implemented natively in TypeScript feels like a natural step, especially considering that it doesn't prevent use in vanilla JavaScript. We'll be building a few different variants of the distributable files regardless.

Older Browser/Runtime compatibility

While I do feel very strongly about backwards compatibility of the library, that doesn't extend to supporting outdated browsers or runtimes. As a result while I'm not planning on doing anything to explicitly break it, I'm also not going to put any effort into supporting older browsers like IE 11 or fairly old versions of Node. Exactly where the cutoff line will land I'm not sure, it'll just depend on which versions support the features that we're utilizing.

ES Modules-first approach

glMatrix has used ES Modules as it's method of tying together multiple source files for a while, and I don't intend to change that. What I am curious about is how much demand there is out there for continuing to distribute other module types, such as CommonJS or AMD modules.

One thing I am fairly reluctant to continue supporting is defining all the library symbols on the global object (like window), since the library itself is already reasonably large in it's current form and the above changes will only make it larger.

Which brings me to a less exciting topic:

Caveats

All of the above features are great, and I'm sure that they'll be a net win for pretty much everybody, but they come with a few caveats that are worth being aware of.

File sizes will be larger

The addition of all the instance methods on top of the existing static methods, not to mention the large number of swizzle operations, would result is the library growing in size by a fair amount. I don't have numbers just yet, but I'd guess that the total size of glMatrix's distributable files growing by about 1/3 to 1/2 is in the right ballpark.

Obviously one aspect of building a well performing web app is keeping the download size down, and I don't want to adversely affect that just for the sake of adding some conveniences to the library.

I also, however, expect that most any developers that are truly size-conscious are already using a tree shaking, minifying build tool that will strip away any code that you're not actively accessing.

To that end, glMatrix's priority would be to avoid doing anything that would interfere with effective tree shaking rather than to try and reduce the base library size by only implementing a bare bones feature set.

length is complicated

One natural attribute that you'd want on a vector alongside the x, y, z, and w attributes is a length that gives the length of the vector itself, something that's already computed by vec{2|3|4}.length(v);

Unfortunately, length is already an attribute of Float32Array, and gives (as one would expect) the number of elements in the array.

We don't want to override length in our vector classes, since that would give nonsensical results in contexts where the object is being used as a Float32Array, which means that while we can retain the static length() function for backwards compat we'll need an alternative for the vector instances. I landed on magnitude (with an shorter alias of mag) as the replacement term, though I personally know I'm going to goof that one up at least once, and spend more time than I should wondering why the "length" of my vector is always 3. 😮‍💨

Vector/Matrix creation time will be slightly worse

This is a big one, as one of the most frequent criticisms leveled at glMatrix is that the creation of vectors and matrices is expensive compared to other similar libraries. This is because (for reasons that honestly escape me) creating TypedArray buffers or views is a slower operation in most JavaScript environments than creating a JavaScript object with the same number of elements/members.

My overall response to this has been that for many apps you'll eventually want to convert the results to a Float32Array anyway for use with one of the graphics APIs, and that avoiding creating a lot of temporary objects is a good practice for performance anyway, regardless of the relative cost of creating those objects. Both of those principles are still true, but it doesn't change the fact that this aspect of glMatrix is simply slower than some competing libraries.

The above proposals will not improve that situation, and in fact are likely to make it a bit worse. Having some extra logic in the extended classes constructor before passing through to the super() constructor will unavoidably add some overhead for the sake of providing a much nicer, more flexible syntax for developers.

If I were starting fresh I may very well take a different approach, but as I said at the top of this post backwards compat is very important to me for a library like this, so this is an area where I'm willing to accept this as a relative weakness of the library to be weighed against what I consider to be it's many strengths. Your milage may vary.

Accessors/instance methods will have overhead

As nice as it is to be able to access the object components by name, using the getter v.x is likely to always be a bit slower than accessing v[0] directly. Similarly some of the instance methods are likely to just pass through to the equivalent static method, especially in cases that would otherwise involve a significant amount of code duplication. For example:

export class Mat4 extends Float32Array {
  multiply(b) {
    return Mat4.multiply(this, this, b);
  }

  static multiply(out, a, b) {
    // Full matrix multiplication implementation.
  }
}

While I'm not necessarily a fan of adding functionality that's known to be less efficient than it could be, in this case I think that the aesthetic/usability benefits are worthwhile. And it's worth considering that there are plenty of times where the clarity of the piece of code will be more valuable than ensuring it uses every clock cycle to it's maximum potential. (I mean, lets be realistic: We're talking about JavaScript here. "Perfectly optimal" was never in the cards to begin with.)

I am happy knowing that in cases where the difference in overhead has a material impact on an application's performance, the conversion from accessors to indices, or to calling the static version of functions directly can be made painlessly and in isolation.

Preview

The code snippets in this post are all pretty simplistic, but I've put a fair amount of effort into validating this approach already, and currently have a WIP version of a potential glMatrix 4.0 available to look through in the glmatrix-next branch of this repo. It is definitely not in a generally useable state at this point, but looking at the vec2.ts, vec3.ts, and mat4.ts files should give you a good idea of how things are likely to look when everything is done.

Most of my efforts so far have gone into ensuring that things can work the way that I wanted, toying with file and directly structure, ensuring that the generated docs are clear and useful, and learning far more than I anticipated about TypeScript's quirks. But now that I'm satisfied that it's possible I wanted to gather some feedback from users of the library before pushing forward with the remainder of the implementation, which will probably be largely mechanical, boring, and time consuming. I'll likely take a week off work at some point to finish it up.

Thank you!

Thank you to everyone who has made use of glMatrix over the last 12 years! It's been incredible and humbling to see all the amazing work that it's been part of. And an even bigger thank you to everyone who has contributed to or helped maintain the library, even when I personally haven't had the time to do so!

[hmans]

Yes to all of these! Some comments:

• Having both a functional and object-based API is great 😍
• I think .x, .y etc. accessors are great, even at whatever performance cost they incur. The documentation could guide users towards using the property accessors for convenience, and the index accessors for performance.
• Don't care much about swizzling. I assume these will be for vectors only? I'm hoping you auto-generated the code in vec4.ts. :-)
• I've seen some libraries implement custom operator faux-overloading through Babel plugins. Maybe this could be an optional thing to provide to people who're already transpiling.

[toji]

Yes, the swizzle code is auto generated! 😆 I am definitely not patient enough to do it manually. And yes, it would just be for the vectors. I don't think it makes sense for quaternions or matrices. I am considering attributes that return vectors representing matrix columns or rows, though.

[Maksims]

Mostly a great improvements while keeping backwards compatibility!

In realtime context, allocations are expensive, especially Float32Array objects are a bit more expensive than simple objects. So it is best to avoid it as much as possible, or provide clear API design to communicate it to the user. One are of such I found that if a getter used from object, then it is expected to be very simple operation, and not an expensive allocation.
In this case while swizzles are great API design - they eventually can lead to unexpected allocations while looking harmless. Using a method notation instead of getter, looks worse from esthetically, but feels more serious.

Regarding of calling static method from class method, it is worth thinking which of them will be more popular. As I would assume class method is more often used so within class static method it could call class methods but not the way around.

[DavidPeicho]

I would be careful with static methods, as it broke bundlers tree-shaking up to 2020. I remember that it happened in major bundling tools: webpack, esbuild, etc... I guess they all use the same tool for optimization / minification etc... It might not be the case anymore, but it would be great to give it another try.
I had back then a sample to debug it, and a single class export could break everything with a static method.

As mentioned above, I would love to see swizzle with an out reference as well to prevent the allocation

[hmans]

One minor argument against moving to classes that inherit from Float32Array. One thing I really appreciate about working with gl-matrix is that the user of my libraries can just provide arrays of numbers as arguments to functions that then feed them to gl-matrix.

In a little game library I'm currently working on, I can do

autorotate: [1, 2, 3]

With the new version as described above, I think I then have to always do:

autorotate: new Vec3(1, 2, 3)

Which is just ever so slightly "worse" (purely in the sense that it increases boilerplate in userland.)

It would be great if the functional-style static functions like Vec3.add(a, b) are still able to operate on normal arrays like the one above. (Which, of course, leads to the question if we will then need to always stick with the functional-style API to make sure arrays can still be used in userland, which then eventually poses the question if we truly need a class-based API.)

Aaaaaah API design, I love/hate it :-)

[ibesora]

Love all this. Swizzling support is awesome.

Minor question about your performance concerns, particularly around accessors/instance methods overhead: Do you have any metrics on that? I wouldn't expect an extra function call to be noticeable

[sketchpunk]

"Vector/Matrix creation time will be slightly worse". How about creating two Vec3 objects that share the same functionality. Like a FVec3 extends Float32Array and JVec3 extends Array, then each one can somehow pull in the same methods, getters & setters.

I've been using a vec3 extended Float32Array for a few years now but in the last year or so I've been moving away by using regular javascript arrays for math heavy applications that doesn't need that gl compatibility that float32arrays provide. Having both an Array and Float32Array type can give users a choice of what sort of data structure to use.

Also, for your constructors, maybe you'd like to try out overloading. Here's how I initialize a vector 3 in typescript. Intellisense can then tell you the various ways to initialize it by having each constructor defined.
https://github.com/sketchpunk/oito/blob/main/packages/core/src/Vec3.ts#L18

So the following is possible

• new Vec3( [0,1,0] )
• new Vec3( 0, 1, 0 )
• new Vec3( 1 ) // great for scale vectors

[shannon]

I am excited for this!

One thing I might offer as a suggestion is the constructor may lead to an unexpected behavior when compared to shader code.

const v1 = new Vec3(5);

As with shader code I would expect it to create a value of (5, 5, 5) here, instead it makes a value of (0, 0, 0, 0, 0), a Float32Array of length 5. Whether this is crucial or not I can't say but it would be nice to have parity there.

Your code would just require an additional check:

export class Vec3 extends Float32Array {
  constructor(...values) {
    switch(values.length) {
      case 3: super(values); break;
      case 2:
      case 1: typeof values[0] === 'number' ? super(3).fill(values[0]) : super(values[0], values[1] ?? 0, 3) ; break;
      default: super(3); break;
    }
  }
}

let v1 = new Vec3(); // Creates a vector with value (0, 0, 0)
let v2 = new Vec3(1, 2, 3); // Creates a vector with value (1, 2, 3)
let v3 = new Vec3(v2); // Creates a copy of v2
let v4 = new Vec3(5) //Creats a vector with value (5, 5, 5)

let arrayBuffer = new Float32Array([0, 1, 2, 3, 4, 5, 6, 7, 8]).buffer;
let v5 = new Vec3(arrayBuffer); // Creates a vector mapped to offset 0 of arrayBuffer (0, 1, 2)
let v6 = new Vec3(arrayBuffer, 16); // Creates a vector mapped to offset 16 of arrayBuffer (4, 5, 6)

This also makes sense because the length as the first argument is no longer useful as it is expected the length will always be 3.

[toji]

Lots of great comments, thanks! Replying to a few real quickly:

Regarding of calling static method from class method, it is worth thinking which of them will be more popular. As I would assume class method is more often used so within class static method it could call class methods but not the way around.

As I have it right now that's not really possible because the static methods are formulated such that the first operand and the output are always the same object, which isn't guaranteed in the static versions. Also, I want the static versions to continue to work on things like raw arrays, so we can't assume that we can always call through to the instance method.

I would be careful with static methods, as it broke bundlers tree-shaking up to 2020.

That's not something I was aware of, thanks for bringing it to my attention! I'll do some more research on it, and if you have any links to related issues/docs/fixes/etc I'd appreciate it.

One thing I really appreciate about working with gl-matrix is that the user of my libraries can just provide arrays of numbers as arguments to functions that then feed them to gl-matrix.

Yeah, I appreciate that too! And the changes described here won't break it, it just requires a bit of discipline on your part as a library author. You'd just need to follow the pattern that the library itself is going to follow: Any time glMatrix returns a new vector/matrix it'll be an instance of the class, but any time it accepts a vector or matrix it only has to be an array-like object with enough elements.

In the TypeScript code I've done so far I declare types such as Vec3Like or Mat4Like which I use for almost all function params.

export type Vec3Like = [number, number, number] | Float32Array;

export class Vec3 extends Float32Array {
  static add(out: Vec3Like, a: Readonly<Vec3Like>, b: Readonly<Vec3Like>): Vec3Like {
    out[0] = a[0] + b[0];
    out[1] = a[1] + b[1];
    out[2] = a[2] + b[2];
    return out;
  }
}

This prevents the use of any of the instance methods in the function implementations, but retains maximum flexibility so it's a quirk I'm willing to deal with. It means that all of the following are still valid:

let v1 = new Vec3(1, 2, 3);
let v2 = new Float32Array([4, 5, 6]);
let v3 = [7, 8, 9];

v1.add(v3);
Vec3.subtract(v2, [9, 9, 9], v1);
v1.multiply(Vec3.scale([0, 0, 0], v3, 2.5));

Minor question about your performance concerns, particularly around accessors/instance methods overhead: Do you have any metrics on that? I wouldn't expect an extra function call to be noticeable

Not good ones yet. I don't expect it to be a lot, but it's probably something you could measure if you were, for instance, doing a tight loop of cascading matrix updates over a large scene graph.

I primarily bring it up because when it was first released I pitched glMatrix as "stupidly fast" and proudly showed benchmarks of it handily trouncing competing libraries at the time. That's not the case today, and I'm not really interested in pursuing the title of "fastest possible thing" at the expense of usability any longer, though performance is still one of my largest considerations. As such I feel it's noteworthy when design decisions are made that involve a performance compromise, big or small, but won't let it be a blocking factor unless it's egregious.

How about creating two Vec3 objects that share the same functionality. Like a FVec3 extends Float32Array and JVec3 extends Array, then each one can somehow pull in the same methods, getters & setters.

I've thought about it. Still trying to work out how to do so without making the API significantly more annoying to work with, especially across multiple libraries. In the meantime, as I pointed out above, the library will still function perfectly well with raw arrays in most cases.

const v1 = new Vec3(5);
As with shader code I would expect it to create a value of (5, 5, 5) here, instead it makes a value of (0, 0, 0, 0, 0), a Float32Array of length 5

Ooh! I'd considered doing expansion of the scalar value here previously but dropped it because I was trying to make the constructor simpler. Having scalar inputs accidentally change the length of the underlying array is bad, though, and I'm embarrassed I overlooked it. It's worth implementing the (admittedly nice) feature just to avoid the problem. Besides, it sounds like there's multiple people who would like that feature anyway! Thanks for pointing this out.

[arodic]

Hi Brandon,

I'm happy to see you are getting back to this project!

Typescript is the way to go, no doubt (I used to be sceptic until I was forced to get into it for work). Also esmodule-first approach is a trend I'd love to see more!

As for backwards compatibility, it is really neat that you've put so much effort to please the current users but having multiple ways of using the library can add confusion. With that in mind, I'd suggest to put deprecation on fast track. Perhaps put deprecation warnings in second release and deprecate old API right after that. That way, users can rip the band-aid off in two simple upgrades.

I landed on magnitude (with an shorter alias of mag) as the replacement term

Did you consider len()?

Swizzling is nice but I almost never use it JS. By the time I need to swizzle my vectors I'm already in GPU land but that's just me.

Would you consider adding unit vectors? I find them handy all the time!

As for library size, I don't think you should feel constrained by it. As long as it is tree-shakable, just have fun writing useful code :) Have you thought about adding geometric primitives, camera matrix utils and so on?

[peterreeves]

I landed on magnitude (with an shorter alias of mag) as the replacement term

What about dimensions?

Everything else looks good, especially TypeScript support. WebGL projects tend to be very large, and large projects are generally written in TypeScript for sanity reasons.

[toji]

Did you consider len()?
What about dimensions?

len() is already a static alias for VecN.length(), and my concern there is developers looking at the static methods and seeing length/len and then looking at the instance attributes and seeing length/len and thinking "Ah! It's the same!" when it very much is not.

As for dimensions, that suggests to me a width/height/depth rather than a scalar value. (IE: The dimensions of a photo are 3"x4", but the length of a Vec2(3, 4) is 5.)

Honestly there's just not any great options here that I've come across.

Would you consider adding unit vectors? I find them handy all the time!

I'm not sure what a "unit vector" means in that context? Like a vector that's constrained to always be 1 unit in length? Could you explain a bit more?

[donmccurdy]

@toji the proposed API changes sound really helpful from usage perspective!

However, I'd be really careful of assuming tree-shaking will just work, without testing against a couple of the more modern bundlers. The issue about tree-shaking class methods in Rollup appears to have closed without any resolution.

Things have generally improved since the comment in rollup/rollup#349 (comment), but I (still) feel that most bundlers are far worse at tree-shaking than Google Closure Compiler today, and very finicky about it. For example — I could easily imagine a line of code like this breaking all dead code elimination on the Vec3 class:

const swizzle = flip ? 'zyx' : 'xyz';

const [a, b, c] = vec[swizzle];

If that turns out to be an issue, perhaps a backup option would be to continue exporting operators as individual functions accepting arrays, but to also provide the classes (with instance methods, no static methods) for users who prefer them.

[toji]

So it turns out that doing all of the swizzle variants I was hoping for on Vec2, Vec3, and Vec4 would likely double the size of the library when minified. 😨 I know I said I wasn't too concerned about the library size but... that's significantly more than I was anticipating, and I'm not really willing to take THAT much of a size hit for one feature, especially one that will probably see reasonably light use.

Gonna experiment with injecting them dynamically (which totally goes against the TypeScript ethos, but whatever.) and if that doesn't work then I may just have to drop swizzles all together.

[nshen]

I prefer drop swizzles

[toji]

Fiddled around with the swizzle code last night and got to a place that I'm happier with.

I changed up the autogen code so that it writes out all the necessary symbols for the swizzle operations to a single, separate file (swizzle.ts), which can then dynamically generate the code necessary for them. But because the dynamic generation takes a moment (a couple of ms on my test machine) I don't have them automatically inject as a side effect of including the file, but instead it exports a function (EnableSwizzle() for now, will probably change that later) which can be called anywhere in your code to enable swizzle operations on all the vector types.

The primary upside to this is that if you don't care about the swizzle operators then you simple don't call that function and there's no overhead involved. If you're importing the types separately, rather than as a bundle, then you won't even need to download the swizzle table (the largest part). If you are using the bundled version the size impact has gone WAY down (from ~50Kb to ~5Kb), and tree shaking should more reliably cull those symbols out if you don't explicitly enable it.

The downsides are that there's an extra step involved if you want this feature, the implementation itself may be a bit slower, depending on how your JS engine optimizes it, and TypeScript doesn't recognize the swizzle operators because they're dynamically declared. That's unfortunate, but I'm wondering if I can get around it by having a generated .d.ts file that explicitly declares each swizzle variant. I'll have to explore that some more later.

Latest WIP code has been pushed to the glmatrix-next branch for anyone who is interested in taking a look!

[sketchpunk]

I'd be interested in adding a Transform object to gl-matrix. Is it something I can do on your glmatrix-next branch so it can launch with v4.0 or is it something I should wait for you to be completely done before contributing. Well, probably the first question should be does it make sense to add a transform object to gl-matrix.

https://github.com/sketchpunk/oito/blob/main/packages/core/src/Transform.ts

[nshen]

@sketchpunk It would be really useful if the transform object could like toCssTransform()
https://developer.mozilla.org/en-US/docs/Web/CSS/transform

[mreinstein]

If I were starting fresh I may very well take a different approach

That is what I'd rather see you do. v4 is going to be a big undertaking, and if you're going through all that bother I would rather see what you come up with, being able to take risks and explore new ideas without being chained via compatibility limitations.

What you're describing is different enough from traditional gl-matrix paradigm that I think it warrants this. There is absolutely no shame in making a new thing; gl-matrix will still live on and be amazing; it will just be different from the next great thing you design. You could even pick a name that isn't tied to gl. :)

operator overloading is what I WISH I could implement here.

Same. Maybe this could be a killer feature in a v4 fresh module not encumbered by legacy concerns. Years back I was looking through gl-matrix's issue tracker and someone raised the idea of a preprocessor/compile step. Maybe this is insane. But then again we live in a world where typescript has taken over, and what is that but an even more comprehensive pre-processor. :) The cumbersome syntax is probably my biggest gripe with gl-matrix, but I gladly put up with it because the awkwardness mostly comes from having to work around javascript spewing objects everywhere in most APIs, and gl-matrix doesn't generate any memory garbage.

because lets face it: typing v[0] instead of v.x is really annoying

I know that's a popular sentiment but is it really that bad typing v[0] ? I mean there's nothing special about .x or .y, they are just symbols we've all gotten used to. After a while v[0] and v[1] become fairly natural to look at.

using the getter v.x is likely to always be a bit slower than accessing v[0]

I don't want to go through every point in the list of ideas for v4 but getter/setter proxy performance is a concern. It's been a few years since I checked but in v8 these are costly. I dropped pixi.js for game dev because after doing some CPU profiling I discovered internally it uses getter/setters in various places like the transform components, accessing these tend to snowball, and it chews up a surprising amount of CPU when you're building non-toy sims/games that operate on thousands of objects in a tight render loop.

Using gl-matrix@3 and somewhat naive webgpu, I was was able to cut the cpu usage in half compared to pixi, and this was only a few weeks ago. Certainly not a scientific measurement but it seems like there are still some perf issues there.

Thank you to everyone who has made use of glMatrix over the last 12 years

Thank you for producing one of the best matrix libraries. People don't appreciate how important these low-level primitive handling libraries are. They are so foundational to make jank free games/sims/animations. I'm excited you have time to spend on this!

[mreinstein]

If you are feeling inclined to spend time on a successor to gl-matrix, I now have author privs for the matrix npm package. I'd be happy to hand this over to you if you'd find it useful @toji

[miko3k]

Strength of glMatrix is its absolute interoperatbility thanks to its universal representation. I personally love this feature. Classes will make it harder.

I would prefer more evolutionary changes, my personal list:

• proper typings: function add<T extends vec3>(out: T, a: ReadonlyVec3, b: ReadonlyVec3): T. I wanted to make PR for this but I would probably require conversion to typescript. I gave up in the end.

• deprecating weird functions, like vec2.transformMat4 (vec3.transformMat4 performs w-division; vec2.transformMat4 does not #415)

• split one function per module, to enable better tree shaking. This is harder than it seems though because of the current module structure. Swizzles could be normal functions. vec3.swizzleXXY and automatically dropped by the bundler.

• there's a only a small number of functions that allocate memory. I would provide two versions of each (vec3.createArray/vec3.createFloat32) and while keeping ARRAY_TYPE as "default". This would enable client code to choose better fitting version.

I realize that classes provide better ergnomy, however I strongly feel they will turn this into different kind of library.

[ova2]

A long time ago I've tried this approach. We have real-time UIs with many points. The problem with this approach was the increased app size. Native arrays are lightweight, but Float32Array has a big size. I prefer to do transformation from native arrays to Float32Array just before rendering step. I could not detect any performance optimizations when working directly with Float32Array. You really have to do a prototype first to be able to messure optimizations before implementing a new approach.

[shannon]

@toji I know you covered this in the description but I wanted to leave some feedback regarding the move to Typescript.

Over the years I have moved away from having any build step in my development process (bundling comes only at the publishing stage now). There are many reasons for this so I won't try to state them all here, but I have rather enjoyed the fact that gl-matrix was one of the libraries that I could just do import { mat4 } from 'https://cdn.jsdelivr.net/gh/toji/gl-matrix@v3.4.1/src/index.js', or get as fine grained as I would like using the individual source files. There are more and more libraries implementing standard ESM so this is becoming easier and I am almost at a point where I can do away with NPM entirely.

I wanted to try experimenting with the v4.0 branch but I soon discovered this approach is no longer possible and that there really is no easy way for me use it right now since it has not been published anywhere in a transpiled form. gl-matrix is something I use a lot and it feels a bit like a step backwards.

There has also been a lot of open discussion about Svelte moving away from using Typescript code (while still using JSDoc to implement Typescript based type checking).
Twitter Discussion: https://twitter.com/jutanium/status/1639341148157140993
This article gives a nice overview of that discussion: https://dev.to/thepassle/using-typescript-without-compilation-3ko4.

The points being made by Rich Harris and others are very pertinent when it comes to a library like gl-matrix. Anyways, I don't really expect you to change the entire code base, but I had hoped this feedback can be of some value for you.

[toji]

Hey @shannon!

This has been an interesting journey for me, because while I've gotten quite comfortable working in TypeScript overall I'd never tried to build middleware with it before now. I've been playing with different ways of building, importing, and using the library in other projects (I'm intent on inflicting any pain on myself before asking the community to endure it), and trying to find the right configuration for maximum ecosystem compatibility.

Being perfectly honest: It sucks. Not really the language or the tooling, but the packaging and distribution story is a mess. There's not a clean way to distribute "A TypeScript Library" that everyone can just use, because you have to match about 50 different assumptions about how their own project is set up to be compatible. Seems like there's some patterns that can be used to make it a bit better or worse, but overall it's just not practical.

Which is a shame, because there's aspects of TypeScript that are really quite nice, even for a library like this. Things like being able to specify certain arguments as Readonly for example, is a great way both to express the intent of the function and keep your implementation honest. I've found that generating docs is cleaner with TypeScript too.

So I'm going to some advice that I got when I first started lamenting the packaging situation online: glMatrix 4.0 will still be written in TypeScript, but when it's packaged for distribution it'll just be JavaScript with some .d.ts files alongside. This seems to be the single most compatible approach across the entire ecosystem (and even it still suffers from incompatibility potholes, but that's more JavaScript's fault than TypeScript at that point.) So if you want to just import directly from a CDN into your vanilla JS project it'll just work, and if you install and import the distributed files in a TypeScript app you'll still get the definitions. Feels a bit silly to write in one language and then compile to another in order to let projects written in the first language consume it, but that just seems to be where TypeScript is at right now.

I'll have the build files available in at least three forms: Separate ES modules files for individual imports, a bundled ES module file with the full library in it, and a Common JS bundle as well, since I think that's still a reasonably common use case. If anyone wants to copy the TypeScript source into their own project and configure it manually they're welcome to do so, but it won't be something I try to support directly.

Anyway, hope that puts some concerns at ease! There will be a build step but it won't be you that has to do it. :)

And sorry that it's taking so long to get this published in an easily consumable place! I actually want to work on that really soon because I need it for my own project. Need to look into the best way to publish "beta" packages with NPM and the like.

[shannon]

Thanks @toji I completely understand your point of view. I don't want to rush you either so I will experiment when it's published :-)

[mreinstein]

Over the years I have moved away from having any build step in my development process (bundling comes only at the publishing stage now)

@shannon 100% agree with you, but unfortunately we're in the minority these days.

[ibgreen]

@toji good to see you returning to this great project and thanks for taking the time to share a roadmap.

FWIW, the vis.gl / deck.gl ecosystem already contains an implementation of gl-matrix-based classes that seems quite similar to what you propose. One of our core component is the @math.gl/core module which contains Vector, Matrix etc classes subclassed from arrays with methods that call gl-matrix functions on this...

We built these wrappers back in 2017 and have used them successfully in many frameworks and applications over the years. Our implementation seems fairly similar to the direction you outline, so if nothing else could be useful as a reference.

Possibly the biggest difference from your proposal is that we chose to subclass from Array rather than Float32Array.

• I don't think subclassing Float33Array was possible back then
• but more importantly, 32 bit precision would simply be too limiting for our use cases
• we need to maintain higher precision math on the JS side (mainly geospatial 3D use cases) before converting final results to Float32 for shader usage.
• Also we wanted our class instances to be able to be used wherever standard JS arrays were expected. So in our case, all the functional gl-matrix functions still work with instances of our vector and matrix classes.

It would certainly be nice if this developed to a point where vis.gl could replace our own @math.gl/core wrapper module with a new official class based gl-matrix, however that depends on the direction your design takes. Regardless all our code is MIT licensed so if you see anything you like, feel free to use it without asking.

PS - Unfortunately, while we have been using gl-matrix as the base for math.gl since 2017, we are just about to fork and drop the gl-matrix dependency altogether since we have not found a way to work around gl-matrix lack of ES module support: #455. We'll basically be adding copies of the gl-matrix functions we use to @math.gl/core until the problem is resolved.

@Pessimistress

[toji]

A first beta release for glMatrix v4 is now available! (Only took me 5 months! Thanks for your patience)

You can install it from npm with the following command:

npm install gl-matrix@beta

Or import it without an install with something like this:

import { Vec3, Mat4 } from 'https://cdn.jsdelivr.net/npm/gl-matrix@beta/dist/esm/index.js';

Documentation is available at https://glmatrix.net/docs/v4/

Usage notes

Since this is a first beta release I expect that it's not going to work for everyone, but that's why I'm putting it out there: So that I can get feedback on usability issues.

If you have a project that already uses glMatrix this should be a mostly drop-in replacement. The biggest thing you'll have to update is the path to the modules. Previously it would have been, for example, node_modules/esm/index.js and now it's node_modules/dist/esm/index.js. I'm still debating whether or not I want to adjust that, because the previous way can cause deployment issues and if I'm going to make a breaking change then now is the time. Feedback appreciated!

Otherwise, a few things to be aware of:

• Obviously if you experience any notable performance drops or correctness errors when using v4 I want to hear about it!
• This release only contains ESM and CJS support. I'm very interested in hearing what environments are still relevant to users, because there's ~3 million ways to package up JS libs and I have no plans to support all of them, but there are some (like is pointed out in ES Module Support #455) that are obviously still useful and that I'll need to investigate adding.
• You can import either the capitalized (newer style) or lowercase (for backwards compatibility) version of each type and they'll work the same way. I find I prefer to use the capitalized versions so that I can tell at a glance if the code I'm using is v4 or v3, but do whatever works for you.
• Most of the methods from the previous version of the library have been ported over, but I've left a select few out (such as the random() methods on the vectors, which were never very good to begin with and were likely to cause you issues if you depended on them) If I've omitted something critical to you, please let me know! No guarantees that it'll ship as a first-class method in v4 but I'll see if I can provide a workaround.
• There is definitely not a 1:1 mapping between the static methods and the instance methods, and I fully intend to keep it that way. Simply put, not every static method makes sense as an instance method.
  • For example, instead of having an instance version of Vec3.clone(v); it makes more sense to simply call new Vec3(v);
  • Similarly methods like Vec3.lerp() don't make much sense with the pattern that I've been using for most of the instance methods (which is generally to perform the same action as the static variant as if it was called with Type.method(this, this, operandB); In those case I'm happy to simply leave them as static-only, because they still work perfectly well with instances of the class.
• That said, there's probably room for more instance methods than I've implemented so far (Quat2 has almost none right now, for example) so let me know if there's any that would be particularly useful!

Thanks in advance for any feedback that you can provide!

[elalish]

Awesome, thanks! I just checked with manifoldCAD.org that I can indeed update to your beta and it continues to work as expected. However, I currently have a huge .d.ts file checked in that I scraped from an old types definition of your library - I'd love to get rid of that and use your new types instead. However, I'm blocked by Monaco (VSCode's editor) only being able to import single, monolithic .d.ts files: https://stackoverflow.com/questions/43058191/how-to-use-addextralib-in-monaco-with-an-external-type-definition/66948535#66948535

Is there a chance you could publish a bundled .d.ts file?

[toji]

Great feedback, thanks! I'll look into how to bundle d.ts files.

[donmccurdy]

In glTF Report (which also uses Monaco) I'm compiling bundled types with rollup-plugin-dts.

TypeScript itself has unfortunately been firm in not supporting bundled output directly:

microsoft/TypeScript#4433

[DavidPeicho]

Great work, thanks a lot! 🙏

I might be wrong but there is no more "exports" used in the package.json? It's pretty nice to import:

import {Vec3} from 'gl-matrix/vec3.js';

I guess this is done to give a true ESM package? Otherwise browsers should support import maps.

[typhonrt]

Greets @toji! Thank you for creating and maintaining this package!

I have fully integrated v4 into my larger runtime library that re-bundles gl-matrix to great success after resolving two issues. Like others have mentioned providing bundled types is very useful and key to how I re-bundle gl-matrix. I have a lot of experience with that and have a fork that uses rollup-plugin-dts to provide bundled types. I also added a proper exports field to package.json. I would be glad to submit a PR, but would like to have a discussion here first.

You can find my fork that solves the bundled types / exports in package.json here:
https://github.com/typhonjs-svelte/gl-matrix/tree/glmatrix-next

You can see a comparison of changes here:
glmatrix-next...typhonjs-svelte:gl-matrix:glmatrix-next

There is a fork of this fork w/ the dist / types checked in, so that proposed changes can be tested now via adding "gl-matrix": "github:typhonjs-svelte/gl-matrix-dist#glmatrix-next" to dependencies / devDependencies in package.json.

In addition to the main bundled esm / cjs export there are two additional sub-path exports for src and src/modern. These sub-path exports provide the unbundled / not minified ESM source and are very useful particularly in a library / re-bundling use case like mine. I do not minify my library leaving minification as a build step to the consumers of my library. Also the src/modern sub-path export simply excludes the additional lowercase compatibility exports of mat2, vec3, etc. There is a new index file in ./src/index-modern.ts that only exports the modern API. When re-bundling gl-matrix I simple use export * from 'gl-matrix/src/modern';

There are two versions of the generated bundled types; one w/ the compatibility support (./types/index.d.ts) and one that excludes the compatibility support (./types/index-modern.d.ts). Only the src/modern sub-path export references the "modern" types.

Added devDependencies include: rollup, rollup-plugin-dts and tslib (necessary for the DTS plugin). Additionally there is a Rollup config file ./rollup.config.mjs and a new NPM script bundle-types. When the build script is run the unbundled types are produced separately in ./.types.

I'd be glad to have a conversation about any of the above and provide clarity as necessary in how these changes add significant flexibility to consuming gl-matrix.

---

There is a further issue to discuss besides bundled types & exports in package.json. There is an issue with tree-shaking gl-matrix due to the module scoped temporary variables at the end of various files. IE all the temporary variables essentially. I have fully tested final project builds with my library and pinpointed the tree-shaking issues to the module scoped temporary variables. Instead of using the actual new instances of the respective classes for the temporary variables simply creating the appropriate arrays for the <X>Like types solves the tree-shaking issue.

In my larger runtime library I only use directly Mat4 and Vec3, however in the bundled result Quat and Vec4 are included. With the temporary variable fix in place a final project build w/ my library only includes Mat4 and Vec3 saving 2k un-minified lines of code in the final output.

To test this indeed fork #3; I made a fork of the gl-matrix-dist repo and modified the module scoped temporary variables committing those changes and using that to build my runtime library. I also for good measure used import type { XXX } in the TS code to explicitly show imports that are for types only.

You can see the comparison here (scroll down to the TS files as this is a fork of the dist repo):
typhonjs-svelte-scratch/gl-matrix-dist@glmatrix-next...typhonjs-svelte:gl-matrix-dist-test:glmatrix-next

This version can be tested now via adding "gl-matrix": "github:typhonjs-svelte/gl-matrix-dist-test#glmatrix-next" to dependencies / devDependencies in package.json. I made sure to of course run the gl-matrix tests and everything passes.

I'd be glad to post a clean follow up PR that corrects the tree-shaking issue after the initial potential PR for bundled types / exports is resolved.

---

I look forward to chatting about and resolving these issues and see gl-matrix v4 be the best that it can be!

[toji]

This is excellent feedback everyone, especially @typhonrt! I'll try to get another version of the code up this week that fixes the exports fields, bundled definitions, and tree shaking issues.

One initial question I had for @typhonrt before I dig in much further (I'll probably have more questions for you as I go) is if your change to bundle both a regular and modern variant of the project had a technical reason behind it (like it was interfering with tree shaking) or if it was simply a usage preference? Maybe it made it easier to catch places where your code needed to be updated?

[typhonrt]

One initial question I had for @typhonrt before I dig in much further (I'll probably have more questions for you as I go) is if your change to bundle both a regular and modern variant of the project had a technical reason behind it (like it was interfering with tree shaking) or if it was simply a usage preference?

Providing the modern API when re-bundling is a preference to provide a minimal API that isn't potentially confusing to users of my library. I don't have legacy concerns with my library and it was easy to update my internal usage to the new gl-matrix API.  Since there is a lot that my library provides keeping the exported symbols in check and streamlined is useful making documentation cleaner as well.

In short, internally in my library I'm using gl-matrix for a Svelte based windowed GUI API / dev experience for optimized window movement and validation against browser bounds and custom regions. It's also potentially used for validation and containment of objects inside app window bounds. While gl-matrix is available as part of my larger runtime library no external devs are consuming it separately yet, so providing the modern API is the cleanest path.

Hopefully I don't stray too far here in the conversation, but this can provide more context on how I consume and re-bundle gl-matrix. Interestingly enough my library is ESM native and I have tooling, esm-d-ts that is almost in "beta" that generates TS declarations from ESM source. When building a library and optionally configured esm-d-ts can retrieve and combine TS declarations from external top level libraries that are re-exported; ala export * from 'gl-matrix/src/modern pulls in the bundled types from gl-matrix and provides a unified TS declaration for the subpath export in my library. This is why providing bundled types in gl-matrix is really useful in my use case.

I also use the TS declarations for my entire library to create documentation. By the end of the week I should have integrated documentation finished, so you'll be able to see how I export the modern API and how it is cross-linked in the documentation from where gl-matrix is re-exported in my library and where it is used in my library.

---

A potential idea for consideration is to switch things around w/ how gl-matrix is distributed. Rather than providing gl-matrix by default with the compatibility API it is possible to provide separate bundles and source subpath exports under a compat subpath. Importing the gl-matrix bundles in compatibility mode could then be accomplished like this import { vec3 } from 'gl-matrix/compat';. By default only the modern API is available: import { Vec3 } from 'gl-matrix';.

This is easy enough for long time users of gl-matrix that need the compatibility mode to adjust the import adding gl-matrix/compat. The nice thing about this approach is that it provides a cleaner implementation moving forward as you can handle the compatibility API just in an index file versus a specific lowercase variable export in each of the respective source files. You could create index-compat.ts and provide the compatibility exports  like this (abbreviated example):

import { Mat2, Mat2Like } from './mat2.js';
import { Mat2d, Mat2dLike } from './mat2d.js';
// more code, etc.

export {
  Mat2, Mat2 as mat2, Mat2Like,
  Mat2d, Mat2d as mat2d, Mat2dLike,
  // more code, etc.
};

I haven't tested if there are additional tree-shaking concerns from the existing default compatibility or pre-bundled / minified output. My assumption is that likely it's only the temporary variables that cause a problem. I'd be glad though to help out with any testing before release to verify aspects like this from a full consumer use case. I can easily test the non-source main exports for tree-shaking concerns.

This proposal above with providing a specific compatibility subpath could lead to an exports field similar to this in package.json:

{
  "exports": {
    ".": {
      "types": "./types/index.d.ts",
      "import": "./dist/esm/gl-matrix.js",
      "require": "./dist/cjs/gl-matrix.js"
    },
    "./compat": {
      "types": "./types/index-compat.d.ts",
      "import": "./dist/esm/gl-matrix-compat.js",
      "require": "./dist/cjs/gl-matrix-compat.js"
    },
    "./src": {
      "types": "./types/index.d.ts",
      "import": "./dist/src/index.js"
    },
    "./src/compat": {
      "types": "./types/index-compat.d.ts",
      "import": "./dist/src/index-compat.js"
    },
    "./package.json": "./package.json"
  }
}

[typhonrt]

I do have one more point of consideration regarding gl-matrix distribution. I'll keep this one short as the last two posts were verbose. It's my general opinion as a library author myself that it is unnecessary to minify libraries insofar that minification is a downstream consumer concern most appropriately handled in a single minification step in any final production bundle on the consumer side. I can provide expanded reasoning here, but keeping this short!

Since you are concerned about backward compatibility from the CJS angle it makes sense to provide a CJS bundle as that is where the CJS conversion is facilitated. However, from the ESM side of the library release you even don't necessarily need to provide an ESM bundle. Any ESM consumer of the library can bundle / minify consumption of gl-matrix downstream.

An example of the exports field in package.json in this setup with the above "compatibility" idea would be:

{
  "exports": {
    ".": {
      "types": "./types/index.d.ts",
      "import": "./dist/src/index.js",
      "require": "./dist/cjs/gl-matrix.js"
    },
    "./compat": {
      "types": "./types/index-compat.d.ts",
      "import": "./dist/src/index-compat.js",
      "require": "./dist/cjs/gl-matrix-compat.js"
    },
    "./package.json": "./package.json"
  }
}

[shannon]

@toji I do have a specific API change question about the next version. It looks like the Quat.fromEuler method is missing the order option added in the v3.4.1.

https://github.com/toji/gl-matrix/blob/v3.4.1/src/quat.js#L460
https://github.com/toji/gl-matrix/blob/glmatrix-next/src/quat.ts#L689

Was this missed or was it omitted intentionally? I find myself in a position where I need to specify the order and further I need to actually use extrinsic instead of intrinsic. I was considering submitting a PR against master with an extrinsic order option (along with a complementary Quat.getEuler method). But I'm wondering if this functionality is not desired in the next version if it makes sense to just keep it as separate functions in my code.

For my use case, I am building a game engine editor UI and often times 3d editors such as blender will use XYZ extrinsic as the default (in the UI) because it is easy to visualize and reason about when creating artistically. gl-matrix v3.4.1 uses ZYX intrinsic by default and in the next branch it appears to be the only option.

I am also asking so that I can further my own understanding of this specific math as I am not an expert and perhaps there is a simpler way to handle this mathematically that I have missed. The order option does lead to quite verbose code, and with the intrinsic/extrinsic option it will be even more so. Or perhaps it would make sense to create a Euler class similar to Three.js.

Further reading on the subject:
#407
#329
https://dominicplein.medium.com/extrinsic-intrinsic-rotation-do-i-multiply-from-right-or-left-357c38c1abfd

Edit: Upon further research I see that you can just reverse the order to switch between extrinsic and intrinsic. So there is no need for more code. But I am still curious if the order option will be added to version 4.

[elalish]

This is perhaps tangential to the purpose here, but I find myself really missing a few of the niceties of glsl in this library, particularly the ability for basic Math operations to apply to vectors. Maybe something like op(v, Math.exp) to apply any supplied scalar function to each component of the vector. Also a few of the common glsl functions would be really convenient to have here: my favorites are clamp, mix, and smoothstep.

[michaeldll]

Any idea on when a first stable(ish) version would be available for use? I really want to try out v4.0, but as I understand it it's still a bit early for actual beta usage.

[blfunex]

For swizzling why not return vector compatible tuples instead of creating the vectors (JS Array vs Vector f32 array buffer view), I reckon js engines have mastered js array optimisations better than array buffer views.

[blfunex]

Since swizzling is opt in, typescript for it should also be opt in, like npm i -D @types/gl-matrix/swizzle, to add them through global interfaces into gl-matrix own definition.

[Swoorup]

Is there a way we could also support higher precision? As @ibgreen mentioned earlier, 32-bit precision is bit limiting for my use case as well.

Hence I've always been setting glMatrix.setMatrixArrayType(Array) at initialisation to retain the precision

[shannon]

Would it make sense to make it configurable to extend Float64Array instead of Float32Array?

[shi-yan]

Is there a way to specify the handedness of glMatrix?

i.e. change it from the right handed system to left handed system.

because I want to use it for WebGPU, and according to the WebGPU spec, it uses the DirectX coordinate systems (left handed system).

[elalish]

Oh no, if WebGPU uses left-handed, I blame you, @toji 😝

Left-handed matrix math should have been expunged from the world years ago... In fact it had been until some DirectX software engineer in the 90s who never took physics thought "I'll just do this my own way".

[psnet]

So long thread, want to say that TS is good choice to put sources in and it will be good if backward compatible will be as much as possible, projects using this lib cannot rewrite themselves for new API...

[bnwa]

Is this still alive or is there a notion of an ETA for v4 stable?

[anxpara]

i, too, am wondering what the roadmap looks like right now

[toji]

This is still alive, I've just been swamped with a lot of other priorities and so haven't been able to spend as much time on it as I was hoping. I'm probably going to take several days over the next month or so and dedicate time to just moving this forward, especially given the excellent feedback on this thread and the obvious interest in it.

Thanks for your patience, all!

[toji]

@Swoorup: Is there a way we could also support higher precision? As @ibgreen mentioned earlier, 32-bit precision is bit limiting for my use case as well.

@shannon: Would it make sense to make it configurable to extend Float64Array instead of Float32Array?

I agree that this is an important use case and thanks for reminding me to investigate it! I've been looking into this today and it's unfortunately not as trivial as it was in the prior version of the library if I want to stick to the Typescript inheritance patterns I've been using so far. Fiddled around with Mixins for a bit to try and make it work but it starts to really mess with the type checking and docs generation. I'm starting to think I might just do a really dumb, basic thing and have a build script that just copies the file and does a search and replace Float32Array -> Float64Array. 🙄

In any case, though, I wanted to get opinions about the mechanism for exposing the different precisions. setMatrixArrayType() won't work any more with the patterns I'm using, so I was instead considering exposing the types as Vec3_F32, Vec3_F64, etc internally, and then aliasing Vec3 to Vec3_F32 by default. But you could still change the "default" at the import step like so:

import { Vec3_F64 as Vec3 } from './gl-matrix/vec3.js';

Is that workable for your use cases? (It would also allow for easier mixing of precisions).

@shannon: It looks like the Quat.fromEuler method is missing the order option added in the v3.4.1.

Huh. I have no idea why I missed that. Wasn't intentional. I'll restore it in the next build, thanks for pointing that out!

[shannon]

I agree that this is an important use case and thanks for reminding me to investigate it! I've been looking into this today and it's unfortunately not as trivial as it was in the prior version of the library if I want to stick to the Typescript inheritance patterns I've been using so far. Fiddled around with Mixins for a bit to try and make it work but it starts to really mess with the type checking and docs generation. I'm starting to think I might just do a really dumb, basic thing and have a build script that just copies the file and does a search and replace Float32Array -> Float64Array. 🙄

Yea I really had expected something like this to work:

const Vec2Factory = <T extends Float32ArrayConstructor | Float64ArrayConstructor>(TypedArray: T) => { 
  class Vec2 extends TypedArray {
    // ... snip ...
  }

  // Instance method alias assignments
  // ... snip ...

  // Static method alias assignments
  // ... snip ...
  
  return Vec2;
}

export const Vec2_F32 = Vec2Factory(Float32Array);
export const Vec2_F64 = Vec2Factory(Float64Array);
export const Vec2 = Vec2_F32;
export const vec2 = Vec2_F32;

The interesting bit is that Typescript doesn't present an error with just the above snippet. It just doesn't extend it and add any of the methods (i.e. set) to the class.

The problem I see is that the Float32Array and Float64Array interfaces are incompatible mainly because the type itself is used in a parameter or returned in various methods (i.e. filter). So I created a new FloatArrayInterface generic that accepted Float32Array | Float64Array.

Then I was able to get the following error:

A mixin class must have a constructor with a single rest parameter of type 'any[]'

So with even more fiddling around the conctructor I got this to work but it may seem a bit odd.

import { EPSILON, FloatArray } from './common.js';
import { Mat2Like } from './mat2.js';
import { Mat2dLike } from './mat2d.js';
import { Mat3Like } from './mat3.js';
import { Mat4Like } from './mat4.js';

interface FloatArrayInterface<T extends Float32Array | Float64Array = Float32Array | Float64Array> {
  readonly BYTES_PER_ELEMENT: number;
  readonly buffer: ArrayBufferLike;
  readonly byteLength: number;
  readonly byteOffset: number;

  copyWithin(target: number, start: number, end?: number): this;
  every(predicate: (value: number, index: number, array: T) => unknown, thisArg?: any): boolean;
  fill(value: number, start?: number, end?: number): this;
  filter(predicate: (value: number, index: number, array: T) => any, thisArg?: any): T;
  find(predicate: (value: number, index: number, obj: T) => boolean, thisArg?: any): number | undefined;
  findIndex(predicate: (value: number, index: number, obj: T) => boolean, thisArg?: any): number;
  forEach(callbackfn: (value: number, index: number, array: T) => void, thisArg?: any): void;
  indexOf(searchElement: number, fromIndex?: number): number;
  join(separator?: string): string;
  lastIndexOf(searchElement: number, fromIndex?: number): number;
  readonly length: number;
  map(callbackfn: (value: number, index: number, array: T) => number, thisArg?: any): T;
  reduce(callbackfn: (previousValue: number, currentValue: number, currentIndex: number, array: T) => number): number;
  reduce(callbackfn: (previousValue: number, currentValue: number, currentIndex: number, array: T) => number, initialValue: number): number;
  reduce<U>(callbackfn: (previousValue: U, currentValue: number, currentIndex: number, array: T) => U, initialValue: U): U;
  reduceRight(callbackfn: (previousValue: number, currentValue: number, currentIndex: number, array: T) => number): number;
  reduceRight(callbackfn: (previousValue: number, currentValue: number, currentIndex: number, array: T) => number, initialValue: number): number;
  reduceRight<U>(callbackfn: (previousValue: U, currentValue: number, currentIndex: number, array: T) => U, initialValue: U): U;
  reverse(): T;
  set(array: ArrayLike<number>, offset?: number): void;
  slice(start?: number, end?: number): T;
  some(predicate: (value: number, index: number, array: T) => unknown, thisArg?: any): boolean;
  sort(compareFn?: (a: number, b: number) => number): this;
  subarray(begin?: number, end?: number): T;
  toLocaleString(): string;
  toString(): string;
  valueOf(): T;
  [index: number]: number;
}

type FloatArrayConstructor = { new (...value: any[]): FloatArrayInterface, BYTES_PER_ELEMENT: number };

/**
 * A 2 dimensional vector given as a {@link Vec2}, a 2-element floating point
 * TypedArray, or an array of 2 numbers.
 */
export type Vec2Like = [number, number] | FloatArrayInterface;


function Vec2Factory<T extends FloatArrayConstructor>(TypedArray: T) {
  /**
   * 2 Dimensional Vector
   */
  class Vec2 extends TypedArray {
    /**
     * The number of bytes in a {@link Vec2}.
     */
    static readonly BYTE_LENGTH = 2 * TypedArray.BYTES_PER_ELEMENT;

    /**
     * Create a {@link Vec2}.
     */
    constructor(...values: any[]) {
      switch(values.length) {
        case 2:{
          const v = values[0];
          if (typeof v === 'number') {
            super([v, values[1]]);
          } else {
            super(v as ArrayBufferLike, values[1], 2);
          }
          break;
        }
        case 1: {
          const v = values[0];
          if (typeof v === 'number') {
            super([v, v]);
          } else {
            super(v as ArrayBufferLike, 0, 2);
          }
          break;
        }
        default:
          super(2); break;
      }
    }

    // ... snip ...
  }

  // Instance method alias assignments
  // ... snip ...

  // Static method alias assignments
  // ... snip ...

  return Vec2
}

export const Vec2_F32 = Vec2Factory(Float32Array);
export const Vec2_F64 = Vec2Factory(Float64Array);
export const Vec2 = Vec2_F32;
export const vec2 = Vec2_F32;

The args on the constructor are listed as any type but when you actually try to create a new Vec2 it shows the appropriate types from the super class (ArrayBufferLike), and will error if you try something else.

I think you would probably just replace your FloatArray with the above interface. And then wrap each class in factory as I have done here. I can submit a PR if this makes sense. I honestly don't know as I am not as familiar with Typescript as I am with Javascript.

*Edit: I see now this does mess up the constructor parameters because we had wanted to allow Vec2(1, 2) and this no longer works (it throws Argument of type 'number' is not assignable to parameter of type 'ArrayBufferLike'). I can try to poke around some more but I don't think Typescript is going to allow this mixin. microsoft/TypeScript#37142

[shannon]

Alternatively, if you are ok getting rid of the new keyword on the api side you can do this:

// ... interface snippet from above ...

function Vec2Factory<T extends FloatArrayConstructor>(TypedArray: T) {
  /**
   * 2 Dimensional Vector
   */
  class Vec2 extends TypedArray {
    /**
     * The number of bytes in a {@link Vec2}.
     */
    static readonly BYTE_LENGTH = 2 * TypedArray.BYTES_PER_ELEMENT;

    // ... constructor removed ....

    // ... snip ...
  }

  // Instance method alias assignments
  // ... snip ...

  const factory = (...values: [Readonly<Vec2Like> | ArrayBufferLike, number?] | number[]) => {
    switch(values.length) {
      case 2:{
        const v = values[0];
        if (typeof v === 'number') {
          return new Vec2([v, values[1]]);
        } else {
          return new Vec2(v as ArrayBufferLike, values[1], 2);
        }
      }
      case 1: {
        const v = values[0];
        if (typeof v === 'number') {
          return new Vec2([v, v]);
        } else {
          return new Vec2(v as ArrayBufferLike, 0, 2);
        }
      }
      default:
        return new Vec2(2);
    }
  }

  factory.BYTE_LENGTH = Vec2.BYTE_LENGTH;

  // Static method alias assignments
  factory.sub = Vec2.subtract;
  // ... snip ...

  return factory;
}


export const Vec2_F32 = Vec2Factory(Float32Array);
export const Vec2_F64 = Vec2Factory(Float64Array);

export const Vec2 = Vec2_F32;
export const vec2 = Vec2_F32;

import { Vec2_F64 as Vec2, vec2 } from './gl-matrix/vec2.js';
const a = Vec2(1, 2);
const b = Vec2([1, 2]);
a.add(b);

vec2.sub(a, a, b)

[toji]

I like using new for a couple of reasons. For one, it's using things the "right way", and I find that it's generally better to try to stick with the design patterns as their meant to be used rather than get too cute with JavaScript tricks, in that as browsers evolve they are more likely to optimize the common patterns and more likely to break or deoptimize One Weird Trick-style code.

It also serves as a pretty clear indicator to devs about the type of work being done. new "feels" more expensive than a function call (despite the fact that that's often laughably untrue), and as such provides a mental nudge to use it accordingly.

That said, I'm still open to changing up the patterns here if there's a clear benefit to doing so, but in this case I'm not convinced that that jumping through template hoops is actually going to yield a better, more usable result than just doing the dumb copy thing. (I tried out that route here, we'll see how I feel about it after a few more revisions.)

The problem I see is that the Float32Array and Float64Array interfaces are incompatible mainly because the type itself is used in a parameter or returned in various methods (i.e. filter). So I created a new FloatArrayInterface generic that accepted Float32Array | Float64Array.

I came to the same realization, so I added a type FloatArray = Float32Array | Float64Array that I'm using for all the Vec/Mat*Like types now. It's simpler than what you proposed, though, and there's aspects of the interface direction that you showed that I kind of like, so it's worth experimenting more. I'm also seeing some aspects of the code you posted that are variants on what I was trying that might make a difference, so I'm definitely going to keep toying with it!

Thanks for the feedback!

[toji]

Just published a new beta version to npm for all you fine people to try out!

There are still things I'm investigating, but there was also enough new here that I felt it was worthwhile to push a new version and get feedback. I'm trying to both pull in feedback from this thread and address a variety of longstanding issues files against the library.

Beta.2 Changelog

• Support for Float64Array backing of all types (See README for usage details)
• Added Mat4.normalFromMat4() method (calculates the transpose inverse and drops the translation component)
• Added missing order arg back to Quat.fromEuler()
• Added Mat4.frustumZO, allowed frustum methods to accept an infinite far plane
• Explicitly add | null to return types of methods that may return null. (Such as matrix inversion)
• Cleaned up some temp variable use to enable better tree shaking
• Added toRadian() and toDegree() to common.ts
• Added Vec*.abs() static and instance method to all vectors
• Cleaned up some comment typos
• Added main, module, exports, and types to package.json

[christjt]

Oh no, if WebGPU uses left-handed, I blame you, @toji 😝

Left-handed matrix math should have been expunged from the world years ago... In fact it had been until some DirectX software engineer in the 90s who never took physics thought "I'll just do this my own way".

WebGPU doesn't make any assumptions of handedness that your app uses. Typically you just define whatever coordinate system you want, and then finally apply the projection matrix which also bakes in the transformation from the coordinate system used by your app to NDC, which in WebGPUs case is indeed left handed. But absolutely nothing is stopping you from writing your entire app in a right-handed coordinate system of your choice.

@toji As WebGPU has z defined from 0..1 in NDC, it also opens up the possibility to reverse z order, which I assume a lot of us will be doing.

I am a little torn here on how much I think gl-matrix should assist in enabling this because it is simply a matter of baking in another transform into the projection matrix produced by gl-matrix. I think the current solution is fine, and it is ok to be opinionated and not expand the API to include handedness when creating the perspective matrix, but maybe it should be documented? Also, if a lot of users struggle with this, then maybe we could add some convinience methods to Mat4 to convert to and from other coordinate systems if we can find an API that makes sense. I am sure you have considered this though, what do you think?