Draft: MESHOPT_compression

[zeux]

This is a draft of MESHOPT_compression extension. I'm sharing this early to get feedback that might drastically change the JSON structure here or get extra input on the rest of the proposal.

Rendered version (Updated 4/15/2020)

This extension is used by gltfpack to implement a substantially different type of compressor vs existing offering. Some of this is covered in the overview, but briefly:

• Compression is performed on a buffer view level - this makes it easy to support all types of glTF data (this includes geometry, morph targets and animation) with minimal JSON complexity
• Compressed buffer views can be decompressed directly into GPU/CPU target memory - this makes decoding faster and makes implementations of this extension very simple, here's a Babylon.JS example https://github.com/zeux/meshoptimizer/blob/master/demo/babylon.MESHOPT_compression.js
• Decompression is very fast - with WASM SIMD it runs at ~1 GB/s on modern desktop CPUs. Native implementations run at ~2+ GB/s.
• Decompression code is small. WASM decoder is 5-7 KB binary; two WASM decoders bundled into a .js file and gzip-compressed are a 7 KB download.
• Encoding is gzip-friendly. The compression method provides a moderate size gain over uncompressed storage, with further size gains possible when the binary is gzip-compressed.

Quick examples: (please note that this just two files, I will need to build a larger comparison table later)

BrainStem.glb, original: 3.1 MB
BrainStem.glb, quantized: 1.1 MB (this also does other types of optimization on the file)
BrainStem.glb, quantized + deflate: 560 KB
BrainStem.glb, with this extension: 440 KB
BrainStem.glb, with this extension + deflate: 330 KB
BrainStem.glb, with Draco: 1.4 MB (animations aren't compressed, unfair example)

Corset.glb, original: 660 KB
Corset.glb, quantized: 340 KB
Corset.glb, quantized + deflate: 180 KB
Corset.glb, with this extension: 145 KB
Corset.glb, with this extension + deflate: 103 KB
Corset.glb, Draco: 112 KB

The BrainStem model above has animation data; the Corset.glb is just a static mesh but it has PBR-ready data set. In my experience, Draco tends to win noticeably on very large meshes with just position + normal data - I'm working on making my codec stronger in this case.

However, all comparisons above are unfair in one way or another to Draco - for example I am not matching quantization settings here and using whatever Draco model comes with glTF-Sample-Models - and in general Draco will win vs this extension in terms of pure size comparison when Draco fully supports the models. So the examples above are intended to highlight the wins that come from this extension, and Draco numbers are just ballpark because I know the question will come up. This extension is designed to complement Draco by providing a full compression solution aimed at high decoding speeds and specification simplicity, not to replace Draco fully in cases where mesh compression ratio is the only important criteria.

Note that the details of the bitstream are still in flux. It is my intention to update this extension with the bitstream encoding as it gets finalized; alternatively, the bitstream can be hosted in a separate document. It's much simpler than Draco's but a formal definition may still take significant space? Unsure.

I am hoping to be able to finalize the bitstream details either by the end of the year, or early next year. Any and all feedback welcome.

[donmccurdy]

Consider MESHOPT_mesh_compression. 😇

[zeux]

Note that this extension is more general and allows compressing animation data as well. Maybe view_compression would be appropriate?

[donmccurdy]

Ah, good point. In that case view_compression sounds reasonable, or I would be open to considering this an exception to the <scope>_<feature> guideline too.

[donmccurdy]

I'm sharing this early to get feedback that might drastically change the JSON structure here or get extra input on the rest of the proposal.

The JSON structure looks good to me, given the context and longer discussion in #1684. I'll probably have comments on various details later, but nothing that seems important yet.

It's much simpler than Draco's but a formal definition may still take significant space? Unsure.

I would suggest an appendix in README.md for now, but a separate document would also be fine.

Quick examples: (please note that this just two files, I will need to build a larger comparison table later)

A larger comparison table would be very helpful — would it be possible to include decode times as well?

[zeux]

A larger comparison table would be very helpful — would it be possible to include decode times as well?

Yeah, sure thing. Brief non-scientific experiment (will include decode numbers in the future) on Buggy.gltf (500K triangles, 400K vertices):

• Draco: 190ms
• meshopt: decodeVertexBuffer 15 ms + decodeIndexBuffer 4.5 ms = 20ms
• meshopt simd: decodeVertexBuffer 9 ms + decodeIndexBuffer 4.5 ms = 15ms (normally SIMD version is ~3x faster but the perf increase on the single small mesh load is generally smaller... need to profile streaming scenario where meshes are being continuously decoded)

On models of a massive scale the difference can be more pronounced especially with the SIMD variant, here's Thai Buddha statue (https://sketchfab.com/3d-models/thai-buddha-cba029e262bd4f22a7ee4fcf064e22ee, 6M triangles / 3M vertices):

• Draco: 9720 ms
• meshopt: 181ms vertex + 90ms index = 270ms
• meshopt simd: 72ms vertex + 92ms index = 165ms

"simd" above refers to WASM SIMD variant that is supported on Chrome Canary when experimental SIMD support is enabled.

[zeux]

I've updated the extension spec to the latest version. Will work on bitstream specification in the next few days; comments welcome otherwise. We'll probably need a better extension name but that can wait.

The new changes added since last update make the extension more competitive from the transmission size perspective. Here's a comparison table in terms of encoded size (see performance numbers above, the ballpark is about the same) for models from glTF-Sample-Models, excluding texture data:

2CylinderEngine                          gltfpack 496864               gltfpack.gz 261707               draco 499784               draco.gz 402837               gltfpack/draco 0.994                gltfpack.gz/draco.gz 0.650
AntiqueCamera                            gltfpack 166612               gltfpack.gz 115040               draco 130240               draco.gz 114831               gltfpack/draco 1.279                gltfpack.gz/draco.gz 1.002
Avocado                                  gltfpack 8232                 gltfpack.gz 5705                 draco 6616                 draco.gz 5300                 gltfpack/draco 1.244                gltfpack.gz/draco.gz 1.076
BarramundiFish                           gltfpack 29924                gltfpack.gz 23982                draco 23368                draco.gz 21069                gltfpack/draco 1.281                gltfpack.gz/draco.gz 1.138
BoomBox                                  gltfpack 47012                gltfpack.gz 37594                draco 37128                draco.gz 34118                gltfpack/draco 1.266                gltfpack.gz/draco.gz 1.102
BrainStem                                gltfpack 370512               gltfpack.gz 256496               draco 380872               draco.gz 247774               gltfpack/draco 0.973                gltfpack.gz/draco.gz 1.035
Buggy                                    gltfpack 2230596              gltfpack.gz 1047707              draco 2733216              draco.gz 2243675              gltfpack/draco 0.816                gltfpack.gz/draco.gz 0.467
CesiumMan                                gltfpack 50356                gltfpack.gz 34063                draco 45860                draco.gz 36237                gltfpack/draco 1.098                gltfpack.gz/draco.gz 0.940
CesiumMilkTruck                          gltfpack 29664                gltfpack.gz 15927                draco 29640                draco.gz 16732                gltfpack/draco 1.001                gltfpack.gz/draco.gz 0.952
Corset                                   gltfpack 133356               gltfpack.gz 85150                draco 113724               draco.gz 105913               gltfpack/draco 1.173                gltfpack.gz/draco.gz 0.804
DamagedHelmet                            gltfpack 136528               gltfpack.gz 105357               draco 102016               draco.gz 91620                gltfpack/draco 1.338                gltfpack.gz/draco.gz 1.150
Duck                                     gltfpack 27636                gltfpack.gz 23031                draco 21096                draco.gz 19471                gltfpack/draco 1.310                gltfpack.gz/draco.gz 1.183
FlightHelmet                             gltfpack 685856               gltfpack.gz 498758               draco 519572               draco.gz 473156               gltfpack/draco 1.320                gltfpack.gz/draco.gz 1.054
GearboxAssy                              gltfpack 2209512              gltfpack.gz 1271787              draco 2212596              draco.gz 1845696              gltfpack/draco 0.999                gltfpack.gz/draco.gz 0.689
Lantern                                  gltfpack 46200                gltfpack.gz 29917                draco 35548                draco.gz 32146                gltfpack/draco 1.300                gltfpack.gz/draco.gz 0.931
Monster                                  gltfpack 40800                gltfpack.gz 23526                draco 53216                draco.gz 31308                gltfpack/draco 0.767                gltfpack.gz/draco.gz 0.751
ReciprocatingSaw                         gltfpack 804880               gltfpack.gz 457507               draco 869368               draco.gz 702841               gltfpack/draco 0.926                gltfpack.gz/draco.gz 0.651
RiggedFigure                             gltfpack 12744                gltfpack.gz 6973                 draco 10728                draco.gz 6538                 gltfpack/draco 1.188                gltfpack.gz/draco.gz 1.067
SciFiHelmet                              gltfpack 670708               gltfpack.gz 537936               draco 182420               draco.gz 173450               gltfpack/draco 3.677                gltfpack.gz/draco.gz 3.101
Sponza                                   gltfpack 1970516              gltfpack.gz 843461               draco 1678516              draco.gz 1453378              gltfpack/draco 1.174                gltfpack.gz/draco.gz 0.580
Suzanne                                  gltfpack 23588                gltfpack.gz 18963                draco 17912                draco.gz 16285                gltfpack/draco 1.317                gltfpack.gz/draco.gz 1.164
WaterBottle                              gltfpack 32952                gltfpack.gz 20436                draco 24384                draco.gz 21490                gltfpack/draco 1.351                gltfpack.gz/draco.gz 0.951

These were generated as follows:

gltfpack -i scene.gltf -o scene_noq.glb -noq
gltfpack -i scene_noq.glb -o scene_glp.glb -cc
gzip -9 -k scene_glp.glb
gltf-pipeline -i scene_noq.glb -o scene_drc.glb -d -t --draco.quantizeNormalBits 8
gzip -9 -k scene_drc.glb

The use of noq (no-quantization) packed GLB as a starting point is important since it establishes a level playing field - gltfpack merges the meshes and filters out redundant attributes/triangles/etc. as a result, so that Draco compression can work off the same data set.

From the numbers above it seems like this extension is an excellent all-rounder alternative to Draco. With recent additions it supports all glTF features (e.g. now supports efficient encoding of line lists and other topologies), and is future-proof - for example, the parts of this extension that allow compressing animation data (notably the quaternion/exponential filters) apply in the exact same way to EXT_mesh_gpu_instancing without any schema changes because of the orthogonal extension design.

My goal over the next week is to document the bitstream and filters, correct any inconsistencies that arise, and possibly add a color filter which I haven't done in the specification yet. Any and all feedback is welcome.

[zeux]

The spec has been updated with detailed documentation on encoding for all modes and filters.

[donmccurdy]

Thanks for the detailed numbers here! For easier review I've formatted them in a sheet.

three.js will very likely support this extension: the improved decoding speed certainly fills a gap that our users have repeatedly reported. By that I mean, in addition to it being possible to implement the extension externally with the upcoming extension API, I think we'd support including the extension in the repo and npm package if that's of interest.

---

A couple minor spec comments. So far, I think all enum values in the glTF spec fall into one of two cases:

1. GL enum value, integer.
2. glTF-specific enum value, string.

From that perspective, the .mode and .filter enums would be more consistent with the rest of the spec if they were given string types. Am I correct in understanding that the value of .mode is not closely associated with bufferView.target? E.g. use of the attribute mode does not necessarily imply bufferView.target === gl.ARRAY_BUFFER?

[zeux]

So far, I think all enum values in the glTF spec fall into one of two cases:

Primitive mode is a small integer, which was part of the motivation here, another part being a slight reduction in glTF size. It's not particularly meaningful so this could be changed to a string.

edit ah I just realized after writing this that primitive mode happens to match GL values that instead of using larger 2-byte values use small integers there.

.g. use of the attribute mode does not necessarily imply bufferView.target === gl.ARRAY_BUFFER?

Yeah that's correct. attribute mode is useful when compressing animation data as well as instance transform data, neither of which would have ARRAY_BUFFER target.

[zeux]

if they were given string types

Is upper case or lower case more consistent with recent additions? It seems that upper case is perhaps a touch more prevalent, but e.g. camera types and node paths use lower case.

[donmccurdy]

Light types are lowercase as well; I think that's the only extension to define an enum-like property since the core spec. @lexaknyazev you'd mentioned some opinions on this before (#1560 (comment)), do you have a preference moving forward?

[zeux]

Separate question: I'd like to rename this to EXT-prefixed extension as it makes more sense to do so - the encoding format/support comes from meshoptimizer/gltfpack but with the spec and existing implementations for three.js and Babylon.JS (both are currently within meshoptimizer repo but can be contributed upstream once this extension lands) this seems to be a cross-vendor extension.

With that in mind, what's a good name? Options OTOH

• EXT_compression_meshopt
• EXT_meshopt_compression // similar to KHR_draco_mesh_compression
• EXT_buffer_compression_meshopt // imprecise - extension works on buffer views - but maybe good enough?
• EXT_view_compression_meshopt // imprecise but I don't like buffer_view prefix as it's really long and two-word...

Other variants?

[donmccurdy]

If the implementations will land upstream — i.e. be accepted by the three.js and/or babylon.js projects — then I agree that the EXT_ prefix is appropriate. I'm optimistic about three.js given the compression numbers here, but can't answer for the project just yet. To clarify for future readers (I think this is what you're aiming for anyway, of course) I believe the expectation for a multi-vendor extension is that it have the support of multiple vendors, not that a single vendor provide third-party plugins to multiple vendors' tools.

On names, I'd be tempted to use bufferview rather than buffer_view, I dislike the two-words part much more than I dislike the length. In that vein, perhaps one of:

• EXT_meshopt_bufferview_compression
  • Maybe a bit long, but consistent with Draco and arguably¹ consistent with the naming guidelines.
• EXT_bufferview_meshopt_compression
  • Maybe a bit long, but consistent with the naming guidelines. Not very consistent with Draco.
• EXT_meshopt_compression
  • Not quite in line with the naming guideline, but I'd be OK with an exception given the broad scope of the extension and the relative obscurity of the 'bufferview' concept. Consistent with Draco.

¹ I say "arguably" because it somewhat resembles the double prefixes used in WebGL, like WEBKIT_OES_vertex_array_object. That isn't really covered by our naming guidelines today.

[zeux]

I believe the expectation for a multi-vendor extension is that it have the support of multiple vendors, not that a single vendor provide third-party plugins to multiple vendors' tools.

Right, that's what I meant, although this kinda goes both ways - if this extension is EXT_ it's more likely to be upstreamable, and if this isn't it's less likely.

My main motivation for EXT_ is that this extension fills large gaps in functionality. Without it, entire classes of valid glTF models don't have a valid compression solution in glTF ecosystem. (already mentioned earlier, but this includes point clouds, lines, morph targets for geometry, and animation data)

The multi-vendor implementation is, I suppose, more of a consequence.

I like EXT_meshopt_compression the most, since it doesn't use the bufferview (which is obscure indeed... all other prefixes read like "extension provides extra data for X", but here bufferView is an implementation detail so to speak, the actual goal is making files smaller), and is in line with Draco naming (Draco has mesh but it only works on meshes...)

[bghgary]

+1 to EXT_meshopt_compression

It might be just me, but I have an aversion to bufferview with a lowercase v.

[zeux]

EXT_meshopt_compression it is. Renaming is a good opportunity to make the other incompatible changes here, I'm considering:

• Removing count property as it's redundant and can be computed from parent buffer view length. I believe I included it to try to keep the extension blob self-sufficient but it doesn't seem like a strong enough reason, since for validation purposes parent length would need to be checked anyway.
• Changing the enums to strings, if this is the preferred direction. It might make sense to keep the indexing in the bitstream spec (e.g. "mode 0") in case a future glTF version has dual indexed & named enums.

The open questions are:

• Should enums be changed to strings?
• Should strings be lower or upper case?

Unsure how to answer these atm.

With some other extensions, I've seen renaming done in-place by editing the PR, or renaming lead to opening a new PR. Which one makes more sense in this case, given that I'd like to move this from "draft" to an actual proposal?

[donmccurdy]

With some other extensions, I've seen renaming done in-place by editing the PR, or renaming lead to opening a new PR.

I think this is up to the preference of the author. When a PR is created in GitHub's web UI it might be easier to restart than to rename folders. It's fine to edit the PR, though.

• Should enums be changed to strings?
• Should strings be lower or upper case?

I think my preference is #1702 (comment) but I'm hoping others may have thoughts on this... I think it's less a question of a "right" answer than of having a rough guideline for the future. @emackey @bghgary @lexaknyazev ?

[emackey]

I don't think I have a strong opinion. The core enums are all legacy from when glTF 1.0 and prior were targeting the WebGL API directly. Was there ever a reason to prevent non-WebGL enums once that link was broken in 2.0? I'm not sure.

In general, consistency is good for the ecosystem. So that would suggest either following the pattern of previous extensions, or explicitly establishing a better pattern for future extensions.

[lexaknyazev]

The existing suboptimal situation reflects three possible enum sources (it's all glTF 1.0 legacy):

• Integers for values compatible with GL entrypoints.
• Lowercase strings for values that could be used as JSON keys in other places (see camera.type / camera.perspective, same for punctual lights).
• Uppercase strings otherwise.

To follow the existing design, this extension's JSON should use uppercase strings. Bitstream is not affected by any of this, of course.

I'd refrain from "establishing a better pattern for future extensions" unless we're talking about glTF Next.

[zeux]

@lexaknyazev Thanks! The distinction is clear, I'm going to amend this to use upper-case strings then.

[zeux]

Since the numbers in the initial post for this PR are out of date and the extension name is going to change, I'm going to close this PR and open a new PR with up-to-date information and new name/spec.