EXT_meshopt_compression extension

[zeux]

Note: this was originally filed as #1702 but was since renamed to EXT_meshopt_compression, hence the new PR

Rendered version (Updated 10/2/2020)

This is an extension designed to reduce the transmission size of glTF files. The structure of this extension is very different from KHR_draco_mesh_compression, as it works on a per-bufferView basis instead of a per-mesh basis - with all of the remaining glTF schema in tact.

During loading of a compressed glTF file, loaders are expected to decompress the compressed bufferView data and then proceed with the loading as usual - this, for example, is compatible with decoding the buffer views directly into GPU-visible buffers.

Any type of data can be compressed; however, this extension isn't exactly a general purpose compressor. Instead, there are several algorithms tailored to different kinds of data that may be stored inside the buffer view - for all of the algorithms, they are designed to provide extremely fast decoding (on modern desktop CPUs decoding runs at ~2+ GB/s for native code and at ~1 GB/s in Wasm, using Wasm SIMD to accelerate parts of the processing). The decoders are implemented in meshoptimizer (https://github.com/zeux/meshoptimizer), and files compressed with this extension can be produced by gltfpack (https://github.com/zeux/meshoptimizer/tree/master/gltf) - or, of course, any other tool that is compliant with this specification.

For each bufferView, an appropriate compression technique must be picked to maximize the compression ratio of the data. The extension provides a compression mode for attribute data (suitable for mesh attributes, animation keys or values, instance transform components), triangle indices (suitable for mesh index data when using triangle list primitive) and indices (suitable for mesh index data for other primitives as well as sparse indices for general accessor storage). It's the job of the encoder to split the compressible data into bufferViews as necessary; additionally, for all compression modes preparing data for compression is important to get high compression ratios - this includes finding the optimal order of the data elements, quantizing them with KHR_mesh_quantization, and possibly other kinds of preprocessing such as animation data resampling.

Additionally, for attribute storage (for mesh/animation/etc.), the encoder may decide to use compression filters. These provide extra savings on top of attribute compression provided by this extension at the cost of extra precision loss; for example, instead of storing quantized quaternion values, filters allow storing only 3 components of a quaternion and reconstructing the quaternion from that - this can result in small precision loss. All filters are designed to be variable bit rate - that is, the encoder can pick the optimal number of bits used for data encoded by the filters, and fewer bits of data used will result in the attribute compression using fewer bytes to represent the data stream.

Unlike Draco or Basis ETC1 that have capable entropy coders embedded into the format, this extension doesn't employ Huffman or rANS or similar entropy coders; instead all algorithms are designed to reduce the data size as much as possible while still representing the data as a byte sequence; the expectation is that for maximum compression ratio a general-purpose compressor such as gzip (which is commonplace for asset delivery on the web) can be used as well. Noteworthy is that this extension acts as a pre-processor for gzip/et al in that by itself, gzip can't get anywhere close to the level of compression this extension provides; so the expectation is that this extension can compress data, gzip can compress the resulting compressed result further, but gzip or an equivalent compressor is optional.

For mesh data, this extension usually is slightly less efficient than Draco in terms of the transmission size. However, it's completely general (Draco glTF can't compress point clouds, non-triangle geometry and morph targets), and supports mesh, animation, instance data and general index/attribute compression with a simple specification.

[zeux]

In terms of compression ratio, here are the results of compressing data using this extension compared to using Draco:

2CylinderEngine                          gltfpack 496864               gltfpack.gz 261707               draco 499784               draco.gz 402837               gltfpack/draco 0.994                gltfpack.gz/draco.gz 0.650
AntiqueCamera                            gltfpack 166612               gltfpack.gz 115040               draco 130240               draco.gz 114831               gltfpack/draco 1.279                gltfpack.gz/draco.gz 1.002
Avocado                                  gltfpack 8232                 gltfpack.gz 5705                 draco 6616                 draco.gz 5300                 gltfpack/draco 1.244                gltfpack.gz/draco.gz 1.076
BarramundiFish                           gltfpack 29924                gltfpack.gz 23982                draco 23368                draco.gz 21069                gltfpack/draco 1.281                gltfpack.gz/draco.gz 1.138
BoomBox                                  gltfpack 47012                gltfpack.gz 37594                draco 37128                draco.gz 34118                gltfpack/draco 1.266                gltfpack.gz/draco.gz 1.102
BrainStem                                gltfpack 370512               gltfpack.gz 256496               draco 380872               draco.gz 247774               gltfpack/draco 0.973                gltfpack.gz/draco.gz 1.035
Buggy                                    gltfpack 2230596              gltfpack.gz 1047707              draco 2733216              draco.gz 2243675              gltfpack/draco 0.816                gltfpack.gz/draco.gz 0.467
CesiumMan                                gltfpack 50356                gltfpack.gz 34063                draco 45860                draco.gz 36237                gltfpack/draco 1.098                gltfpack.gz/draco.gz 0.940
CesiumMilkTruck                          gltfpack 29664                gltfpack.gz 15927                draco 29640                draco.gz 16732                gltfpack/draco 1.001                gltfpack.gz/draco.gz 0.952
Corset                                   gltfpack 133356               gltfpack.gz 85150                draco 113724               draco.gz 105913               gltfpack/draco 1.173                gltfpack.gz/draco.gz 0.804
DamagedHelmet                            gltfpack 136528               gltfpack.gz 105357               draco 102016               draco.gz 91620                gltfpack/draco 1.338                gltfpack.gz/draco.gz 1.150
Duck                                     gltfpack 27636                gltfpack.gz 23031                draco 21096                draco.gz 19471                gltfpack/draco 1.310                gltfpack.gz/draco.gz 1.183
FlightHelmet                             gltfpack 685856               gltfpack.gz 498758               draco 519572               draco.gz 473156               gltfpack/draco 1.320                gltfpack.gz/draco.gz 1.054
GearboxAssy                              gltfpack 2209512              gltfpack.gz 1271787              draco 2212596              draco.gz 1845696              gltfpack/draco 0.999                gltfpack.gz/draco.gz 0.689
Lantern                                  gltfpack 46200                gltfpack.gz 29917                draco 35548                draco.gz 32146                gltfpack/draco 1.300                gltfpack.gz/draco.gz 0.931
Monster                                  gltfpack 40800                gltfpack.gz 23526                draco 53216                draco.gz 31308                gltfpack/draco 0.767                gltfpack.gz/draco.gz 0.751
ReciprocatingSaw                         gltfpack 804880               gltfpack.gz 457507               draco 869368               draco.gz 702841               gltfpack/draco 0.926                gltfpack.gz/draco.gz 0.651
RiggedFigure                             gltfpack 12744                gltfpack.gz 6973                 draco 10728                draco.gz 6538                 gltfpack/draco 1.188                gltfpack.gz/draco.gz 1.067
SciFiHelmet                              gltfpack 670708               gltfpack.gz 537936               draco 182420               draco.gz 173450               gltfpack/draco 3.677                gltfpack.gz/draco.gz 3.101
Sponza                                   gltfpack 1970516              gltfpack.gz 843461               draco 1678516              draco.gz 1453378              gltfpack/draco 1.174                gltfpack.gz/draco.gz 0.580
Suzanne                                  gltfpack 23588                gltfpack.gz 18963                draco 17912                draco.gz 16285                gltfpack/draco 1.317                gltfpack.gz/draco.gz 1.164
WaterBottle                              gltfpack 32952                gltfpack.gz 20436                draco 24384                draco.gz 21490                gltfpack/draco 1.351                gltfpack.gz/draco.gz 0.951

These were generated as follows:

gltfpack -i scene.gltf -o scene_noq.glb -noq
gltfpack -i scene_noq.glb -o scene_glp.glb -cc
gzip -9 -k scene_glp.glb
gltf-pipeline -i scene_noq.glb -o scene_drc.glb -d -t --draco.quantizeNormalBits 8
gzip -9 -k scene_drc.glb

Using gltfpack as a pre-processor in addition to Draco is important to establish a level playing field, as gltfpack does a lot of high level scene processing as well.

Results as a Google Spreadsheet: https://docs.google.com/spreadsheets/d/1V0jls9QSb7DRE3-JseCHRcnAzIAp-uSBssnEvx2dQxI/edit#gid=0

[zeux]

In terms of decompression time, this tends to vary a bit between the modes and filters used. Using two models as a test, Buggy.gltf from glTF-Sample-Models (300K triangles, 245K vertices) and Thai Buddha (https://sketchfab.com/3d-models/thai-buddha-cba029e262bd4f22a7ee4fcf064e22ee, 6M triangles, 3M vertices), we get the following timings on Chrome Canary on i7 8700K:

Buggy.gltf:
meshopt: decodeVertexBuffer 4.5 ms, decodeIndexBuffer 2.7 ms, total ~7.5ms
Draco: total 186 ms (25x slower)

Thai Buddha:
meshopt: decodeVertexBuffer 78ms, decodeIndexBuffer 118ms, total 196ms
Draco: 6.19s DecodeBufferToMesh, 7.63s total (39x slower)

Based on the size and performance metrics, it looks like while Draco still is a viable alternative in cases when download size is critical and the content consists purely of elements that Draco supports well, this extension may provide a better balance and supports compression for all non-texture data that can be encoded in glTF.

[zeux]

@donmccurdy @lexaknyazev JFYI I've resubmitted this with the new extension name and the enum changes we discussed on the previous PR. Also all numbers in this PR are now current (old PR had some numbers from earlier implementation that was less efficient). This PR should now be the central one, please let me know if you have further feedback! (also please let me know if you'd like me to add you to the contributor list, I don't know how that works)

[donmccurdy]

The specification seems clear to me, and (not that this is required) I'm very happy with the performance and compression upsides here. I don't think I'm able to usefully review the bitstream, but I'm happy to merge this if others agree.

Aside, I don't think I'd realized the meshopt WASM decoder was only 6kb gzipped. That's easily the smallest useful WASM library I've ever seen, so thank you and great work on that.

[bghgary]

Seems like a good plan. 👍

[zeux]

@lexaknyazev Thanks a lot for the feedback, I believe I've addressed it (mostly tried to use one commit per comment for easier diff review)

[lexaknyazev]

@zeux
Sorry for all the delays with reviewing, this seems to be ready to go now. Feel free to open another PR if/when the status should be updated from "Draft" to "Complete".

[zeux]

@lexaknyazev No worries, thanks a lot for the help!